Tractable ADMM schemes for computing KKT points and local minimizers for $$\ell _0$$ ℓ 0 -minimization problems

Yue Xie () and Uday V. Shanbhag ()
Additional contact information
Yue Xie: Wisconsin Institute for Discovery
Uday V. Shanbhag: Pennsylvania State University

Computational Optimization and Applications, 2021, vol. 78, issue 1, No 2, 43-85

Abstract: Abstract We consider an $$\ell _0$$ ℓ 0 -minimization problem where $$f(x) + \gamma \Vert x\Vert _0$$ f ( x ) + γ ‖ x ‖ 0 is minimized over a polyhedral set and the $$\ell _0$$ ℓ 0 -norm regularizer implicitly emphasizes the sparsity of the solution. Such a setting captures a range of problems in image processing and statistical learning. Given the nonconvex and discontinuous nature of this norm, convex regularizers as substitutes are often employed and studied, but less is known about directly solving the $$\ell _0$$ ℓ 0 -minimization problem. Inspired by Feng et al. (Pac J Optim 14:273–305, 2018), we consider resolving an equivalent formulation of the $$\ell _0$$ ℓ 0 -minimization problem as a mathematical program with complementarity constraints (MPCC) and make the following contributions towards the characterization and computation of its KKT points: (i) First, we show that feasible points of this formulation satisfy the relatively weak Guignard constraint qualification. Furthermore, if f is convex, an equivalence is derived between first-order KKT points and local minimizers of the MPCC formulation. (ii) Next, we apply two alternating direction method of multiplier (ADMM) algorithms, named (ADMM $$_{\mathrm{cf}}^{\mu , \alpha , \rho }$$ cf μ , α , ρ ) and (ADMM $$_{\mathrm{cf}}$$ cf ), to exploit the special structure of the MPCC formulation. Both schemes feature tractable subproblems. Specifically, in spite of the overall nonconvexity, it is shown that the first update can be effectively reduced to a closed-form expression by recognizing a hidden convexity property while the second necessitates solving a tractable convex program. In (ADMM $$_{\mathrm{cf}}^{\mu , \alpha , \rho }$$ cf μ , α , ρ ), subsequential convergence to a perturbed KKT point under mild assumptions is proved. Preliminary numerical experiments suggest that the proposed tractable ADMM schemes are more scalable than their standard counterpart while (ADMM $$_{\mathrm{cf}}$$ cf ) compares well with its competitors in solving the $$\ell _0$$ ℓ 0 -minimization problem.

Keywords: Nonconvex sparse recovery; Constraint qualifications and KKT conditions; Nonconvex ADMM; Tractability; Convergence analysis (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10589-020-00227-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:coopap:v:78:y:2021:i:1:d:10.1007_s10589-020-00227-6

Ordering information: This journal article can be ordered from
http://www.springer.com/math/journal/10589

DOI: 10.1007/s10589-020-00227-6

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().