Hybrid Random Concentrated Optimization Without Convexity Assumption

Pierre Bertrand (), Michel Broniatowski and Wolfgang Stummer
Additional contact information
Pierre Bertrand: Aix-Marseille Univ., CNRS, AMSE, Marseille, France, https://www.amse-aixmarseille.fr/en/members/bertrandp
Michel Broniatowski: LPSM, Sorbonne University, Paris, France
Wolfgang Stummer: Maths Department, University of Erlangen-N¨urnberg (FAU), Erlangen, Germany

No 2524, AMSE Working Papers from Aix-Marseille School of Economics, France

Abstract: We propose a new random method to minimize deterministic continuous functions over subsetsSof high-dimensional spaceR K without assuming convexity. Our procedure alternates between a Global Search(GS) regime to identify candidates and a Concentrated Search (CS) regime to improve an eligible candidate in the constraint setS. Beyond the alternation between those completely different regimes, the originality of our approach lies in leveraging high dimensionality. We demonstrate rigorous concentration properties under theCSregime. In parallel, we also show thatGSreaches any point inSin finite time. Finally, we demonstrate the relevance of our new method by giving two concrete applications. The first deals with the reduction of theℓ1−norm of a LASSO solution. Secondly, we compress a neural network by pruning weights while maintaining performance; our approach achieves significant weight reduction with minimal performance loss, offering an effective solution for network optimization.

Keywords: High-dimensional optimization; Stochastic search; LASSO; Basis pursuit denoising; Neural network compression (search for similar items in EconPapers)
Date: 2025-12
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.amse-aixmarseille.fr/sites/default/fil ... rs/wp_2025_nr_24.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aim:wpaimx:2524

Access Statistics for this paper

More papers in AMSE Working Papers from Aix-Marseille School of Economics, France AMU-AMSE - 5-9 Boulevard Maurice Bourdet, CS 50498 - 13205 Marseille Cedex 1. Contact information at EDIRC.
Bibliographic data for series maintained by Gregory Cornu ().