Stochastic gradient langevin dynamics for (weakly) log-concave posterior distributions

Marelys Crespo (), Sébastien Gadat () and Xavier Gendre ()
Additional contact information
Marelys Crespo: TSE-R - Toulouse School of Economics - UT Capitole - Université Toulouse Capitole - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement
Sébastien Gadat: TSE-R - Toulouse School of Economics - UT Capitole - Université Toulouse Capitole - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement
Xavier Gendre: IMT - Institut de Mathématiques de Toulouse UMR5219 - UT Capitole - Université Toulouse Capitole - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - INSA Toulouse - Institut National des Sciences Appliquées - Toulouse - INSA - Institut National des Sciences Appliquées - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - UT2J - Université Toulouse - Jean Jaurès - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - UT3 - Université Toulouse III - Paul Sabatier - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - CNRS - Centre National de la Recherche Scientifique

Post-Print from HAL

Abstract: In this paper, we investigate a continuous time version of the Stochastic Langevin Monte Carlo method, introduced in [39], that incorporates a stochastic sampling step inside the traditional overdamped Langevin diffusion. This method is popular in machine learning for sampling posterior distribution. We will pay specific attention in our work to the computational cost in terms of n (the number of observations that produces the posterior distribution), and d (the dimension of the ambient space where the parameter of interest is living). We derive our analysis in the weakly convex framework, which is parameterized with the help of the Kurdyka- Lojasiewicz (KL) inequality, that permits to handle a vanishing curvature settings, which is far less restrictive when compared to the simple strongly convex case. We establish that the final horizon of simulation to obtain an ε approximation (in terms of entropy) is of the order (d log(n)²)(1+r)² [log²(ε−1) + n²d²(1+r) log4(1+r)(n)] with a Poissonian subsampling of parameter n(d log²(n))1+r)−1, where the parameter r is involved in the KL inequality and varies between 0 (strongly convex case) and 1 (limiting Laplace situation).

Keywords: Log-concave models; Stochastic gradient Langevin dynamics; Weak convexity (search for similar items in EconPapers)
Date: 2024
Note: View the original document on HAL open archive server: https://hal.science/hal-04943092v1
References: Add references at CitEc
Citations:

Published in Electronic Journal of Probability, 2024, Vol. 29, pp.1-40. ⟨10.1214/24-EJP1235⟩

Downloads: (external link)
https://hal.science/hal-04943092v1/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-04943092

DOI: 10.1214/24-EJP1235

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().