Generalized Optimization Algorithms for Complex Objective Functions

Mario Martinoli, Raffaello Seri and Fulvio Corsi

LEM Papers Series from Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy

Abstract: The paper pursues two goals. First, by linking the statistics and machine learning literatures, we provide new general results on the convergence of stochastic approximation schemes and inexact Newton methods. Second, by building on these results, we put forward a new optimization scheme that we call local approximation inexact Newton method (LAINM). For each iteration of the optimization algorithm, the method proceeds through these steps: the objective function is computed or approximated in a neighborhood of the iterate; the value of the function, of its gradient and of its Hessian are approximated through a polynomial regression; these quantities are then used in a Newton-like iteration. We extensively discuss the theoretical and the computational aspects of LAINM. The results apply to both deterministic and stochastic approximation schemes, and are particular effective in the case in which the objective function to be optimized is highly irregular and/or the stochastic equicontinuity hypothesis, that is generally assumed in simulation-based estimation, is violated. Examples are common in dynamic discrete choice models (DDCM) and complex simulation models characterized by nonlinearities and high levels of heterogeneity. The theory is supported by extensive Monte Carlo experiments and an application to a binary response model with serially correlated errors.

Keywords: Optimization; approximation algorithms; stochastic approximation; methods of quasi-Newton type (search for similar items in EconPapers)
Date: 2024-07-23
New Economics Papers: this item is included in nep-cmp and nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.lem.sssup.it/WPLem/files/2024-18.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ssa:lemwps:2024/18

Access Statistics for this paper

More papers in LEM Papers Series from Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy Contact information at EDIRC.
Bibliographic data for series maintained by ( this e-mail address is bad, please contact ).