Learning with Stochastic Gradient

Adrian Barbu and Song-Chun Zhu
Additional contact information
Adrian Barbu: Florida State University, Department of Statistics
Song-Chun Zhu: University of California, Los Angeles, Departments of Statistics and Computer Science

Chapter 10 in Monte Carlo Methods, 2020, pp 327-366 from Springer

Abstract: Abstract Statistical learning often involves minimizing an objective function to find a suitable value of the model parameter. A simple and ubiquitous method for minimizing a differentiable objective function is Gradient Descent, which uses iterative parameter updates in the direction of steepest descent to minimize the objective. However, there are situations where the calculation of the objective function gradient is either analytically intractable or computationally infeasible. Two important examples are parameter estimation for Gibbs models and weight optimization in deep neural networks. Stochastic Gradient methods, which use a random but unbiased estimate of the full gradient, can be a useful tool for overcoming situations where the full gradient is unavailable. In the first half of the chapter, several theorems concerning the approximation of the true gradient from per-observation gradients are presented, and an important connection between Stochastic Gradient and Langevin Dynamics is discussed. The second section covers parameter estimation for Markov Random Field models, and the final section presents MCMC learning methods for deep image models. Stochastic gradient

Date: 2020
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-981-13-2971-5_10

Ordering information: This item can be ordered from
http://www.springer.com/9789811329715

DOI: 10.1007/978-981-13-2971-5_10

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().