How Well Generative Adversarial Networks Learn Distributions

Tengyuan Liang ()
Additional contact information
Tengyuan Liang: University of Chicago - Booth School of Business

No 2020-154, Working Papers from Becker Friedman Institute for Research In Economics

Abstract: This paper studies the rates of convergence for learning distributions implicitly with the adversarial framework and Generative Adversarial Networks (GAN), which subsume Wasserstein, Sobolev, MMD GAN, and Generalized/Simulated Method of Moments (GMM/SMM) as special cases. We study a wide range of parametric and nonparametric target distributions, under a host of objective evaluation metrics. We investigate how to obtain a good statistical guarantee for GANs through the lens of regularization. On the nonparametric end, we derive the optimal minimax rates for distribution estimation under the adversarial framework. On the parametric end, we establish a theory for general neural network classes (including deep leaky ReLU networks), that characterizes the interplay on the choice of generator and discriminator pair. We discover and isolate a new notion of regularization, called the generator-discriminator-pair regularization, that sheds light on the advantage of GANs compared to classical parametric and nonparametric approaches for explicit distribution estimation. We develop novel oracle inequalities as the main technical tools for analyzing GANs, which is of independent interest.

Keywords: Generative adversarial networks; implicit distribution estimation; simulated method of moments; oracle inequality; neural network learning; mini- max problem; pair regularization (search for similar items in EconPapers)
Pages: 31 pages
Date: 2020
New Economics Papers: this item is included in nep-cmp and nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://repec.bfi.uchicago.edu/RePEc/pdfs/BFI_WP_2020154.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bfi:wpaper:2020-154

Access Statistics for this paper

More papers in Working Papers from Becker Friedman Institute for Research In Economics Contact information at EDIRC.
Bibliographic data for series maintained by Toni Shears ( this e-mail address is bad, please contact ).