Transformer-Based Parameter Estimation in Statistics

Xiaoxin Yin () and David S. Yin
Additional contact information
Xiaoxin Yin: Independent Researchers, San Jose, CA 95129, USA
David S. Yin: Independent Researchers, San Jose, CA 95129, USA

Mathematics, 2024, vol. 12, issue 7, 1-11

Abstract: Parameter estimation is one of the most important tasks in statistics, and is key to helping people understand the distribution behind a sample of observations. Traditionally, parameter estimation is done either by closed-form solutions (e.g., maximum likelihood estimation for Gaussian distribution) or by iterative numerical methods such as the Newton–Raphson method when a closed-form solution does not exist (e.g., for Beta distribution). In this paper, we propose a transformer-based approach to parameter estimation. Compared with existing solutions, our approach does not require a closed-form solution or any mathematical derivations. It does not even require knowing the probability density function, which is needed by numerical methods. After the transformer model is trained, only a single inference is needed to estimate the parameters of the underlying distribution based on a sample of observations. In the empirical study, we compared our approach with maximum likelihood estimation on commonly used distributions such as normal distribution, exponential distribution and beta distribution. It is shown that our approach achieves similar or better accuracy as measured by mean-square-errors.

Keywords: parameter estimation; transformer; deep learning (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/7/1040/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/7/1040/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:7:p:1040-:d:1367516

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().