An Analysis of Vectorised Automatic Differentiation for Statistical Applications

Chun Fung Kwok, Dan Zhu () and Liana Jacobi
Additional contact information
Chun Fung Kwok: St. Vincent’s Institute of Medical Research, Melbourne 3065, Australia
Dan Zhu: Department of Econometrics and Business Statistics, Monash University, Melbourne 3800, Australia
Liana Jacobi: Department of Economics, University of Melbourne, Melbourne 3010, Australia

Stats, 2025, vol. 8, issue 2, 1-27

Abstract: Automatic differentiation (AD) is a general method for computing exact derivatives in complex sensitivity analyses and optimisation tasks, particularly when closed-form solutions are unavailable and traditional analytical or numerical methods fall short. This paper introduces a vectorised formulation of AD grounded in matrix calculus. It aligns naturally with the matrix-oriented style prevalent in statistics, supports convenient implementations, and takes advantage of sparse matrix representation and other high-level optimisation techniques that are not available in the scalar counterpart. Our formulation is well-suited to high-dimensional statistical applications, where finite differences (FD) scale poorly due to the need to repeat computations for each input dimension, resulting in significant overhead, and is advantageous in simulation-intensive settings—such as Markov Chain Monte Carlo (MCMC)-based inference—where FD requires repeated sampling and multiple function evaluations, while AD can compute exact derivatives in a single pass, substantially reducing computational cost. Numerical studies are presented to demonstrate the efficacy and speed of the proposed AD method compared with FD schemes.

Keywords: automatic differentiation; derivative computation; matrix calculus; MCMC; MLE; optimisation; simulation-based inference (search for similar items in EconPapers)
JEL-codes: C1 C10 C11 C14 C15 C16 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2571-905X/8/2/40/pdf (application/pdf)
https://www.mdpi.com/2571-905X/8/2/40/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jstats:v:8:y:2025:i:2:p:40-:d:1659287

Access Statistics for this article

Stats is currently edited by Mrs. Minnie Li

More articles in Stats from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().