Optimize to generalize in Gaussian processes: An alternative objective based on the Rényi divergence

Xubo Yue and Raed Al Kontar

IISE Transactions, 2024, vol. 56, issue 6, 600-610

Abstract: We introduce an alternative closed-form objective function α-ELBO for improved parameter estimation in the Gaussian process (GP) based on the Rényi α-divergence. We use a decreasing temperature parameter α to iteratively deform the objective function during optimization. Ultimately, our objective function converges to the exact log-marginal likelihood function of GP. At early optimization stages, α-ELBO can be viewed as a regularizer that smoothes some unwanted critical points. At late stages, α-ELBO recovers the exact log-marginal likelihood function that guides the optimizer to solutions that best explain the observed data. Theoretically, we derive an upper bound of the Rényi divergence under the proposed objective and derive convergence rates for a class of smooth and non-smooth kernels. Case studies on a wide range of real-life engineering applications demonstrate that our proposed objective is a practical alternative that offers improved prediction performance over several state-of-the-art inference techniques.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/24725854.2023.2219468 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:uiiexx:v:56:y:2024:i:6:p:600-610

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uiie20

DOI: 10.1080/24725854.2023.2219468

Access Statistics for this article

IISE Transactions is currently edited by Jianjun Shi

More articles in IISE Transactions from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().