A Two‐Stage Training Method for Modeling Constrained Systems With Neural Networks

C. Coelho, M. Fernanda P. Costa and L.L. Ferrás

Journal of Forecasting, 2025, vol. 44, issue 5, 1785-1805

Abstract: Real‐world systems are often formulated as constrained optimization problems. Techniques to incorporate constraints into neural networks (NN), such as neural ordinary differential equations (Neural ODEs), have been used. However, these introduce hyperparameters that require manual tuning through trial and error, raising doubts about the successful incorporation of constraints into the generated model. This paper describes in detail the two‐stage training method for Neural ODEs, a simple, effective, and penalty parameter‐free approach to model constrained systems. In this approach, the constrained optimization problem is rewritten as two optimization subproblems that are solved in two stages. The first stage aims at finding feasible NN parameters by minimizing a measure of constraints violation. The second stage aims to find the optimal NN parameters by minimizing the loss function while keeping inside the feasible region. We experimentally demonstrate that our method produces models that satisfy the constraints and also improves their predictive performance, thus ensuring compliance with critical system properties and also contributing to reducing data quantity requirements. Furthermore, we show that the proposed method improves the convergence to an optimal solution and improves the explainability of Neural ODE models. Our proposed two‐stage training method can be used with any NN architectures.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/for.3270

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:jforec:v:44:y:2025:i:5:p:1785-1805

Access Statistics for this article

Journal of Forecasting is currently edited by Derek W. Bunn

More articles in Journal of Forecasting from John Wiley & Sons, Ltd.
Bibliographic data for series maintained by Wiley Content Delivery ().