Meta-Learned Adaptive Proximal Operators for Neural Network Optimization

Dr. Sahayarajjoseph Nirmalkumar S. ()

International Journal of Innovative Science and Research Technology (IJISRT), 2026, vol. 11, issue 06, 800-812

Abstract: We propose a novel approach to neural network optimization by replacing traditional proximal operators with meta-learned adaptive updates, thereby unifying the optimization of step sizes, regularization strengths, and sparsity thresholds into a single learned process. The proposed method introduces a bi-level optimization framework in which a recurrent meta-learner dynamically produces task- and architecture-specific proximal parameters during training, thereby removing the necessity for manual tuning. The system’s central mechanism relies on an LSTM-driven meta-learner handling optimization trajectories and architectural embeddings, with the latter derived from a graph neural network to support generalization across architectures. The proximal updates that emerge blend learned parameters with a continuous shrinkage operator, which prevents gradient discontinuities and preserves sparsity. The meta-learner is trained by optimizing a bi-level objective aimed at reducing the anticipated final loss over tasks, with gradients estimated by truncated backpropagation through time. The framework operates smoothly with traditional neural network training by substituting standard optimizer steps with updates derived from meta-learning. Experiments show that the method adjusts to various architectures and tasks, achieving better performance than fixed proximal approaches and diminishing the need for manual hyperparameter adjustment. Furthermore, the architectural embeddings support zero-shot generalization to novel network structures, which renders the approach especially appropriate for automated machine learning pipelines. This work is important because it moves away from strict optimization heuristics, adopting an approach that learns optimization strategies which inherently adjust to both task demands and architectural limitations.

Keywords: Proximal Operators; Neural Network Optimization; LSTM-Driven Meta-Learner; Zero-Shot Generalization. (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.ijisrt.com/metalearned-adaptive-proxim ... network-optimization (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cvr:ijisrt:2026:06:ijisrt26jun632

DOI: 10.38124/ijisrt/26jun632

Access Statistics for this article

More articles in International Journal of Innovative Science and Research Technology (IJISRT) from IJISRT Publication
Bibliographic data for series maintained by Rahul Goyel ().