 A comparative analysis of optimisation methods for classification on various datasetsSimanta Das and Soumitra Das International Journal of Complexity in Applied Science and Technology, 2026, vol. 2, issue 2, 109-127 Abstract: Optimisation studies how a variety of mathematical structures can be analysed through the minimisation or maximisation of a function. In deep learning (DL), optimisation encompasses everything from hyperparameter tuning to weight and bias adjustment until the convergence of a loss or cost function (J), such that the model's performance metrics and reliability in classification and regression are increased. Over the last few years, stochastic gradient methods and their variants - or adaptive gradient methods - have become very popular, with varying levels of success or otherwise. This study provides a neat comparison of adaptive gradient methods with respect to their accuracies and cross-entropy loss (CEL) in the mentioned tasks; it tested nine optimisation algorithms across three CNN architectures on MNIST, Fashion-MNIST, and CIFAR-10 datasets over 30 epochs. The overall top-performing optimisers were SGD, RMSProp, Adam, and Nadam, whereas Adagrad and Adadelta consistently performed lower. Keywords: adaptive gradient methods; optimisation methods; convolutional neural networks; CNNs; MNIST; FashionMNIST; CIFAR10. (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:ids:ijcast:v:2:y:2026:i:2:p:109-127 Access Statistics for this article More articles in International Journal of Complexity in Applied Science and Technology from Inderscience Enterprises Ltd |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.