Scaling in Deep and Shallow Learning Architectures

Ella Koresh, Tal Halevi, Yuval Meir, Dolev Dilmoney, Tamar Dror, Ronit Gross, Ofek Tevet, Shiri Hodassman and Ido Kanter

Physica A: Statistical Mechanics and its Applications, 2024, vol. 646, issue C

Abstract: The realization of classification tasks using deep learning is a primary goal of artificial intelligence; however, its possible universal behavior remains unexplored. Herein, we demonstrate a scaling behavior for the test error, ϵ, as a function of the number of classified labels, K. For trained utmost deep architectures on CIFAR-100 ϵ(K)∝Kρ with ρ∼1, and in case of reduced deep architectures, ρ continuously decreases until a crossover to ϵ(K)∝log(K) is observed for shallow architectures. A similar crossover is observed for shallow architectures, where the number of filters in the convolutional layers is proportionally increased. This unified the scaling behavior of deep and shallow architectures, which yields a reduced latency method. The dependence of Δϵ/ΔK on the trained architecture is expected to be crucial in learning scenarios involving dynamic number of labels.

Keywords: Deep learning; Machine learning; Shallow learning (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437124004187
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:646:y:2024:i:c:s0378437124004187

DOI: 10.1016/j.physa.2024.129909

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().