Neural Teleportation

Marco Armenta (), Thierry Judge, Nathan Painchaud, Youssef Skandarani, Carl Lemaire, Gabriel Gibeau Sanchez, Philippe Spino and Pierre-Marc Jodoin
Additional contact information
Marco Armenta: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Thierry Judge: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Nathan Painchaud: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Youssef Skandarani: Department of Computer Science, Université de Bourgogne Franche-Comte, 21000 Dijon, France
Carl Lemaire: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Gabriel Gibeau Sanchez: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Philippe Spino: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada
Pierre-Marc Jodoin: Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, Canada

Mathematics, 2023, vol. 11, issue 2, 1-27

Abstract: In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation teleports a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on the surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process.

Keywords: quiver representations; neural networks; teleportation (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/2/480/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/2/480/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:2:p:480-:d:1037552

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().