Topological Regularization for Representation Learning via Persistent Homology

Muyi Chen (), Daling Wang, Shi Feng and Yifei Zhang
Additional contact information
Muyi Chen: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Daling Wang: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Shi Feng: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Yifei Zhang: School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China

Mathematics, 2023, vol. 11, issue 4, 1-22

Abstract: Generalization is challenging in small-sample-size regimes with over-parameterized deep neural networks, and a better representation is generally beneficial for generalization. In this paper, we present a novel method for controlling the internal representation of deep neural networks from a topological perspective. Leveraging the power of topology data analysis (TDA), we study the push-forward probability measure induced by the feature extractor, and we formulate a notion of “separation” to characterize a property of this measure in terms of persistent homology for the first time. Moreover, we perform a theoretical analysis of this property and prove that enforcing this property leads to better generalization. To impose this property, we propose a novel weight function to extract topological information, and we introduce a new regularizer including three items to guide the representation learning in a topology-aware manner. Experimental results in the point cloud optimization task show that our method is effective and powerful. Furthermore, results in the image classification task show that our method outperforms the previous methods by a significant margin.

Keywords: deep neural network; representation space; persistent homology; push-forward probability measure (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/4/1008/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/4/1008/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:4:p:1008-:d:1070398

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().