Active Learning: Encoder-Decoder-Outlayer and Vector Space Diversification Sampling

Hongyi Zeng () and Fanyi Kong
Additional contact information
Hongyi Zeng: Department of Computer Science, University of Toronto, Toronto, ON M5S 0A5, Canada
Fanyi Kong: Department of Mechanical & Industrial Engineering, Northeastern University, Boston, MA 02115, USA

Mathematics, 2023, vol. 11, issue 13, 1-17

Abstract: This study introduces a training pipeline comprising two components: the Encoder-Decoder-Outlayer framework and the Vector Space Diversification Sampling method. This framework efficiently separates the pre-training and fine-tuning stages, while the sampling method employs pivot nodes to divide the subvector space and selectively choose unlabeled data, thereby reducing the reliance on human labeling. The pipeline offers numerous advantages, including rapid training, parallelization, buffer capability, flexibility, low GPU memory usage, and a sample method with nearly linear time complexity. Experimental results demonstrate that models trained with the proposed sampling algorithm generally outperform those trained with random sampling on small datasets. These characteristics make it a highly efficient and effective training approach for machine learning models. Further details can be found in the project repository on GitHub.

Keywords: neural network; training pipeline; Encoder-Decoder-Outlayer framework; Vector Space Diversification Sampling method; human labeling; GPU memory (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/13/2819/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/13/2819/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:13:p:2819-:d:1177602

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().