LAMBERT: Leveraging Attention Mechanisms to Improve the BERT Fine-Tuning Model for Encrypted Traffic Classification

Tao Liu, Xiting Ma, Ling Liu, Xin Liu, Yue Zhao, Ning Hu () and Kayhan Zrar Ghafoor
Additional contact information
Tao Liu: Institute of Cyberspace Security, Guangzhou University, Guangzhou 510006, China
Xiting Ma: Institute of Cyberspace Security, Guangzhou University, Guangzhou 510006, China
Ling Liu: Institute of Cyberspace Security, Guangzhou University, Guangzhou 510006, China
Xin Liu: College of Computer Engineering and Applied Math, Changsha University, Changsha 410022, China
Yue Zhao: Science and Technology on Communication Security Laboratory, Chengdu 610041, China
Ning Hu: Institute of Cyberspace Security, Guangzhou University, Guangzhou 510006, China
Kayhan Zrar Ghafoor: Department of Computer Science, Knowledge University, Erbil 44001, Iraq

Mathematics, 2024, vol. 12, issue 11, 1-22

Abstract: Encrypted traffic classification is a crucial part of privacy-preserving research. With the great success of artificial intelligence technology in fields such as image recognition and natural language processing, how to classify encrypted traffic based on AI technology has become an attractive topic in information security. With good generalization ability and high training accuracy, pre-training-based encrypted traffic classification methods have become the first option. The accuracy of this type of method depends highly on the fine-tuning model. However, it is a challenge for existing fine-tuned models to effectively integrate the representation of packet and byte features extracted via pre-training. A novel fine-tuning model, LAMBERT, is proposed in this article. By introducing an attention mechanism to capture the relationship between BiGRU and byte sequences, LAMBERT not only effectively improves the sequence loss phenomenon of BiGRU but also improves the processing performance of encrypted stream classification. LAMBERT can quickly and accurately classify multiple types of encrypted traffic. The experimental results show that our model performs well on datasets with uneven sample distribution, no pre-training, and large sample classification. LAMBERT was tested on four datasets, namely, ISCX-VPN-Service, ISCX-VPN-APP, USTC-TFC and CSTNET-TLS 1.3, and the F1 scores reached 99.15%, 99.52%, 99.30%, and 97.41%, respectively.

Keywords: information security; encrypted traffic classification; privacy protection; fine-tuning model; attention mechanism (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/11/1624/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/11/1624/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:11:p:1624-:d:1399516

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().