Classification of Hacker’s Posts Based on Zero-Shot, Few-Shot, and Fine-Tuned LLMs in Environments with Constrained Resources

Theodoros Giannilias, Andreas Papadakis (), Nikolaos Nikolaou and Theodore Zahariadis
Additional contact information
Theodoros Giannilias: Synelixis Solutions S.A., Farmakidou 10, 34100 Chalkida, Greece
Andreas Papadakis: Synelixis Solutions S.A., Farmakidou 10, 34100 Chalkida, Greece
Nikolaos Nikolaou: Synelixis Solutions S.A., Farmakidou 10, 34100 Chalkida, Greece
Theodore Zahariadis: Synelixis Solutions S.A., Farmakidou 10, 34100 Chalkida, Greece

Future Internet, 2025, vol. 17, issue 5, 1-22

Abstract: This paper investigates, applies, and evaluates state-of-the-art Large Language Models (LLMs) for the classification of posts from a dark web hackers’ forum into four cyber-security categories. The LLMs applied included Mistral-7B-Instruct-v0.2, Gemma-1.1-7B, Llama-3-8B-Instruct, and Llama-2-7B, with zero-shot learning, few-shot learning, and fine-tuning. The four cyber-security categories consisted of “Access Control and Management”, “Availability Protection and Security by Design Mechanisms”, “Software and Firmware Flaws”, and “not relevant”. The hackers’ posts were also classified and labelled by a human cyber-security expert, allowing a detailed evaluation of the classification accuracy per each LLM and customization/learning method. We verified LLM fine-tuning as the most effective mechanism to enhance the accuracy and reliability of the classifications. The results include the methodology applied and the labelled hackers’ posts dataset.

Keywords: large language model; mistral; gemma; llama; cybersecurity; dark web; text classification; few-shot learning; fine-tuning (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/17/5/207/pdf (application/pdf)
https://www.mdpi.com/1999-5903/17/5/207/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:17:y:2025:i:5:p:207-:d:1649266

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().