Malware Detection Based on API Call Sequence Analysis: A Gated Recurrent Unit–Generative Adversarial Network Model Approach

Nsikak Owoh (), John Adejoh, Salaheddin Hosseinzadeh, Moses Ashawa, Jude Osamor and Ayyaz Qureshi
Additional contact information
Nsikak Owoh: Department of Cyber Security and Networks, Glasgow Caledonian University, Glasgow G4 0BA, UK
John Adejoh: Department of Software Engineering, African University of Science and Technology, Abuja 900107, Nigeria
Salaheddin Hosseinzadeh: Department of Cyber Security and Networks, Glasgow Caledonian University, Glasgow G4 0BA, UK
Moses Ashawa: Department of Cyber Security and Networks, Glasgow Caledonian University, Glasgow G4 0BA, UK
Jude Osamor: School of Computer Science and Engineering, University of Westminster, 309 Regent Street, London W1B 2HW, UK
Ayyaz Qureshi: Department of Cyber Security and Networks, Glasgow Caledonian University, Glasgow G4 0BA, UK

Future Internet, 2024, vol. 16, issue 10, 1-29

Abstract: Malware remains a major threat to computer systems, with a vast number of new samples being identified and documented regularly. Windows systems are particularly vulnerable to malicious programs like viruses, worms, and trojans. Dynamic analysis, which involves observing malware behavior during execution in a controlled environment, has emerged as a powerful technique for detection. This approach often focuses on analyzing Application Programming Interface (API) calls, which represent the interactions between the malware and the operating system. Recent advances in deep learning have shown promise in improving malware detection accuracy using API call sequence data. However, the potential of Generative Adversarial Networks (GANs) for this purpose remains largely unexplored. This paper proposes a novel hybrid deep learning model combining Gated Recurrent Units (GRUs) and GANs to enhance malware detection based on API call sequences from Windows portable executable files. We evaluate our GRU–GAN model against other approaches like Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU) on multiple datasets. Results demonstrated the superior performance of our hybrid model, achieving 98.9% accuracy on the most challenging dataset. It outperformed existing models in resource utilization, with faster training and testing times and low memory usage.

Keywords: malware detection; dynamic malware analysis; API call sequence; deep learning; Gated Recurrent Unit; Generative Adversarial Network (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/16/10/369/pdf (application/pdf)
https://www.mdpi.com/1999-5903/16/10/369/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:16:y:2024:i:10:p:369-:d:1497719

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().