Robust Android Malware Detection System Against Adversarial Attacks Using Q-Learning

Hemant Rathore (), Sanjay K. Sahay (), Piyush Nikam () and Mohit Sewak ()
Additional contact information
Hemant Rathore: BITS
Sanjay K. Sahay: BITS
Piyush Nikam: BITS
Mohit Sewak: BITS

Information Systems Frontiers, 2021, vol. 23, issue 4, No 5, 867-882

Abstract: Abstract Since the inception of Andoroid OS, smartphones sales have been growing exponentially, and today it enjoys the monopoly in the smartphone marketplace. The widespread adoption of Android smartphones has drawn the attention of malware designers, which threatens the Android ecosystem. The current state-of-the-art Android malware detection systems are based on machine learning and deep learning models. Despite having superior performance, these models are susceptible to adversarial attack. Therefore in this paper, we developed eight Android malware detection models based on machine learning and deep neural network and investigated their robustness against the adversarial attacks. For the purpose, we created new variants of malware using Reinforcement Learning, which will be misclassified as benign by the existing Android malware detection models. We propose two novel attack strategies, namely single policy attack and multiple policy attack using reinforcement learning for white-box and grey-box scenario respectively. Putting ourselves in adversary’ shoes, we designed adversarial attacks on the detection models with the goal of maximising fooling rate, while making minimum modifications to the Android application and ensuring that the app’s functionality and behaviour does not change. We achieved an average fooling rate of 44.21% and 53.20% across all the eight detection models with maximum five modifications using a single policy attack and multiple policy attack, respectively. The highest fooling rate of 86.09% with five changes was attained against the decision tree based model using the multiple policy approach. Finally, we propose an adversarial defence strategy which reduces the average fooling rate by threefold to 15.22% against a single policy attack, thereby increasing the robustness of the detection models i.e. the proposed model can effectively detect variants (metamorphic) of malware. The experimental analysis shows that our proposed Android malware detection system using reinforcement learning is more robust against adversarial attacks.

Keywords: Adversarial learning; Android; Malware detection; Machine learning; Reinforcement learning; Static analysis (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://link.springer.com/10.1007/s10796-020-10083-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:infosf:v:23:y:2021:i:4:d:10.1007_s10796-020-10083-8

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10796

DOI: 10.1007/s10796-020-10083-8

Access Statistics for this article

Information Systems Frontiers is currently edited by Ram Ramesh and Raghav Rao

More articles in Information Systems Frontiers from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().