Models versus Datasets: Reducing Bias through Building a Comprehensive IDS Benchmark

Rasheed Ahmad, Izzat Alsmadi, Wasim Alhamdani and Lo’ai Tawalbeh
Additional contact information
Rasheed Ahmad: Department of Computer Information Sciences, University of the Cumberlands, 6178 College Station Drive, Williamsburg, KY 40769, USA
Izzat Alsmadi: Department of computing and cyber security, University of Texas A&M San Antonio, One University Way, San Antonio, TX 78224, USA
Wasim Alhamdani: Department of Computer Information Sciences, University of the Cumberlands, 6178 College Station Drive, Williamsburg, KY 40769, USA
Lo’ai Tawalbeh: Department of computing and cyber security, University of Texas A&M San Antonio, One University Way, San Antonio, TX 78224, USA

Future Internet, 2021, vol. 13, issue 12, 1-22

Abstract: Today, deep learning approaches are widely used to build Intrusion Detection Systems for securing IoT environments. However, the models’ hidden and complex nature raises various concerns, such as trusting the model output and understanding why the model made certain decisions. Researchers generally publish their proposed model’s settings and performance results based on a specific dataset and a classification model but do not report the proposed model’s output and findings. Similarly, many researchers suggest an IDS solution by focusing only on a single benchmark dataset and classifier. Such solutions are prone to generating inaccurate and biased results. This paper overcomes these limitations in previous work by analyzing various benchmark datasets and various individual and hybrid deep learning classifiers towards finding the best IDS solution for IoT that is efficient, lightweight, and comprehensive in detecting network anomalies. We also showed the model’s localized predictions and analyzed the top contributing features impacting the global performance of deep learning models. This paper aims to extract the aggregate knowledge from various datasets and classifiers and analyze the commonalities to avoid any possible bias in results and increase the trust and transparency of deep learning models. We believe this paper’s findings will help future researchers build a comprehensive IDS based on well-performing classifiers and utilize the aggregated knowledge and the minimum set of significantly contributing features.

Keywords: Intrusion Detection System (IDS); deep learning; feature extraction; Internet of Things (IoT); model interpretation (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/13/12/318/pdf (application/pdf)
https://www.mdpi.com/1999-5903/13/12/318/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:13:y:2021:i:12:p:318-:d:705271

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().