A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems
Jacob W. Ulvila () and
John E. Gaffney ()
Additional contact information
Jacob W. Ulvila: Decision Science Associates, Inc., P.O. Box 969, Vienna, Virginia 22183
John E. Gaffney: Lockheed Martin, 700 North Frederick Avenue, Gaithersburg, Maryland 20879
Decision Analysis, 2004, vol. 1, issue 1, 35-50
This paper presents a decision analysis method for evaluating computer intrusion detection systems. The method integrates and extends receiver operating characteristic (ROC) and cost analysis methods to provide an expected cost metric. We demonstrate that both the ROC analysis and cost analysis methods are incomplete. Furthermore, we demonstrate how a decision tree can combine and extend the ROC and cost analysis methods to provide an expected cost metric that reflects the intrusion detection system's ROC curve, costs, and assessments of the hostility of the environment as summarized by the prior probability of intrusion. We further demonstrate how this method can be used to decide the optimal operating point on an intrusion detector's ROC curve, choose the best intrusion detection system, compare the value of one intrusion detection system with another's, determine the value of an intrusion detector over no detector, and determine how to adjust the operation of an intrusion detector to respond to changes in its environment. General results are given and the method is illustrated in several numerical examples that involve both hypothetical and real intrusion detection systems. We demonstrate that, contrary to common advice, the value of an intrusion detection system depends not only on its ROC curve, but also on various costs (such as those associated with making incorrect decisions about detection) and the hostility of the operating environment. Conclusions are drawn about the design and evaluation of intrusion detection systems and the role for decision analysis in that design and evaluation.
Keywords: intrusion detection; decision tree; evaluation; ROC; IDS (search for similar items in EconPapers)
References: View complete reference list from CitEc
Citations: View citations in EconPapers (11) Track citations by RSS feed
Downloads: (external link)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:inm:ordeca:v:1:y:2004:i:1:p:35-50
Access Statistics for this article
More articles in Decision Analysis from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Matthew Walls ().