Metrics for evaluating performance and uncertainty of Bayesian network models

Bruce G. Marcot

Ecological Modelling, 2012, vol. 230, issue C, 50-62

Abstract: This paper presents a selected set of existing and new metrics for gauging Bayesian network model performance and uncertainty. Selected existing and new metrics are discussed for conducting model sensitivity analysis (variance reduction, entropy reduction, case file simulation); evaluating scenarios (influence analysis); depicting model complexity (numbers of model variables, links, node states, conditional probabilities, and node cliques); assessing prediction performance (confusion tables, covariate- and conditional probability-weighted confusion error rates, area under receiver operating characteristic curves, k-fold cross-validation, spherical payoff, Schwarz’ Bayesian information criterion, true skill statistic, Cohen's kappa); and evaluating uncertainty of model posterior probability distributions (Bayesian credible interval, posterior probability certainty index, certainty envelope, Gini coefficient). Examples are presented of applying the metrics to 3 real-world models of wildlife population analysis and management. Using such metrics can vitally bolster model credibility, acceptance, and appropriate application, particularly when informing management decisions.

Keywords: Bayesian network model; Uncertainty; Model performance; Model validation; Sensitivity analysis; Error rates; Probability analysis (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (36)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0304380012000245
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ecomod:v:230:y:2012:i:c:p:50-62

DOI: 10.1016/j.ecolmodel.2012.01.013

Access Statistics for this article

Ecological Modelling is currently edited by Brian D. Fath

More articles in Ecological Modelling from Elsevier
Bibliographic data for series maintained by Catherine Liu ().