Measuring Prediction Accuracy in a Maritime Accident Warning System

Jason R. W. Merrick, Claire A. Dorsey, Bo Wang, Martha Grabowski and John R. Harrald

Production and Operations Management, 2022, vol. 31, issue 2, 819-827

Abstract: Advances in machine learning methods and the availability of new data sources show promise for improving prediction of operational risk. Maritime transportation is the backbone of global supply chains and maritime accidents can lead to costly disruptions. We describe a case study performed for the United States Coast Guard (USCG) to develop a prototype risk prediction system to provide early alerts of elevated risk levels to vessel traffic managers and operators in the Lower Mississippi River, the second largest port of entry in the United States. Integrating incident and accident data from the USCG with environmental and traffic data sources, we tested existing machine learning algorithms in their predictive ability. We found poor accident prediction accuracy in cross‐validation using the traditional measures of precision and sensitivity. In this specific operational context, however, such single‐class accuracy metrics can be misleading. We define action precision and action sensitivity metrics that measure the accuracy of predictions in engendering the correct behavioral response (actions) among vessel operators, rather than getting the specific event classification correct. We use these operationally appropriate measures for maritime risk prediction to choose an algorithm for our prototype system. While the traditional metrics indicated that none of the algorithms would perform sufficiently well to use in the early warning system, the modified metrics show that the top performing algorithm will perform well in this operational context.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.1111/poms.13581

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:popmgt:v:31:y:2022:i:2:p:819-827

Ordering information: This journal article can be ordered from
http://onlinelibrary ... 1111/(ISSN)1937-5956

Access Statistics for this article

Production and Operations Management is currently edited by Kalyan Singhal

More articles in Production and Operations Management from Production and Operations Management Society
Bibliographic data for series maintained by Wiley Content Delivery ().