Thick Data Analytics (TDA): An Iterative and Inductive Framework for Algorithmic Improvement

Minh Nguyen, Tiffany Eulalio, Ben J. Marafino, Christian Rose, Jonathan H. Chen and Michael Baiocchi

The American Statistician, 2024, vol. 78, issue 4, 456-464

Abstract: A gap remains between developing risk prediction models and deploying models to support real-world decision making, especially in high-stakes situations. Human-experts’ reasoning abilities remain critical in identifying potential improvements and ensuring safety. We propose a thick data analytics (TDA) framework for eliciting and combining expert-human insight into the evaluation of models. The insight is 3-fold: (a) statistical methods are limited to using joint distributions of observable quantities for predictions but often there is more information available in a real-world than what is usable for algorithms, (b) domain experts can access more information (e.g., patient files) than an algorithm and bring additional knowledge into their assessments through leveraging insights and experiences, and (c) experts can re-frame and re-evaluate prediction problems to suit real-world situations. Here, we revisit an example of predicting temporal risk for intensive care admission within 24 hr of hospitalization. We propose a sampling procedure for identifying informative cases for deeper inspection. Expert feedback is used to understand sources of information to improve model development and deployment. We recommend model assessment based on objective evaluation metrics derived from subjective evaluations of the problem formulation. TDA insights facilitate iterative model development toward safer, actionable, and acceptable risk predictions.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00031305.2024.2327535 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:amstat:v:78:y:2024:i:4:p:456-464

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/UTAS20

DOI: 10.1080/00031305.2024.2327535

Access Statistics for this article

The American Statistician is currently edited by Eric Sampson

More articles in The American Statistician from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().