Improving the Efficiency of Human-in-the-Loop Systems: Adding Artificial to Human Experts

Johannes Jakubik (), Daniel Weber (), Patrick Hemmer (), Michael Vössing () and Gerhard Satzger ()
Additional contact information
Johannes Jakubik: Karlsruhe Institute of Technology
Daniel Weber: Karlsruhe Institute of Technology
Patrick Hemmer: Karlsruhe Institute of Technology
Michael Vössing: Karlsruhe Institute of Technology
Gerhard Satzger: Karlsruhe Institute of Technology

A chapter in Solutions and Technologies for Responsible Digitalization, 2025, pp 131-147 from Springer

Abstract: Abstract Information systems increasingly leverage artificial intelligence (AI) and machine learning (ML) to generate value from vast amounts of data. However, ML models are imperfect and can generate incorrect classifications. Hence, human- in-the-loop (HITL) extensions to ML models add a human review for instances that are difficult to classify. This study argues that continuously relying on human experts to handle difficult model classifications leads to a strong increase in human effort, which strains limited resources. To address this issue, we propose a hybrid system that creates artificial experts that learn to classify data instances from unknown classes previously reviewed by human experts. Our hybrid system assesses which artificial expert is suitable for classifying an instance from an unknown class and automatically assigns it. Over time, this reduces human effort and increases the efficiency of the system. Our experiments demonstrate that our approach outperforms traditional HITL systems for several benchmarks on image classification.

Keywords: Human-in-the-loop systems; Artificial experts; Human-AI collaboration; Unknown data (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:lnichp:978-3-031-80122-8_9

Ordering information: This item can be ordered from
http://www.springer.com/9783031801228

DOI: 10.1007/978-3-031-80122-8_9

Access Statistics for this chapter

More chapters in Lecture Notes in Information Systems and Organization from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().