Algorithmic decision-making and system destructiveness: A case of automatic debt recovery

Tapani Rinta-Kahila, Ida Someh, Nicole Gillespie, Marta Indulska and Shirley Gregor

European Journal of Information Systems, 2022, vol. 31, issue 3, 313-338

Abstract: Governments are increasingly relying on algorithmic decision-making (ADM) to deliver public services. Recent information systems literature has raised concerns regarding ADM’s negative unintended consequences, such as widespread discrimination, which in extreme cases can be destructive to society. The extant empirical literature, however, has not sufficiently examined the destructive effects of governmental ADM. In this paper, we report on a case study of the Australian government’s “Robodebt” programme that was designed to automatically calculate and collect welfare overpayment debts from citizens but ended up causing severe distress to citizens and welfare agency staff. Employing perspectives from systems thinking and organisational limits, we develop a research model that explains how a socially destructive government ADM programme was initiated, sustained, and delegitimized. The model offers a set of generalisable mechanisms that can benefit investigations of ADM’s consequences. Our findings contribute to the literature of unintended consequences of ADM and demonstrate to practitioners the importance of setting up robust governance infrastructures for ADM programmes.

Date: 2022
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://hdl.handle.net/10.1080/0960085X.2021.1960905 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tjisxx:v:31:y:2022:i:3:p:313-338

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/tjis20

DOI: 10.1080/0960085X.2021.1960905

Access Statistics for this article

European Journal of Information Systems is currently edited by Par Agerfalk

More articles in European Journal of Information Systems from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().