The Use of Experimental Methods by IS Scholars: An Illustrated Typology

Marta Ballatore, Lise Arena and Agnès Festré
Additional contact information
Marta Ballatore: GREDEG CNRS

No 2020-28, GREDEG Working Papers from Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France

Abstract: This article aims at making an updated typology of recent experimental studies in the IS literature on the period 1999-2019. Based on a full-text search within the Association for Information Systems (AIS) “basket” of eight top IS journals (EJIS, ISR, JAIS, ISJ, JIT, JMIS, JSIS and MISQ), this research gathered 392 articles and highlights the use of 5 different types of experiments in IS, mainly: artificial simulations, laboratory experiments, field experiments, online experiments (scenario simulation game-based; brainstorming-based. . . ) and natural experiments. Each category is discussed through the perspective of its degree of control, and technological realism. Results show the significant predominance of laboratory experiments over field and natural experiments on the period. This, in turn, stresses the preferred tendency followed by IS scholars to perceive experimental methods as a way to control the source of variations of variables under study. In addition, this paper provides a better understanding of the context of use of a specific experimental method. Overall, it is shown that laboratory experiments (including scenario-based lab experiments) are mainly used, in a deterministic manner, to assess or test the impact of an IS on human decision-making or behaviour. By contrast, artificial simulations experiments are more appropriate to study emergent phenomena and to make predictions, often providing key insights about quality and effectiveness of IS.

Keywords: Methodology; IS; Meta-research; Experimental method; Behavioural Science; Experimental Economics (search for similar items in EconPapers)
Pages: 18 pages
Date: 2020-06
New Economics Papers: this item is included in nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://195.220.190.85/GREDEG-WP-2020-28.pdf First version, 2020 (application/pdf)

Related works:
Working Paper: The Use of Experimental Methods by IS Scholars: An Illustrated Typology (2020)
Working Paper: The Use of Experimental Methods by IS Scholars: An Illustrated Typology (2020)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gre:wpaper:2020-28

Access Statistics for this paper

More papers in GREDEG Working Papers from Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France Contact information at EDIRC.
Bibliographic data for series maintained by Patrice Bougette ().