Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

Nate Breznau, Eike Mark Rinke, Alexander Wuttke, Hung H. V. Nguyen, Muna Adem, Jule Adriaans, Amalia Alvarez-Benjumea, Henrik K. Andersen, Daniel Auer, Flavio Azevedo, Oke Bahnsen, Dave Balzer, Gerrit Bauer, Paul C. Bauer, Markus Baumann, Sharon Baute, Verena Benoit, Julian Bernauer, Carl Berning, Anna Berthold, Felix S. Bethke, Thomas Biegert, Katharina Blinzler, Johannes N. Blumenberg, Licia Bobzien, Andrea Bohman, Thijs Bol, Amie Bostic, Zuzanna Brzozowska, Katharina Burgdorf, Kaspar Burger, Kathrin B. Busch, Juan Carlos-Castillo, Nathan Chan, Pablo Christmann, Roxanne Connelly, Christian S. Czymara, Elena Damian, Alejandro Ecker, Achim Edelmann, Maureen A. Eger, Simon Ellerbrock, Anna Forke, Andrea Forster, Chris Gaasendam, Konstantin Gavras, Vernon Gayle, Theresa Gessler, Timo Gnambs, Amélie Godefroidt, Max Grömping, Martin Groß, Stefan Gruber, Tobias Gummer, Andreas Hadjar, Jan Paul Heisig, Sebastian Hellmeier, Stefanie Heyne, Magdalena Hirsch, Mikael Hjerm, Oshrat Hochman, Andreas Hövermann, Sophia Hunger, Christian Hunkler, Nora Huth, Zsófia S. Ignácz, Laura Jacobs, Jannes Jacobsen, Bastian Jaeger, Sebastian Jungkunz, Nils Jungmann, Mathias Kauff, Manuel Kleinert, Julia Klinger, Jan-Philipp Kolb, Marta Kołczyńska, John Kuk, Katharina Kunißen, Dafina Kurti Sinatra, Alexander Langenkamp, Philipp M. Lersch, Lea-Maria Löbel, Philipp Lutscher, Matthias Mader, Joan E. Madia, Natalia Malancu, Luis Maldonado, Helge Marahrens, Nicole Martin, Paul Martinez, Jochen Mayerl, Oscar J. Mayorga, Patricia McManus, Kyle McWagner, Cecil Meeusen, Daniel Meierrieks, Jonathan Mellon, Friedolin Merhout, Samuel Merk, Daniel Meyer, Leticia Micheli, Jonathan Mijs, Cristóbal Moya, Marcel Neunhoeffer, Daniel Nüst, Olav Nygård, Fabian Ochsenfeld, Gunnar Otte, Anna O. Pechenkina, Christopher Prosser, Louis Raes, Kevin Ralston, Miguel R. Ramos, Arne Roets, Jonathan Rogers, Guido Ropers, Robin Samuel, Gregor Sand, Ariela Schachter, Merlin Schaeffer, David Schieferdecker, Elmar Schlueter, Regine Schmidt, Katja M. Schmidt, Alexander Schmidt-Catran, Claudia Schmiedeberg, Jürgen Schneider, Martijn Schoonvelde, Julia Schulte-Cloos, Sandy Schumann, Reinhard Schunck, Jürgen Schupp, Julian Seuring, Henning Silber, Willem Sleegers, Nico Sonntag, Alexander Staudt, Nadia Steiber, Nils Steiner, Sebastian Sternberg, Dieter Stiers, Dragana Stojmenovska, Nora Storz, Erich Striessnig, Anne-Kathrin Stroppe, Janna Teltemann, Andrey Tibajev, Brian Tung, Giacomo Vagni, Jasper Van Assche, Meta van der Linden, Jolanda van der Noll, Arno Van Hootegem, Stefan Vogtenhuber, Bogdan Voicu, Fieke Wagemans, Nadja Wehl, Hannah Werner, Brenton M. Wiernik, Fabian Winter, Christof Wolf, Yuki Yamada, Nan Zhang, Conrad Ziller, Stefan Zins and Tomasz Żółtak
Additional contact information
Nate Breznau: a Research Center on Inequality and Social Policy (SOCIUM), University of Bremen, Bremen, 28359, Germany;
Eike Mark Rinke: b School of Politics and International Studies, University of Leeds, Leeds, LS2 9JT, United Kingdom;
Alexander Wuttke: l Department of Political Science, Ludwig Maximilian University, 80539 Munich, Germany;
Hung H. V. Nguyen: a Research Center on Inequality and Social Policy (SOCIUM), University of Bremen, Bremen, 28359, Germany;; d Bremen International Graduate School of Social Sciences, 28359 Bremen, Germany;
Muna Adem: e Department of Sociology, Indiana University, Bloomington, IN 47405;
Jule Adriaans: f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;
Amalia Alvarez-Benjumea: g Mechanisms of Normative Change, Max Planck Institute for Research on Collective Goods, 53113 Bonn, Germany;
Henrik K. Andersen: h Institute of Sociology, Chemnitz University of Technology, 09126 Chemnitz, Germany;
Daniel Auer: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Flavio Azevedo: j Department of Psychology, University of Cambridge, Cambridge, CB23RQ, United Kingdom;
Oke Bahnsen: i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Dave Balzer: k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;
Gerrit Bauer: oooo Department of Sociology, Ludwig Maximilian University, 80801 Munich, Germany;
Paul C. Bauer: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Markus Baumann: m Heidelberg University, 69117 Heidelberg, Germany;; dd Institute for Political Science, Goethe University Frankfurt, 60323 Frankfurt, Germany;
Sharon Baute: n Comparative Political Economy, University of Konstanz, 78457 Konstanz, Germany;
Verena Benoit: l Department of Political Science, Ludwig Maximilian University, 80539 Munich, Germany;; o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;
Julian Bernauer: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Carl Berning: pppp Institute for Political Science, Johannes Gutenberg University Mainz, 55099 Mainz, Germany;
Anna Berthold: o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;
Felix S. Bethke: p Research Department on Intrastate Conflict, Peace Research Institute Frankfurt, 60329 Frankfurt, Germany;
Thomas Biegert: q Department of Social Policy, London School of Economics and Political Science, London, WC2A 2AE, United Kingdom;
Katharina Blinzler: r Survey Data Curation, Leibniz Institute for the Social Sciences (GESIS), 50667 Cologne, Germany;
Johannes N. Blumenberg: rrrr Knowledge Exchange and Outreach, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany;
Licia Bobzien: s Jacques Delors Centre, Hertie School, 10117 Berlin, Germany;
Andrea Bohman: t Department of Sociology, Umeå University, 90187 Umeå, Sweden;
Thijs Bol: x Social Research Institute, Institute of Education, University College London, London, WC1H 0AL, United Kingdom;; uuuu Department of Sociology, University of Amsterdam, 1001 Amsterdam, The Netherlands;
Amie Bostic: u Department of Sociology, The University of Texas Rio Grande Valley, Brownsville, TX 78520;
Zuzanna Brzozowska: v Vienna Institute of Demography, Austrian Academy of Sciences, 1030 Vienna, Austria;; w Austrian National Public Health Institute, Gesundheit Österreich (GÖG), 1030 Vienna, Austria;
Katharina Burgdorf: i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Kaspar Burger: vvvv Jacobs Center for Productive Youth, University of Zurich, 8050 Zurich, Switzerland;; y Department of Sociology, University of Zurich, 8050 Zurich, Switzerland;
Kathrin B. Busch: z Independent researcher;
Juan Carlos-Castillo: aa Department of Sociology, University of Chile, Santiago, 7800284, Chile;; mmmm Center for Social Conflict and Cohesion Studies (COES), Pontificia Universidad Católica de Chile, Santiago, 8331150, Chile;
Nathan Chan: nnnn Department of Political Science and International Relations, Loyola Marymount University, Los Angeles, CA 90045;
Pablo Christmann: ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;
Roxanne Connelly: cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;
Christian S. Czymara: qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;
Elena Damian: yyyy Lifestyle and Chronic Diseases, Epidemiology and Public Health, Sciensano, 1000 Brussels, Belgium;
Alejandro Ecker: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Achim Edelmann: ff Médialab, Sciences Po, 75007 Paris, France;
Maureen A. Eger: t Department of Sociology, Umeå University, 90187 Umeå, Sweden;
Simon Ellerbrock: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;; i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Anna Forke: z Independent researcher;
Andrea Forster: pp Empirical Educational and Higher Education Research, Freie Universität Berlin, 14195 Berlin, Germany;
Chris Gaasendam: ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;
Konstantin Gavras: i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Vernon Gayle: cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;
Theresa Gessler: hhhh Kulturwissenschaftliche Fakultät, European University Viadrina, 15230 Frankfurt (Oder), Germany;
Timo Gnambs: gg Educational Measurement, Leibniz Institute for Educational Trajectories, 96047 Bamberg, Germany;
Amélie Godefroidt: aaaaa Centre for Research on Peace and Development, KU Leuven, 3000 Leuven, Belgium;
Max Grömping: hh School of Government and International Relations, Griffith University, Nathan, QLD, 4111, Australia;
Martin Groß: ii Department of Sociology, University of Tübingen, 72074 Tübingen, Germany;
Stefan Gruber: jj Max Planck Institute for Social Law and Social Policy, 80799 Munich, Germany;
Tobias Gummer: ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;
Andreas Hadjar: rrrrr Department of Social Sciences, University of Luxembourg, 4366 Esch-sur-Alzette, Luxembourg;; mm Wirtschafts- und Sozialwissenschaftliches Institut (WSI), Hans Böckler Foundation, 40474 Düsseldorf, Germany;; llll University of Fribourg, 1700 Fribourg, Switzerland;
Jan Paul Heisig: mmmmm Research Group "Health and Social Inequality", Berlin Social Science Center (WZB), 10785 Berlin, Germany;
Sebastian Hellmeier: nnnnn Transformations of Democracy Unit, Berlin Social Science Center (WZB), 10785 Berlin, Germany;
Stefanie Heyne: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Magdalena Hirsch: ooooo Research Unit Migration, Integration, Transnationalization, Berlin Social Science Center (WZB), 10785 Berlin, Germany;
Mikael Hjerm: t Department of Sociology, Umeå University, 90187 Umeå, Sweden;
Oshrat Hochman: ssss Data and Research on Society, Leibniz Institute for the Social Sciences, 68159 Mannheim, Germany;
Andreas Hövermann: mm Wirtschafts- und Sozialwissenschaftliches Institut (WSI), Hans Böckler Foundation, 40474 Düsseldorf, Germany;; qq German Socio-Economic Panel Survey, 10117 Berlin, Germany;
Sophia Hunger: ppppp Center for Civil Society Research, Berlin Social Science Center, 10785 Berlin, Germany;
Christian Hunkler: nn Berlin Institute for Integration and Migration Research (BIM), Humboldt University Berlin, 10099 Berlin, Germany;
Nora Huth: oo School of Human and Social Sciences, University of Wuppertal, 42119 Wuppertal, Germany;
Zsófia S. Ignácz: qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;
Laura Jacobs: ll Department of Political Science, Université Libre de Bruxelles, 1050 Bruxelles, Belgium;
Jannes Jacobsen: tt Zeppelin University, 88045 Friedrichshafen, Germany;; jjjj Cluster "Data-Methods-Monitoring", German Center for Integration and Migration Research (DeZIM),10117 Berlin, Germany;
Bastian Jaeger: rr Department of Social Psychology, Tilburg University, 5037AB Tilburg, The Netherlands;
Sebastian Jungkunz: iiiii Chair of Political Sociology, University of Bamberg, 96052 Bamberg, Germany;; lll Institute of Political Science, University of Münster, 48149 Münster, Germany;
Mathias Kauff: uu Department of Psychology, Medical School Hamburg, 20457 Hamburg, Germany;
Manuel Kleinert: uuu Institute of Sociology, Justus Liebig University of Giessen, 35394 Giessen, Germany;
Julia Klinger: jjjjj Institute of Sociology and Social Psychology, University of Cologne, 50931 Cologne, Germany;
Jan-Philipp Kolb: vv Federal Statistics Office Germany, Destatis, 65189 Wiesbaden, Germany;
Marta Kołczyńska: ww Department of Research on Social and Institutional Transformations, Institute of Political Studies of the Polish Academy of Sciences, 00-625 Warsaw, Poland;
John Kuk: xx Department of Political Science, University of Oklahoma, Norman, OK 73019;
Katharina Kunißen: k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;
Dafina Kurti Sinatra: z Independent researcher;
Alexander Langenkamp: qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;
Philipp M. Lersch: ddddd Department of Social Sciences, Humboldt University Berlin, 10099 Berlin, Germany;
Lea-Maria Löbel: f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;
Philipp Lutscher: yy Department of Political Science, University of Oslo, 0851 Oslo, Norway;
Matthias Mader: zz Department of Politics and Public Administration, University of Konstanz, 78457 Konstanz, Germany;
Joan E. Madia: lllll Institute for the Evaluation of Public Policies, Fondazione Bruno Kessler, 38122 Trento, Italy;
Natalia Malancu: bbb The Institute of Citizenship Studies (InCite), University of Geneva, 1205 Geneva, Switzerland;
Luis Maldonado: ccc Instituto de Sociologia, Pontifical Catholic University of Chile, Santiago, 7820436, Chile;
Helge Marahrens: e Department of Sociology, Indiana University, Bloomington, IN 47405;
Nicole Martin: ddd Department of Politics, University of Manchester, Manchester, M19 2JS, United Kingdom;
Paul Martinez: eee Department of Institutional Research, Western Governors University, Salt Lake City, UT 84107;
Jochen Mayerl: h Institute of Sociology, Chemnitz University of Technology, 09126 Chemnitz, Germany;
Oscar J. Mayorga: fff Department of Sociology, University of California, Los Angeles, CA 90095;
Patricia McManus: e Department of Sociology, Indiana University, Bloomington, IN 47405;
Kyle McWagner: bb Department of Political Science, The University of California, Irvine, CA 92617;
Cecil Meeusen: ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;
Daniel Meierrieks: ooooo Research Unit Migration, Integration, Transnationalization, Berlin Social Science Center (WZB), 10785 Berlin, Germany;
Jonathan Mellon: ddd Department of Politics, University of Manchester, Manchester, M19 2JS, United Kingdom;
Friedolin Merhout: ggg Department of Sociology and Centre for Social Data Science, University of Copenhagen, 1353 Copenhagen, Denmark;
Samuel Merk: hhh Department of School Development, University of Education Karlsruhe, 76133 Karlsruhe, Germany;
Daniel Meyer: kkkkk Department of Education and Social Sciences, University of Cologne, 50931 Cologne, Germany;
Leticia Micheli: iii Department of Psychology III, Julius-Maximilians University Würzburg, 97070 Würzburg, Germany;
Jonathan Mijs: jjj Department of Sociology, Boston University, Boston, MA 02215;
Cristóbal Moya: kkk Faculty of Sociology, Bielefeld University, 33615 Bielefeld, Germany;
Marcel Neunhoeffer: i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Daniel Nüst: hhhhh Department of Geosciences, University of Münster, 49149 Münster, Germany;
Olav Nygård: mmm Division of Migration, Ethnicity and Society (REMESO), Linköping University, 60174 Linköping, Sweden;
Fabian Ochsenfeld: nnn Administrative Headquarters, Max Planck Society, 80539 Berlin, Germany;
Gunnar Otte: k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;
Anna O. Pechenkina: ooo Department of Political Science, Utah State University, Logan, UT 84321;
Christopher Prosser: ppp Department of Politics, International Relations and Philosophy, Royal Holloway University of London, London, TW20 0EX, United Kingdom;
Kevin Ralston: cc School of Social and Political Science, University of Edinburgh, Edinburgh, EH8 9LD, United Kingdom;
Miguel R. Ramos: qqq Department of Social Policy, Sociology and Criminology, University of Birmingham, Birmingham, B15 2TT, United Kingdom;
Arne Roets: rrr Department of Developmental, Personality and Social Psychology, Ghent University, 9000 Ghent, Belgium;
Jonathan Rogers: sss Division of Social Science, New York University Abu Dhabi, Abu Dhabi, 10276, United Arab Emirates;
Guido Ropers: i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Robin Samuel: rrrrr Department of Social Sciences, University of Luxembourg, 4366 Esch-sur-Alzette, Luxembourg;
Gregor Sand: jj Max Planck Institute for Social Law and Social Policy, 80799 Munich, Germany;
Ariela Schachter: ttt Department of Sociology, Washington University in St. Louis, St. Louis, MO 63130;
Merlin Schaeffer: qqqqq Department of Sociology, University of Copenhagen, 1353 Copenhagen, Denmark;
David Schieferdecker: xxxx Institute for Media and Communication Studies, Freie Universität Berlin, 14195 Berlin, Germany;
Elmar Schlueter: uuu Institute of Sociology, Justus Liebig University of Giessen, 35394 Giessen, Germany;
Regine Schmidt: o Faculty of Social Sciences, Economics, and Business Administration, University of Bamberg, 96052 Bamberg, Germany;
Katja M. Schmidt: f Socio-Economic Panel Study (SOEP), German Institute for Economic Research (DIW), 10117 Berlin, Germany;
Alexander Schmidt-Catran: qqqq Institute of Sociology, Goethe University Frankfurt, 60323 Frankfurt, Germany;
Claudia Schmiedeberg: oooo Department of Sociology, Ludwig Maximilian University, 80801 Munich, Germany;
Jürgen Schneider: ccccc Tübingen School of Education, University of Tübingen, 72074 Tübingen, Germany;
Martijn Schoonvelde: sssss Department of European Languages and Cultures, University of Groningen, 9712 EK Groningen, The Netherlands;
Julia Schulte-Cloos: kkkk Robert Schuman Center for Advanced Studies, European University Institute, 50133 Florence, Italy;
Sandy Schumann: wwww Department of Security and Crime Science, University College London, London,WC1E 6BT, United Kingdom;
Julian Seuring: bbbbb Department of Migration, Leibniz Institute for Educational Trajectories, 96047 Bamberg, Germany;
Henning Silber: tttt Department of Survey Design and Methodology, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany;
Willem Sleegers: rr Department of Social Psychology, Tilburg University, 5037AB Tilburg, The Netherlands;
Nico Sonntag: k Institute of Sociology, Johannes Gutenberg University Mainz, 55128 Mainz, Germany;
Alexander Staudt: z Independent researcher;
Nadia Steiber: www Department of Sociology, University of Vienna, 1090 Vienna, Austria;
Nils Steiner: pppp Institute for Political Science, Johannes Gutenberg University Mainz, 55099 Mainz, Germany;
Sebastian Sternberg: z Independent researcher;
Dieter Stiers: zzzz Center for Political Science Research, KU Leuven, 3000 Leuven, Belgium;
Dragana Stojmenovska: uuuu Department of Sociology, University of Amsterdam, 1001 Amsterdam, The Netherlands;
Nora Storz: xxx Interdisciplinary Social Science, Utrecht University, 3584 Utrecht, The Netherlands;
Erich Striessnig: ttttt Department of Demography, University of Vienna, 1010 Vienna, Austria;
Anne-Kathrin Stroppe: r Survey Data Curation, Leibniz Institute for the Social Sciences (GESIS), 50667 Cologne, Germany;
Janna Teltemann: yyy Institute for Social Sciences, University of Hildesheim, 31141 Hildesheim, Germany;
Andrey Tibajev: mmm Division of Migration, Ethnicity and Society (REMESO), Linköping University, 60174 Linköping, Sweden;
Brian Tung: ttt Department of Sociology, Washington University in St. Louis, St. Louis, MO 63130;
Giacomo Vagni: x Social Research Institute, Institute of Education, University College London, London, WC1H 0AL, United Kingdom;
Jasper Van Assche: eeeee Center for Social and Cultural Psychology, Université Libre de Bruxelles, 1050 Brussels, Belgium;
Meta van der Linden: xxx Interdisciplinary Social Science, Utrecht University, 3584 Utrecht, The Netherlands;
Jolanda van der Noll: zzz Department of Psychology, University of Hagen, 58097 Hagen, Germany;
Arno Van Hootegem: ee Department of Sociology, Center for Sociological Research, KU Leuven, 3000 Leuven, Belgium;
Stefan Vogtenhuber: uuuuu Education and Employment, Institute for Advanced Studies, University of Vienna, Vienna, 1080 Austria;
Bogdan Voicu: aaaa Research Institute for Quality of Life, Romanian Academy, 010071 Bucharest, Romania;; bbbb Department of Sociology, Lucian Blaga University of Sibiu, 550024 Sibiu, Romania;
Fieke Wagemans: vvvvv Policy Perspectives, Citizen Perspectives, and Behaviors, Netherlands Institute for Social Research, 2594 The Hague, The Netherlands;
Nadja Wehl: dddd Research Cluster "The Politics of Inequality", University of Konstanz, 78464 Konstanz, Germany;
Hannah Werner: zzzz Center for Political Science Research, KU Leuven, 3000 Leuven, Belgium;
Brenton M. Wiernik: eeee Department of Psychology, University of South Florida, Tampa, FL 33620;
Fabian Winter: g Mechanisms of Normative Change, Max Planck Institute for Research on Collective Goods, 53113 Bonn, Germany;
Christof Wolf: wwwww President, Leibniz Institute for the Social Sciences (GESIS), 68159 Mannheim, Germany; i School of Social Sciences, University of Mannheim, 68159 Mannheim, Germany;
Yuki Yamada: ffff Faculty of Arts and Science, Kyushu University, Fukuoka, 819-0395, Japan;
Nan Zhang: c Mannheim Centre for European Social Research, University of Mannheim, 68131 Mannheim, Germany;
Conrad Ziller: ggggg Department of Political Science, University of Duisburg-Essen, 47057 Duisburg, Germany;
Stefan Zins: gggg Institute for Employment Research, Federal Employment Agency, 90478 Nuremberg, Germany;
Tomasz Żółtak: ww Department of Research on Social and Institutional Transformations, Institute of Political Studies of the Polish Academy of Sciences, 00-625 Warsaw, Poland;

Proceedings of the National Academy of Sciences, 2022, vol. 119, issue 44, e2203150119

Abstract: Will different researchers converge on similar findings when analyzing the same data? Seventy-three independent research teams used identical cross-country survey data to test a prominent social science hypothesis: that more immigration will reduce public support for government provision of social policies. Instead of convergence, teams’ results varied greatly, ranging from large negative to large positive effects of immigration on social policy support. The choices made by the research teams in designing their statistical tests explain very little of this variation; a hidden universe of uncertainty remains. Considering this variation, scientists, especially those working with the complexities of human societies and behavior, should exercise humility and strive to better account for the uncertainty in their work.

Keywords: metascience; many analysts; researcher degrees of freedom; analytical flexibility; immigration and policy preferences (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (9)

Downloads: (external link)
http://www.pnas.org/content/119/44/e2203150119.full (application/pdf)

Related works:
Journal Article: Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty (2022)
Working Paper: Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty (2022)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nas:journl:v:119:y:2022:p:e2203150119

Access Statistics for this article

More articles in Proceedings of the National Academy of Sciences from Proceedings of the National Academy of Sciences
Bibliographic data for series maintained by PNAS Product Team ().