Measuring the predictability of life outcomes with a scientific mass collaboration

Matthew J. Salganik (), Ian Lundberg, Alexander T. Kindel, Caitlin E. Ahearn, Khaled Al-Ghoneim, Abdullah Almaatouq, Drew M. Altschul, Jennie E. Brand, Nicole Bohme Carnegie, Ryan James Compton, Debanjan Datta, Thomas Davidson, Anna Filippova, Connor Gilroy, Brian J. Goode, Eaman Jahani, Ridhi Kashyap, Antje Kirchner, Stephen McKay, Allison C. Morgan, Alex Pentland, Kivan Polimis, Louis Raes, Daniel E. Rigobon, Claudia V. Roberts, Diana M. Stanescu, Yoshihiko Suhara, Adaner Usmani, Erik H. Wang, Muna Adem, Abdulla Alhajri, Bedoor AlShebli, Redwane Amin, Ryan B. Amos, Lisa P. Argyle, Livia Baer-Bositis, Moritz Büchi, Bo-Ryehn Chung, William Eggert, Gregory Faletto, Zhilin Fan, Jeremy Freese, Tejomay Gadgil, Josh Gagné, Yue Gao, Andrew Halpern-Manners, Sonia P. Hashim, Sonia Hausen, Guanhua He, Kimberly Higuera, Bernie Hogan, Ilana M. Horwitz, Lisa M. Hummel, Naman Jain, Kun Jin, David Jurgens, Patrick Kaminski, Areg Karapetyan, E. H. Kim, Ben Leizman, Naijia Liu, Malte Möser, Andrew E. Mack, Mayank Mahajan, Noah Mandell, Helge Marahrens, Diana Mercado-Garcia, Viola Mocz, Katariina Mueller-Gastell, Ahmed Musse, Qiankun Niu, William Nowak, Hamidreza Omidvar, Andrew Or, Karen Ouyang, Katy M. Pinto, Ethan Porter, Kristin E. Porter, Crystal Qian, Tamkinat Rauf, Anahit Sargsyan, Thomas Schaffner, Landon Schnabel, Bryan Schonfeld, Ben Sender, Jonathan D. Tang, Emma Tsurkov, Austin van Loon, Onur Varol, Xiafei Wang, Zhi Wang, Julia Wang, Flora Wang, Samantha Weissman, Kirstie Whitaker, Maria K. Wolters, Wei Lee Woon, James Wu, Catherine Wu, Kengran Yang, Jingwen Yin, Bingyu Zhao, Chenyun Zhu, Jeanne Brooks-Gunn, Barbara E. Engelhardt, Moritz Hardt, Dean Knox, Karen Levy, Arvind Narayanan, Brandon M. Stewart, Duncan J. Watts and Sara McLanahan ()
Additional contact information
Matthew J. Salganik: Department of Sociology, Princeton University, Princeton, NJ 08544
Ian Lundberg: Department of Sociology, Princeton University, Princeton, NJ 08544
Alexander T. Kindel: Department of Sociology, Princeton University, Princeton, NJ 08544
Caitlin E. Ahearn: Department of Sociology, University of California, Los Angeles, CA 90095
Khaled Al-Ghoneim: Hawaz, Riyadh 12363, Saudi Arabia
Abdullah Almaatouq: Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142; Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139
Drew M. Altschul: Mental Health Data Science Scotland, Department of Psychology, The University of Edinburgh, Edinburgh EH8 9JZ, United Kingdom;
Jennie E. Brand: Department of Sociology, University of California, Los Angeles, CA 90095; Department of Statistics, University of California, Los Angeles, CA 90095
Nicole Bohme Carnegie: Department of Mathematical Sciences, Montana State University, Bozeman, MT 59717
Ryan James Compton: Human Computer Interaction Lab, University of California, Santa Cruz, CA 95064
Debanjan Datta: Discovery Analytics Center, Virginia Polytechnic Institute and State University, Arlington, VA 22203
Thomas Davidson: Department of Sociology, Cornell University, Ithaca, NY 14853
Anna Filippova: GitHub, San Francisco, CA 94107
Connor Gilroy: Department of Sociology, University of Washington, Seattle, WA 98105
Brian J. Goode: Social and Decision Analytics Laboratory, Fralin Life Sciences Institute, Virginia Polytechnic Institute and State University, Arlington, VA 22203
Eaman Jahani: Institute for Data, Systems and Society, Massachusetts Institute of Technology, Cambridge, MA 02139
Ridhi Kashyap: Department of Sociology, University of Oxford, Oxford OX1 1JD, United Kingdom; Nuffield College, University of Oxford, Oxford OX1 1NF, United Kingdom;School of Anthropology and Museum Ethnography, University of Oxford, Oxford OX2 6PE, United Kingdom
Antje Kirchner: Program for Research in Survey Methodology, Survey Research Division, RTI International, Research Triangle Park, NC 27709
Stephen McKay: School of Social and Political Sciences, University of Lincoln, Brayford Pool, Lincoln LN6 7TS, United Kingdom
Allison C. Morgan: Department of Computer Science, University of Colorado, Boulder, CO 80309
Alex Pentland: Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139
Kivan Polimis: Center for the Study of Demography and Ecology, University of Washington, Seattle, WA 98105
Daniel E. Rigobon: Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ 08544
Claudia V. Roberts: Department of Computer Science, Princeton University, Princeton, NJ 08544
Diana M. Stanescu: Department of Politics, Princeton University,Princeton, NJ, 08544
Yoshihiko Suhara: Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139
Adaner Usmani: Department of Sociology, Harvard University, Cambridge, MA 02138
Erik H. Wang: Department of Politics, Princeton University,Princeton, NJ, 08544
Muna Adem: Department of Sociology, Indiana University, Bloomington, IN 47405
Abdulla Alhajri: Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139
Bedoor AlShebli: Computational Social Science Lab, Social Science Division, New York University Abu Dhabi, 129188 Abu Dhabi, United Arab Emirates
Redwane Amin: Bendheim Center for Finance, Princeton University, Princeton, NJ 08544
Ryan B. Amos: Department of Computer Science, Princeton University, Princeton, NJ 08544
Lisa P. Argyle: Department of Political Science, Brigham Young University, Provo, UT 84602
Livia Baer-Bositis: Department of Sociology, Stanford University, Stanford, CA 94305
Moritz Büchi: Department of Communication and Media Research, University of Zurich, Zurich, Switzerland, ZH-8050
Bo-Ryehn Chung: Center for Statistics & Machine Learning, Princeton University, Princeton, NJ 08544
William Eggert: Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, NJ 08544
Gregory Faletto: Statistics Group, Department of Data Sciences and Operations, Marshall School of Business, University of Southern California, Los Angeles, CA 90089
Zhilin Fan: Department of Statistics, Columbia University, New York, NY 10027
Jeremy Freese: Department of Sociology, Stanford University, Stanford, CA 94305
Tejomay Gadgil: Center for Data Science, New York University, New York, NY 10011
Josh Gagné: Department of Sociology, Stanford University, Stanford, CA 94305
Yue Gao: Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027
Andrew Halpern-Manners: Department of Sociology, Indiana University, Bloomington, IN 47405
Sonia P. Hashim: Department of Computer Science, Princeton University, Princeton, NJ 08544
Sonia Hausen: Department of Sociology, Stanford University, Stanford, CA 94305
Guanhua He: Department of Molecular Biology, Princeton University, Princeton, NJ 08544
Kimberly Higuera: Department of Sociology, Stanford University, Stanford, CA 94305
Bernie Hogan: Oxford Internet Institute, University of Oxford, Oxford OX1 3JS, United Kingdom
Ilana M. Horwitz: Graduate School of Education, Stanford University, Stanford, CA, 94305
Lisa M. Hummel: Department of Sociology, Stanford University, Stanford, CA 94305
Naman Jain: Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ 08544
Kun Jin: Department of Computer Science, Ohio State University, Columbus, OH 43210
David Jurgens: School of Information, University of Michigan, Ann Arbor, MI 48104
Patrick Kaminski: Department of Sociology, Indiana University, Bloomington, IN 47405; Center for Complex Networks and Systems Research, Indiana University, Bloomington, IN 47405
Areg Karapetyan: Department of Computer Science, Masdar Institute, Khalifa University, 127788 Abu Dhabi, United Arab Emirates; Research Institute for Mathematical Sciences, Kyoto University, Kyoto 606-8502, Japan
E. H. Kim: Department of Sociology, Stanford University, Stanford, CA 94305
Ben Leizman: Department of Computer Science, Princeton University, Princeton, NJ 08544
Naijia Liu: Department of Politics, Princeton University,Princeton, NJ, 08544
Malte Möser: Department of Computer Science, Princeton University, Princeton, NJ 08544
Andrew E. Mack: Department of Politics, Princeton University,Princeton, NJ, 08544
Mayank Mahajan: Department of Computer Science, Princeton University, Princeton, NJ 08544
Noah Mandell: Department of Astrophysical Sciences, Princeton University, Princeton, NJ 08544
Helge Marahrens: Department of Sociology, Indiana University, Bloomington, IN 47405
Diana Mercado-Garcia: Graduate School of Education, Stanford University, Stanford, CA, 94305
Viola Mocz: Department of Neuroscience, Princeton University, Princeton, NJ 08544
Katariina Mueller-Gastell: Department of Sociology, Stanford University, Stanford, CA 94305
Ahmed Musse: Department of Electrical Engineering, Princeton University, Princeton, NJ, 08544
Qiankun Niu: Bendheim Center for Finance, Princeton University, Princeton, NJ 08544
William Nowak: Dataiku, New York, NY 10010
Hamidreza Omidvar: Department of Civil and Environmental Engineering, Princeton University, Princeton, NJ 08544
Andrew Or: Department of Computer Science, Princeton University, Princeton, NJ 08544
Karen Ouyang: Department of Computer Science, Princeton University, Princeton, NJ 08544
Katy M. Pinto: Department of Sociology, California State University, Dominguez Hills, Carson, CA 90747
Ethan Porter: School of Media and Public Affairs, George Washington University, Washington, DC 20052
Kristin E. Porter: School of Media and Public Affairs, George Washington University, Washington, DC 20052
Crystal Qian: Center for Data Insights, MDRC, Oakland, CA 94612
Tamkinat Rauf: Department of Computer Science, Princeton University, Princeton, NJ 08544
Anahit Sargsyan: Department of Sociology, Stanford University, Stanford, CA 94305
Thomas Schaffner: Social Science Division, New York University Abu Dhabi, 129188 Abu Dhabi, United Arab Emirates
Landon Schnabel: Department of Computer Science, Princeton University, Princeton, NJ 08544
Bryan Schonfeld: Department of Sociology, Stanford University, Stanford, CA 94305
Ben Sender: Department of Politics, Princeton University,Princeton, NJ, 08544
Jonathan D. Tang: Department of Economics, Princeton University, Princeton, NJ 08544
Emma Tsurkov: Department of Sociology, Stanford University, Stanford, CA 94305
Austin van Loon: Department of Sociology, Stanford University, Stanford, CA 94305
Onur Varol: Center for Complex Network Research, Northeastern University Networks Science Institute, Boston, MA 02115 Luddy School of Informatics, Computing, & Engineering, Indiana University, Bloomington, IN 47408
Xiafei Wang: School of Social Work, David B. Falk College of Sport and Human Dynamics, Syracuse University, NY 13244
Zhi Wang: Luddy School of Informatics, Computing, & Engineering, Indiana University, Bloomington, IN 47408; School of Public Health, Indiana University, Bloomington, IN 47408
Julia Wang: Department of Computer Science, Princeton University, Princeton, NJ 08544
Flora Wang: Department of Economics, Princeton University, Princeton, NJ 08544
Samantha Weissman: Department of Computer Science, Princeton University, Princeton, NJ 08544
Kirstie Whitaker: The Alan Turing Institute, London NW1 2DB, United Kingdom
Maria K. Wolters: Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, United Kingdom
Wei Lee Woon: School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, United Kingdom
James Wu: Department of Marketplaces & Yield Data Science, Expedia Group, Seattle, WA 98119
Catherine Wu: Department of the Applied Statistics, Social Science, and Humanities, New York University, New York, NY 10003
Kengran Yang: Department of Computer Science, Princeton University, Princeton, NJ 08544
Jingwen Yin: Department of Civil and Environmental Engineering, Princeton University, Princeton, NJ 08544
Bingyu Zhao: Department of Statistics, Columbia University, New York, NY 10027
Chenyun Zhu: Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, United Kingdom
Jeanne Brooks-Gunn: Department of Statistics, Columbia University, New York, NY 10027
Barbara E. Engelhardt: Department of Human Development, Teachers College, Columbia University, New York, NY 10027; Department of Pediatrics, Vagelos College of Physicians and Surgeons, Columbia University, New York, NY 10032
Moritz Hardt: Department of Computer Science, Princeton University, Princeton, NJ 08544
Dean Knox: Center for Statistics & Machine Learning, Princeton University, Princeton, NJ 08544
Karen Levy: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA 94720
Arvind Narayanan: Department of Computer Science, Princeton University, Princeton, NJ 08544
Brandon M. Stewart: Department of Sociology, Princeton University, Princeton, NJ 08544
Duncan J. Watts: Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104; Annenberg School of Communication, University of Pennsylvania, Philadelphia, PA 19104; Operations, Information and Decisions Department, University of Pennsylvania, Philadelphia, PA 19104
Sara McLanahan: Department of Sociology, Princeton University, Princeton, NJ 08544

Proceedings of the National Academy of Sciences, 2020, vol. 117, issue 14, 8398-8403

Abstract: How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life outcomes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

Keywords: life course; prediction; machine learning; mass collaboration (search for similar items in EconPapers)
Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.pnas.org/content/117/15/8398.full (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nas:journl:v:117:y:2020:p:8398-8403

Access Statistics for this article

More articles in Proceedings of the National Academy of Sciences from Proceedings of the National Academy of Sciences
Bibliographic data for series maintained by PNAS Product Team ().