Experimentation and Start-up Performance: Evidence from A/B Testing

Rembrand Koning (), Sharique Hasan () and Aaron Chatterji ()
Additional contact information
Rembrand Koning: Harvard Business School, Harvard University, Boston, Massachusetts 02163
Sharique Hasan: Fuqua School of Business, Duke University, Durham, North Carolina 27708
Aaron Chatterji: Fuqua School of Business, Duke University, Durham, North Carolina 27708; National Bureau of Economic Research, Cambridge, Massachusetts 02138

Management Science, 2022, vol. 68, issue 9, 6434-6453

Abstract: Recent scholarship argues that experimentation should be the organizing principle for entrepreneurial strategy. Experimentation leads to organizational learning, which drives improvements in firm performance. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of testing business ideas. Our results provide the first evidence on how digital experimentation affects a large sample of high-technology start-ups using data that tracks their growth, technology use, and products. We find that, although relatively few firms adopt A/B testing, among those that do, performance improves by 30%–100% after a year of use. We then argue that this substantial effect and relatively low adoption rate arises because start-ups do not only test one-off incremental changes, but also use A/B testing as part of a broader strategy of experimentation. Qualitative insights and additional quantitative analyses show that experimentation improves organizational learning, which helps start-ups develop more new products, identify and scale promising ideas, and fail faster when they receive negative signals. These findings inform the literatures on entrepreneurial strategy, organizational learning, and data-driven decision making.

Keywords: entrepreneurship; entrepreneurial strategy; experimentation; innovation; organizational learning (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.2021.4209 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:68:y:2022:i:9:p:6434-6453

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().