Using Internet Data for Economic Research

Benjamin Edelman

Journal of Economic Perspectives, 2012, vol. 26, issue 2, 189-206

Abstract: The data used by economists can be broadly divided into two categories. First, structured datasets arise when a government agency, trade association, or company can justify the expense of assembling records. The Internet has transformed how economists interact with these datasets by lowering the cost of storing, updating, distributing, finding, and retrieving this information. Second, some economic researchers affirmatively collect data of interest. For researcher-collected data, the Internet opens exceptional possibilities both by increasing the amount of information available for researchers to gather and by lowering researchers' costs of collecting information. In this paper, I explore the Internet's new datasets, present methods for harnessing their wealth, and survey a sampling of the research questions these data help to answer. The first section of this paper discusses "scraping" the Internet for data—that is, collecting data on prices, quantities, and key characteristics that are already available on websites but not yet organized in a form useful for economic research. A second part of the paper considers online experiments, including experiments that the economic researcher observes but does not control (for example, when Amazon or eBay alters site design or bidding rules); and experiments in which a researcher participates in design, including those conducted in partnership with a company or website, and online versions of laboratory experiments. Finally, I discuss certain limits to this type of data collection, including both "terms of use" restrictions on websites and concerns about privacy and confidentiality.

JEL-codes: C80 (search for similar items in EconPapers)
Date: 2012
Note: DOI: 10.1257/jep.26.2.189
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (49)

Downloads: (external link)
http://www.aeaweb.org/articles.php?doi=10.1257/jep.26.2.189 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aea:jecper:v:26:y:2012:i:2:p:189-206

Ordering information: This journal article can be ordered from
https://www.aeaweb.org/journals/subscriptions

Access Statistics for this article

Journal of Economic Perspectives is currently edited by Enrico Moretti

More articles in Journal of Economic Perspectives from American Economic Association Contact information at EDIRC.
Bibliographic data for series maintained by Michael P. Albert ().