D-efficient or deficient? A robustness analysis of stated choice experimental designs
Joan L. Walker (),
Yanqiao Wang (),
Mikkel Thorhauge () and
Moshe Ben-Akiva ()
Additional contact information
Joan L. Walker: University of California, Berkeley
Yanqiao Wang: University of California, Berkeley
Mikkel Thorhauge: Technical University of Denmark
Moshe Ben-Akiva: Massachusetts Institute of Technology
Theory and Decision, 2018, vol. 84, issue 2, 215-238
Abstract This paper is motivated by the increasing popularity of efficient designs for stated choice experiments. The objective in efficient designs is to create a stated choice experiment that minimizes the standard errors of the estimated parameters. In order to do so, such designs require specifying prior values for the parameters to be estimated. While there is significant literature demonstrating the efficiency improvements (and cost savings) of employing efficient designs, the bulk of the literature tests conditions where the priors used to generate the efficient design are assumed to be accurate. However, there is substantially less literature that compares how different design types perform under varying degree of error of the prior. The literature that does exist assumes small fractions are used (e.g., under 20 unique choice tasks generated), which is in contrast to computer-aided surveys that readily allow for large fractions. Further, the results in the literature are abstract in that there is no reference point (i.e., meaningful units) to provide clear insight on the magnitude of any issue. Our objective is to analyze the robustness of different designs within a typical stated choice experiment context of a trade-off between price and quality. We use as an example transportation mode choice, where the key parameter to estimate is the value of time (VOT). Within this context, we test many designs to examine how robust efficient designs are against a misspecification of the prior parameters. The simple mode choice setting allows for insightful visualizations of the designs themselves and also an interpretable reference point (VOT) for the range in which each design is robust. Not surprisingly, the D-efficient design is most efficient in the region where the true population VOT is near the prior used to generate the design: the prior is $20/h and the efficient range is $10–$30/h. However, the D-efficient design quickly becomes the most inefficient outside of this range (under $5/h and above $40/h), and the estimation significantly degrades above $50/h. The orthogonal and random designs are robust for a much larger range of VOT. The robustness of Bayesian efficient designs varies depending on the variance that the prior assumes. Implementing two-stage designs that first use a small sample to estimate priors are also not robust relative to uninformative designs. Arguably, the random design (which is the easiest to generate) performs as well as any design, and it (as well as any design) will perform even better if data cleaning is done to remove choice tasks where one alternative dominates the other.
Keywords: Stated choice experiments; Robustness; Mode choice model; Value-of-time; Experimental design; D-efficient (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2) Track citations by RSS feed
Downloads: (external link)
http://link.springer.com/10.1007/s11238-017-9647-3 Abstract (text/html)
Access to full text is restricted to subscribers.
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:kap:theord:v:84:y:2018:i:2:d:10.1007_s11238-017-9647-3
Ordering information: This journal article can be ordered from
http://www.springer. ... ry/journal/11238/PS2
Access Statistics for this article
Theory and Decision is currently edited by Mohammed Abdellaoui
More articles in Theory and Decision from Springer
Bibliographic data for series maintained by Sonal Shukla ().