Skip to main content
Log in

Sample size requirements for stated choice experiments

  • Published:
Transportation Aims and scope Submit manuscript

Abstract

Stated choice (SC) experiments represent the dominant data paradigm in the study of behavioral responses of individuals, households as well as other organizations, yet in the past little has been known about the sample size requirements for models estimated from such data. Traditional orthogonal designs and existing sampling theories does not adequately address the issue and hence researchers have had to resort to simple rules of thumb or ignore the issue and collect samples of arbitrary size, hoping that the sample is sufficiently large enough to produce reliable parameter estimates, or are forced to make assumptions about the data that are unlikely to hold in practice. In this paper, we demonstrate how a recently proposed sample size computation can be used to generate so-called S-efficient designs using prior parameter values to estimate panel mixed multinomial logit models. Sample size requirements for such designs in SC studies are investigated. In a numerical case study is shown that a D-efficient and even more an S-efficient design require a (much) smaller sample size than a random orthogonal design in order to estimate all parameters at the level of statistical significance. Furthermore, it is shown that wide level range has a significant positive influence on the efficiency of the design and therefore on the reliability of the parameter estimates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. As has been demonstrated by Bliemer and Rose (2011) for the MNL model, there is no need to generate large designs and then give subsets of choice tasks to respondents. In fact, this could be even suboptimal, as small optimal designs contain the best choice tasks that retrieve the most information, and larger optimal designs are likely to contain inferior choice tasks. This has also been argued by Kanninen (2002), who demonstrates that an optimal dataset is simply a repetition of the smallest set of optimal choice tasks. However, there exist several cases in which respondent-specific designs may be better, for example in case of segmentation of the population, or to make choice sets more realistic by pivoting off a respondent-specific reference alternative, see Rose et al. (2008).

References

  • Ben-Akiva, M., Lerman, S.R.: Discrete choice analysis: theory and application to travel Demand. MIT Press, Cambridge (1985)

    Google Scholar 

  • Bliemer, M.C.J., Rose J.M.,: Efficiency and Sample Size Requirements for Stated Choice Studies. Working paper ITLS-WP-05-08, Institute of Transport and Logistics Studies, The University of Sydney (2005)

  • Bliemer , M.C.J, Rose J.M.,: Efficiency and sample size requirements for stated choice studies. Presented at the 88th annual meeting of the transportation research board, Washington DC (2009)

  • Bliemer, M.C.J., Rose, J.M., Hensher, D.A.: Efficient stated choice experiments for estimating nested logit models. Transp. Res. Part B 43(1), 19–35 (2009)

    Article  Google Scholar 

  • Bliemer, M.C.J., Rose, J.M., Hess, S.: Approximation of Bayesian efficiency in experimental choice designs. J. Choice Model. 1(1), 98–127 (2008)

    Article  Google Scholar 

  • Bliemer, M.C.J., Rose, J.M.: Experimental design influences on stated choice outputs: an empirical study in air travel choice. Transp. Res. Part A. 45(1), 63–79 (2011)

    Google Scholar 

  • Bliemer, M.C.J., Rose, J.M.: Construction of Experimental Designs for Mixed Logit Models Allowing for Correlation Across Choice Observations. Transp. Res. Part B. 46(3), 720–734 (2010)

    Article  Google Scholar 

  • Bunch, D.S., Louviere, J.J., Anderson, D.: A Comparison of Experimental Design Strategies for Choice-Based Conjoint Analysis with Generic-Attribute Multinomial Logit Models, Working Paper. Graduate School of Management, University of California, Davis (1996)

    Google Scholar 

  • Carlsson, F., Peter Martinsson, P.: Design techniques for stated preference methods in health economics. Health Econ. 12(4), 281–294 (2003)

    Article  Google Scholar 

  • Garrod, G.D., Scarpa, R., Willis, K.G.: Estimating the benefits of traffic calming on through routes: a choice experiment approach. J. Transp. Econ. Policy 36(2), 211–232 (2002)

    Google Scholar 

  • Hensher, D.A., Prioni, P.: A service quality index for area-wide contract performance assessment. J. Transp. Econ. Policy 36(1), 93–114 (2002)

    Google Scholar 

  • Hensher, D.A., Rose, J.M., Greene, W.H.: Applied choice analysis: a primer. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  • Huber, J., Zwerina, K.: The importance of utility balance in efficient choice designs. J. Mark. Res. 33, 307–317 (1996)

    Article  Google Scholar 

  • Johnson, R., Orme, B. (2003) Getting the most from CBC, Sawtooth Software Research Paper Series, Sawtooth Software, Sequim

  • Kanninen, B.J.: Optimal design for multinomial choice experiments. J. Mark. Res. 39, 214–217 (2002)

    Article  Google Scholar 

  • Kessels, R., Bradley, B., Goos, P., Vandebroek, M.: An efficient algorithm for constructing Bayesian optimal choice designs. J. Bus. Econ. Stat. 27(2), 279–291 (2009)

    Article  Google Scholar 

  • Louviere, J.J., Hensher, D.A., Swait, J.D.: Stated choice methods: analysis and application. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  • Manski, C.F., McFadden, D.: Alternative Estimators and Sample Designs for Discrete Choice Analysis. In: Manski, C.F., McFadden, D. (eds.) Structural analysis of discrete data with econometric applications, pp. 2–50. MIT Press, Cambridge (1981)

    Google Scholar 

  • McFadden, D.: Econometric analysis of qualitative response models. In: Griliches, Z., Intriligator, M.D. (eds.) Handbook of econometrics II, pp. 1395–1457. Elseviere Science, Amsterdam (1984)

    Chapter  Google Scholar 

  • McFadden, D.: Conditional logit analysis of qualitative choice behavior. In: Zarembka, (ed.) Frontiers in econometrics, pp. 105–142. Academic Press, New York (1974)

    Google Scholar 

  • ChoiceMetrics (2012) Ngene 1.1.1 User Manual & Reference Guide, Australia, www.choice-metrics.com. Accessed 14 March 2012

  • Orme, B. (1998) Sample size issues for conjoint analysis studies, Sawtooth Software Technical Paper, Sequim

  • Orme, B. (2010) Sample Size Issues for Conjoint Analysis Studies, Sawtooth Software Technical Paper, Sequim

  • Quan, W., Rose, J.M., Collins, A.T., Bliemer, M.C.J.: A comparison of Algorithms for Generating Efficient Choice Experiments, Working Paper ITLS-WP-11-19. Institute of Transport and Logistics Studies, The University of Sydney, Sydney (2011)

    Google Scholar 

  • Rose, J.M., Bliemer, M.C.J.: Sample Optimality in the Design of Stated Choice Experiments, Working paper ITLS-WP-05-13. Institute of Transport and Logistics Studies, The Univesrity of Sydney, Sydney (2005a)

    Google Scholar 

  • Rose, J.M., Bliemer, M.C.J.: Constructing Efficient Choice Experiments, Working Paper, ITLS-WP-05-07. Institute of Transport and Logistics Studies, The University of Sydney, Sydney (2005b)

    Google Scholar 

  • Rose, J.M., Bliemer, M.C.J., Hensher, D.A., Collins, A.T.: Designing efficient stated choice experiments in the presence of reference alternatives. Transp. Res. Part B 42(4), 396–406 (2008)

    Article  Google Scholar 

  • Sándor, Z., Wedel, M.: Designing conjoint choice experiments using managers’ prior beliefs. J. Mark. Res. 38, 430–444 (2001)

    Article  Google Scholar 

  • Sándor, Z., Wedel, M.: Profile construction in experimental choice designs for mixed logit models. Marketing Science 21(4), 455–476 (2002)

    Article  Google Scholar 

  • Yu, J., Goos, P.P., Vandebroek, M.: Efficient conjoint choice designs in the presence of respondent heterogeneity. Marketing Science 28, 122–135 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to John M. Rose.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rose, J.M., Bliemer, M.C.J. Sample size requirements for stated choice experiments. Transportation 40, 1021–1041 (2013). https://doi.org/10.1007/s11116-013-9451-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11116-013-9451-z

Keywords

Navigation