When Random Assignment Is Not Enough: Accounting for Item Selectivity in Experimental Research

• Autores: Fred M. Feinberg, Linda Court Salisbury, Yuanping Ying
• Localización: Marketing science, ISSN 0732-2399, Vol. 35, Nº 6, 2016, págs. 976-994
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • Experimental methods are critical tools in marketing, psychology, and economics to isolate the effects of key variables from vagaries intrinsic to field data. As such, they are often considered exempt from the sort of sample selectivity artifacts widely documented in empirical research, in part because participants are randomly assigned to experimental conditions. To conserve time and resources, experiments often focus on items participants have chosen or are familiar with, for example, postchoice satisfaction ratings, certain free recall tasks, or specifying consideration sets preceding brand choice. When consumer input even partially influences the items about which researchers request subsequent data, the potential for item selectivity arises. In such situations, analyses are contingent on both the choice context(s) of the experiment and the alternatives participants elect to evaluate, potentially leading to substantial item selectivity overall and to differing degrees across conditions. We examine situations in which a nonignorable ?choose one of many? (polytomous) selection process limits which items offer up subsequent information, and develop methods to allow substantive results to pertain to the full set of items, not only those selected. The framework is illustrated via two experiments in which participants choose and then evaluate a frequently purchased consumer good as well as data first examined by Ratner et al. [Ratner RK, Kahn BE, Kahneman D (1999) Choosing less-preferred experiences for the sake of variety. J. Consumer Res. 26(1):1?15]. Results indicate substantial item selectivity that, when corrected for, can lead to markedly different interpretations of focal variable effects, such as large effect size changes and even sign reversal. Moreover, failing to flexibly account for item selectivity across experimental conditions, even in well-designed experimental settings, can lead to inaccurate substantive inferences about consumers? evaluative criteria. We further demonstrate robustness to theoretically driven (but not overtly misspecified) selection rules and provide researchers with a simple, ?two-step? exploratory procedure akin to a ?control function? approach?involving just one additional variable added to standard models?to determine whether and to what degree item selectivity may be affecting their substantive results. Data, as supplemental material, are available at https://doi.org/10.1287/mksc.2016.0991.