THE ACHILLES HEEL OF PSYCHOLOGY: HOW CONVENIENCE SAMPLING AFFECTS PARAMETER ESTIMATES

Abstract

Many have commented on potential problems associated with using under- graduate psychology students as research participants (e.g. Arnett, 2008; Hen- rich, Heine, & Norenzayan, 2010; Highhouse & Gillespie, 2009; Rosenthal & Ros- now, 1969). However, little research has been directed at demonstrating the extent of bias that may result from such practices and how to address this bias. In this dissertation, I investigate how the F statistic and treatment effects are affected when researchers use a convenience sample. I show that without measur- ing and modeling the selection variable, these parameter estimates are biased. I also show that covariate adjustments can mitigate bias when interactions do not occur between the treatment effect(s) and the selection variable. When interac- tions do exist, however, it is impossible to eliminate bias, particularly for the F statistic.