The Assessment of Reliability Under Range Restriction: A Comparison of α, ω, and Test–Retest Reliability for Dichotomous Data

Abstract

Though much research and attention has been directed at assessing the correlation coefficient under range restriction, the assessment of reliability under range restriction has been largely ignored. This article uses item response theory to simulate dichotomous item-level data to assess the robustness of KR-20 (α), ω, and test–retest under varying selection ratios. These estimators, both corrected and uncorrected for range restriction, were compared in terms of both bias and precision. Test–retest reliability was usually the best estimator of reliability across a variety of conditions. Only under indirect range restriction did KR-20 and ω performed well. All estimators suffered imprecision as a function of range restriction, above and beyond the reduction in sample size. Based on the results, a set of recommendations are proposed.