All Times ET
Keywords: Latent variable modeling, rating scales, item response theory, psychometrics
Drawing influence from recent studies analyzing the effects of trait-unrelated response style on rating scale responses, this study uses multidimensional item response theory to examine the correlation between a person’s subjective numeracy and their extreme response style. The subjective numeracy measure was based on an 8-item questionnaire with a 6-point rating scale, measuring comfort with mathematical operations. The extreme response style measure was based on the 16-item GreenLeaf questionnaire of “content-free” questions, on 5, 7, or 9-point rating scales. Measures were constructed using data collected from 456 MTurk participants. Each item for both subjective numeracy and GreenLeaf was split into dichotomous pseudo-items representing sub-scale response processes modeled by a multinomial processing tree. These pseudo-items distinguished between whether the respondent was on the upper or lower half of the scale, on a middle point or not, and on an extreme point or not. Three models were used: 1) Subjective numeracy and GreenLeaf pseudo-items loaded onto separate variables, 2) Model 1 but with subjective numeracy pseudo-items split into extreme and non-extreme variables, and 3) Extreme subjective numeracy pseudo-items cross-loaded onto both a subjective numeracy and extreme responding variable. Subjective numeracy and extreme responding had the highest correlation in the first model, suggesting response style may influence the subjective numeracy measure. The lowest correlation was observed in the third model, showing how this method could remove extreme responding from measures. AIC and BIC suggests that the second model fits best, indicating that extreme responding may not generalize across measures. This approach to modeling rating scale data has broad applications: response style can be removed from the data, and these modeling techniques can be used to examine the relationship between response style and other variables.