In studies where data has not been collected for a variable of interest the substitution of values generated by a prediction model is an appealing technique. Justification for such methods can be found by noting that, with standard assumptions, this is equivalent to fitting a regression model for an outcome variable when at least one covariate is measured with Berkson error. Under this setting, it is known that consistent or nearly-consistent inference can be obtained under many linear and non-linear outcome models. In this work, we focus on the linear regression outcome model and show that this consistency property does not hold when the outcome model is misspecified, in which case the marginal inference based on a covariate measured with Berkson error differs from the same inference based on observed covariates. Since outcome model misspecification is ubiquitous in applications, this severely limits the practical use of such measurements. These issues are illustrated using data from the National Health and Nutrition Examination Survey to study the joint association of total percent body fat and BMI with HbA1c.