There is a widespread belief that design and execution factors are related to bias of results in randomized trials. We used meta-analytic techniques to explore the association between study outcomes and 11 quality items proposed by the Cochrane Back Review Group. We reviewed 261 randomized back pain trials. Each study was scored on 11 quality items. Effect sizes were calculated for one outcome per study. Bivariate meta-regressions were estimated while controlling for 1) each of the 11 quality items; 2) the sum of the 11 quality items; and 3) a “pseudo-jadad” score. We found that most of the validity items are associated with lower estimates of effect, although no item reaches the conventional level of statistical significance. There is a relationship between the number of internal validity items scored positive and lower estimates of effect, which reaches conventional statistical signif