Medical devices reaching a certain risk threshold must be evaluated as to their safety and efficacy before they can be legally marketed in the United States. Evaluation of radiological imaging devices poses certain complications because some level of subjective reader interpretation is necessary to utilize them. A thorough study of an imaging system, such as a mammography system or a computer aided detection system for chest x-rays will include reader interpretations. Many sources of bias can affect such studies. This presentation will discuss sources of bias and other complications in reader studies and some steps that can be taken to mitigate them.