Patient, Clinician, and Observer Reported Outcomes all need intensive development. However, for some instruments complicated work up front is left behind when scoring systems, meaningful differences and responder definitions are discussed. Too often responses for items or the number of symptoms are simply added together to come up with a total score, ignoring the associations among the individual items and symptoms. This can be exacerbated when evaluating group and individual level responder analyses. Several modern methods use a variety of statistical methods to develop scoring systems. These will be discussed in addition to issues that can arise with single item instruments.