Validity and Reliability of Multiplicity Measures Among Men Who Have Sex with Men (MSM) in 16 U.S. Cities (303608)

Keywords: validity, reliability, measurement, data quality, venue attendance, multiplicity

Accurate measurement of venue attendance frequency (multiplicity) is important to weight data collected using venue-based sampling (VBS). CDC’s National HIV Behavioral Surveillance (NHBS) system currently uses a 12- month recall period and categorical responses. Review of survey literature and expert consultation with NHBS statisticians suggested that a 30-day recall period and a continuous response scale would improve the measure and weighted estimates. In 2014, NHBS conducted a pilot to assess validity and reliability of alternative multiplicity measures.

Data were collected from MSM recruited and interviewed at venues in 16 US cities (n=7,734). We used a split ballot design to assess two multiplicity measures using a 30-day recall period and either a continuous (number of times) or 7-point categorical (“Never” to “More than once a day”) response scale. We assessed construct validity, item nonresponse, interviewer variance, and recall method, using t-tests, intra-class correlation coefficients (ICC), and graphical examination of each measure’s response distribution.

T-test results provided evidence of construct validity for the continuous response measure; multiplicity was higher among men who identified as gay, (p=<0.0001), and men who had disclosed their sexual identity to others (p=0.0007). Item nonresponse was low for each measure and in the range of nonresponse typical for NHBS (=1%). Interviewer variance for each measure (ICC=.03, continuous, ICC=.02, categorical) was also within the range typically observed for face-to-face surveys. Responses to the continuous measure clustered around common multiples, similar to other measures of this type.

The use of a 30-day recall period and continuous response scale was found to be a reliable and valid measure of multiplicity. This measure could also allow more variation for the creation of sampling weights. We recommend its implementation in future studies utilizing venue-based sampling.