Using Propensity Score Analysis to Assess the Effectiveness of Social Marketing Campaigns in Healthcare: An Example from Medicare Open Enrollment
Clarese Astrin, Centers for Medicare & Medicaid Services 
Diane Field, Centers for Medicare & Medicaid Services 
*Frank Funderburk, Centers for Medicare & Medicaid Services 

Keywords: propensity scores, social marketing, evaluation methods

The Medicare Open Enrollment (OE) period is a time during which Medicare beneficiaries can alter their prescription drug plan choices. Immediately prior to and during the OE period CMS conducts a campaign (using both earned and paid media) designed to support beneficiaries in choosing a plan that will best meet their needs. Key outcomes of interest include increased awareness of the OE period, knowledge of options available during the OE period and the extent to which beneficiaries engage in specific behavioral actions (reviewing current coverage, comparing current coverage with other available plans). Effectiveness of the campaign is assessed in part via beneficiary surveys (pre- and post-campaign) that examine a variety of independent variables including recognition of campaign messages, channels of exposure (e.g., radio, TV, print), and type of content (e.g., news, advertisement) in relation to the key outcomes noted above. However, as is common in many health promotion and communication evaluations, traditional analytic methods are limited in the extent to which causal inferences can be made from such observational data. This is especially true when person-level variables contribute to self-selection of the media and content exposures through which the campaign must exert its effects. In this presentation we discuss how propensity score techniques can be used to correct for selection bias and obtain better estimates of campaign (i.e., “treatment”) effects that can be used to evaluate campaign success. Matching on propensity to have seen Medicare paid media was implemented using STATA module psmatch2. Covariates included in the analysis were age, gender, race, education, self-reported health status, level of prescription drug use, marital status, health decision-making style, and low income subsidy eligibility. Outcomes included OE awareness and behavioral reports of review and comparison of available insurance plans. In general, results supported the effectiveness of the campaign, but the magnitude of the effects was less than suggested by simple uncorrected association measures. Although illustrated for a social marketing campaign aimed at Medicare beneficiaries, the analytic strategy is broadly applicable to comparative effectiveness research studies that will be increasingly important in the current healthcare and health insurance environment.