Keywords: ICD, administrative data, counterfactual, time series
In most administrative health datasets, International Classification of Diseases (ICD) diagnosis codes are used to classify patient morbidities. U.S. healthcare providers switched from ICD version 9 to 10 codes in October 2015, which may bias measures of temporal change in disease prevalence crossing the switch. We present a method for evaluating that bias.
Monthly changes in the prevalence of a diagnosis may fluctuate from year to year, with possible secular and cyclical trends. We examined whether change across the ICD switch was atypical for a diagnosis by constructing counterfactual scenarios if the switch did not happen and comparing the sampling distribution of the scenarios with the observed change.
We developed a multi-year time series model of diagnosis prevalence to create the counterfactual scenario and calculated the mean absolute scaled error (MASE) of a 3-month forecast beginning in Oct. 2015 to quantify the difference between the counterfactual and the observed prevalence. We create the MASE sampling distribution from rolling origin 3-month forecasts across a diagnosis’ historical data. We illustrate the method in the context of pneumococcal vaccine impact studies.