Selection and Verification Propensity Matching in External Validation Studies

• Activity Number: 191 - Causal Inference
• Type: Contributed
• Date/Time: Tuesday, August 4, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Biometrics Section
• Author(s): Donna Pauler Ankerst* and Yiyao Chen and Bryan E Shepherd and Ruth Pfeiffer
• Companies: Technical University of Munich and Technical University of Munich and Vanderbilt University and National Cancer Institute
• Keywords: selection bias; verification bias; AUC; prostate cancer; risk prediction; propensity matching

Abstract

We develop a propensity-weighted method for adjusting for differential selection and verification bias between training and validation cohorts. We used the Prostate, Lung, Colorectal, and Ovarian Trial (PLCO) for training a prostate cancer risk model (30245 participants,12.6% verified by biopsy) and the Selenium and Vitamin E Cancer Prevention Trial (SELECT) for validation (33982,12.7%). We calculated the probability of being in PLCO versus SELECT by logistic regression on participants from both trials with PLCO as the outcome using the six established prostate cancer risk factors. We similarly calculated the probability of verification by biopsy among the PLCO verified and unverified participants. Multiples of these probabilities formed propensity scores to weight validation metrics calculated on verified participants in SELECT, including the area-under-the-receiver-operating-characteristic-curve (AUC). Propensity-matching deteriorated the AUC of 0.675 from the verified-only SELECT analysis to 0.603, and similarly for other calibration metrics. This could be attributable to changes in the prostate biopsy technique that occurred during the time period between the two trials.