Parameter Estimation for Record Linkage

• Activity Number: 23
• Type: Topic Contributed
• Date/Time: Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Joshua Tokle*+
• Companies: U.S. Census Bureau
• Keywords: record linkage ; EM ; optimality

Abstract

For each pair of records from two files, the Fellegi-Sunter model for record linkage provides a matching weight or score that is an aggregate of similarity scores for the fields being compared. The matching weight is the likelihood conditional on (typically unobserved) match status. In this paper, I focus on accurate estimates of these conditional probabilities. The probabilities themselves can vary greatly by data source and are heavily dependent on typographical error in the quasi-identifying fields such as name, address and date-of-birth. In the Decennial Census, computing these probabilities automatically without training data in more than 400 contiguous regions yields increased accuracy of matching and reduces clerical review by as much as 2/3. In this talk I will discuss the probability models that have been applied in this setting. In particular, I will discuss EM fitting with little or no training data and possibly alternative models that may not be as 'optimal' but can still work well.