Reducing the Structure of Statistical Models for Probabilistic Record Linkage

• Activity Number: 419
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 2:00 PM to 3:50 PM
• Sponsor: SSC
• Author(s): Abel C. Dasylva*+ and Sanjoy Sinha
• Companies: Statistics Canada and Carleton University
• Keywords: record-linkage ; expectation maximization ; nonparametric ; correlation ; Fellegi-Sunter ; identifiability

Abstract

Fellegi and Sunter have suggested the conditional independence assumption among all linkage variables to facilitate the estimation of linkage weights. However this solution ignores the interactions that are observed among linkage variables in actual datasets. In this work, we will present new statistical models with interactions, including nonparametric models, i.e. models with an arbitrary correlation structure. The proposed models are discussed, including the question of their identifiability and estimation with data-driven algorithms such as Expectation Maximization (EM). The proposed models are also compared to models based on log-linear mixtures with interactions.