Keywords: Biostatistics, EM algorithms, Self-consistency, cancer models
Developing estimation algorithms with semiparametric and other high-dimensional models (that contain high-dimensional parameters) is a challenge except for rare special cases. Elucidating intrinsic self-consistency structures of such models provides a way to overcome the computational challenge by reducing the model to a simpler one that, when subjected to a self-consistency condition, turns into the target original model. Some general theory and examples of the development of estimation algorithms are presented for a variety of models, including categorical data models and univariate (single response variable) and multivariate (multiple correlated response variables) survival models, including dynamic joint models. In medicine, diseases often do not generate directly observable data that would allow direct inference on the progression mechanism and targeted treatment development. More often, disease progression mechanisms remain latent, and observed phenotype offers limited information to be decoded. The proposed theory and algorithms are used to formulate a variety of disease models and apply them to analyze real large medical datasets.