The development and clinical implementation of evidence-based precision medicine strategies has become a realistic possibility, due to rapid accumulation of large-scale genomics and pharmacological data from diverse model systems: patients, cell-lines and drug perturbation studies. We introduce a novel Bayesian modeling framework called the Individualized theRapeutic indeX (iRx) model to integrate high-throughput pharmacogenomic data across model systems. Our iRx model achieves three main goals: first, it exploits the conserved biology between patients and cell-lines to calibrate therapeutic response of drugs in patients; second, it finds optimal cell line avatars as proxies for patient(s); and finally, it identifies key genomic drivers explaining cell line-patient similarities. This is achieved through a semi-supervised learning approach, that conflates (unsupervised) sparse latent factor models with (supervised) penalized regression techniques. We illustrate and validate our approach using two existing clinical trial datasets. We show that our iRx model improves accuracy compared to existing alternative approaches, both in simulation scenarios as well as real clinical examples.