Data from randomized controlled trials (RCTs) are used to infer valid individualized treatment rules (ITRs) using machine learning methods. However, RCTs are usually conducted under certain criteria, thus limiting the generalizability of ITRs to a broader population. Since patient's electronic health records (EHRs) document treatment prescriptions in the real world, transferring information in EHRs to RCTs could potentially improve the performance of ITRs. In this work, we propose a new domain adaptation method to learn ITRs by incorporating evidence from EHRs. We first pre-train "super" features from EHRs that summarize physicians' treatment decisions in the real world and then include super features to augment the feature space of the RCT and learn the optimal ITRs stratifying by these features. We adopt Q-learning and a modifed matched-learning algorithm for estimation. We present justifications of our proposed method and conduct simulation studies to demonstrate its superiority. Finally, we apply our method to transfer information learned from EHRs of T2D patients to improve learning individualized insulin therapies from a RCT.