Observational studies with variables of high dimension often raise challenges for estimation of treatment effects, upon which proper control of confounding is critically required. For efficiently estimating treatment effects, especially by doubly robust approach, improvements can be gained through covariates related both to the propensity scores and outcome responses. However, selecting covariates also incorporating balanced features in supporting the ignorability condition of propensity scores has not been investigated widely. We propose methods of covariate selection by two focals (1) prioritizing covariates related to treatments and outcomes via marginal and subset selection steps (2) hybrid information criterion aimed to capture true propensity score and outcome models, while balanced propensity scores are kept optimally. The steps of prioritizing covariates perform feature screening so that sparse parameters are ruled out. The hybrid information criterion binding the loss functions of propensity scores and the outcome models with extra trade off terms based on a deviance of distributions. We evaluate our selected models to perform estimation of treatment effects via inverse probability weighting and doubly robust estimation. Numerical examples are presented for the methods.