We consider scenarios in which the likelihood function for a semiparametric regression model factors into separate components, with an efficient estimator of the regression parameter available for each component. An optimal weighted combination of the component estimators, named an ensemble estimator, may be employed as an overall estimate, and may be fully efficient. This approach is useful when the full likelihood is difficult to maximize but the components are easy to maximize. As a motivating example we consider Cox regression with prospective doubly censored data, in which the likelihood factors into a current status data likelihood and a left truncated right censored data likelihood. Variable selection is important in such regression modelling but the applicability of existing methods is unclear in the ensemble approach. We propose ensemble variable selection using the least squares approximation technique on the ensemble estimator, followed by ensemble re-estimation under the selected model. The estimator from the ensemble procedure has the oracle property such that the set of nonzero parameters is recovered and semiparametric efficiency is achieved for this parameter set.