In high-dimensional data problems, the sample covariance matrix of the predictors is often singular either due to correlations among the predictors or due to a n < < p setting. Most sufficient dimension reduction methods rely on the inverse of the sample covariance as part of the estimation process. To conquer the challenge brought by the singular or near-singular sample covariance matrix, we propose a pseudo estimation approach by artificially adding random noises to the observed data. We show that with a careful control of the added noises, the resulting estimator based on the perturbed data can still be consistent. In addition, a new variable selection procedure is proposed based on the pseudo estimator. The advantages of the proposed method are demonstrated by both simulation studies and real data analyses.