The partial correlation function (PARCOR) provides a characterization of time series processes. We present a Bayesian PARCOR modeling approach for fast and accurate inference in multivariate non-stationary time series settings. Our formulation models the forward and backward PARCOR coefficients of a multivariate time series process using multivariate dynamic linear models. We obtain computationally efficient approximate inference of the variance-covariance matrices and the multivariate time-varying PARCOR coefficients. Approximate inference on the implied time-varying vector autoregressive (TV-VAR) coefficients can also be obtained using Whittle's algorithm. Similarly, approximate inference on functions of these parameters such as the multivariate time-frequency spectra, coherence, and partial coherence can also be obtained. A key aspect of the PARCOR approach is that it requires multivariate DLM representations of lower dimension than those required by commonly used multivariate non-stationary models such as TV-VARs. The performance of the PARCOR approach is shown in simulations and in the analysis of multivariate temporal data from neurosciences and environmental applications.