Panel count data appear in many clinical and observational studies, where one can only observe the counts of events between two consecutive observation times. When we consider time-varying covariates and coefficient effects, most of the previous studies focused on the proportional mean model because the likelihood function under the rate model involves intractable integration. However, the rate model is more realistic and efficient. Hence, we propose a semi-parametric MLE method under the rate model for panel count data with time-dependent covariates and time-varying coefficients. B-spline functions are employed to approximate time-dependent coefficients and an efficient Expectation-Maximization-type algorithm is developed to overcome the computational difficulty. The resulting estimators are shown to be consistent and asymptotically efficient. Monte Carlo simulation studies demonstrate that the proposed method enjoys desirable finite-sample properties. An application is provided.