Temporal dynamic graphical models are of great interest in the literature to describe time-varying interactions among a group of entities such as individuals in social networks and genes or species in biological networks. In many applications, it is critical to understand the temporal dynamics of multivariate time-dependent data, but existing time-varying graphical models treat graphs at different time points as independent and hence fail to account the correlation between sequential graphs. Therefore, we propose a novel regularization method that can estimate both the time-varying graphical structure among the nodes and as well as the time dependence between graphs using a regression-based approach. To produce sparse graphs, the new method employs a regularization approach to identify links and capture patterns in temporal dynamics. In order to solve our problem efficiently, we have implemented an alternating direction method of multipliers (ADMM) algorithm, and the empirical performance is demonstrated through a set of numerical examples including a case study of the changes in species abundances in an ecological system.