A flexible approach to build stationary time-dependent processes exploits the concept of conjugacy in a Bayesian framework: the transition law of the process is defined as the predictive distribution of an underlying Bayesian model. If the model is conjugate, the transition kernel can be analytically derived, making the approach particularly appealing. We aim at achieving such a convenient mathematical tractability in the context of completely random measures (CRMs), i.e. when the variables exhibiting a time dependence are CRMs. In order to take advantage of the conjugacy, we consider the wide family of exponential completely random measures. This leads to a simple description of the process which has a autoregressive structure. The proposed process can be straightforwardly employed to extend CRM-based Bayesian nonparametric models such as feature allocation models to time-dependent data. These processes can be applied to problems from modern real life applications in very different fields, from computer science to biology. Here, we develop a dependent latent feature model for the identification of features in images and a dynamic Poisson factor analysis for topic modelling.