Balancing computational and statistical efficiency is a modern challenge widespread in statistics, machine learning and data science. In this talk we discuss new methods for parameter estimation in time series and random fields which addresses this very challenge. Specifically, we propose a class of new pseudo-likelihood estimators which are order NlogN to compute, and yield parameter estimates with the optimal root N convergence under weaker assumptions than alternative methods. Our procedure is inspired from the Whittle likelihood, and as thus is based in the frequency domain, but we make important bias corrections to vastly improve performance. We also extend the procedure to include missing data and irregular spatial shapes, as well as non-linear, non-stationary and anisotropic stochastic processes. We demonstrate the applicability of our techniques to massive datasets across oceanography and the geosciences.