Regularization is essential for analyzing the large and complex data that are generated in economics, neuroscience, astronomy, and many other fields. However, Lasso, Ridge Regression, Graphical Lasso, and other regularized methods depend on tuning parameters that are difficult to calibrate both in theory and in practice. In this talk, we present two approaches to this challenge. The first approach is based on a testing scheme and is to date the only method that is equipped with both fast algorithms and optimal finite sample guarantees. The second approach is based on the minimization of an objective function that avoids tuning parameters altogether. We show that, quite surprisingly, this estimator can be computed efficiently despite it being highly non-convex.