CSP 2017 Online Program

Designing Automated Workflows for Model Selection and Optimization (303379)

Keywords: Model Selection, automation, data science, machine learning, gradient boosting, random forests

Whether you just got a new project for work or you are hacking through a sanitized dataset for your hobby, competition, or consulting project, you can relate to the struggle of grinding through many models in both the exploratory and the model tuning phases of your process. How can we automate exploration of model construction methods to get started more quickly? How can we reduce the need for human oversight and input for iterative tests of model construction methods hyper parameters? Automating the model selection and optimization process provides opportunities to gain advantages for saving time, providing data science as a service, and increasing performance of the final model. Presented are automation tools to perform “pre-packaged experiments” to improve model performance. Using these tools, workflows will be demonstrated on real data to show how we can construct many models automatically and begin thinking about algorithmic strategies to use performance metrics to inform model selection and further testing of parameters to approach an optimal model quickly. Strategies for storing models and performance metrics reports will be covered to demonstrate practices for applying practical considerations and human oversight for model comparison post-hoc without interrupting the workflow or restricting the set of models constructed during automated selection and optimization.