High-Dimensional Forecasting for Web Data

• Activity Number: 45
• Type: Contributed
• Date/Time: Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Souvik Ghosh*+ and Deepak Agarwal
• Companies: LinkedIn Corp and LinkedIn
• Keywords: high dimensional time series ; forecasting ; web data

Abstract

Web data consists of user visits to different websites. Users are often described by categorical attributes like demographics, behavioral, profile information etc. For several applications like computational advertising, financial modeling, capacity planning and others, one has to forecast visit volume for arbitrary queries spanning subset of user attributes. The forecast horizons in such applications can range from a few hours to a few months. The curse of dimensionality, arbitrary time horizons and the requirement to obtain answers to queries interactively make it a challenging statistical problem. We develop an adaptive algorithm to solve this problem. The algorithm consists of two steps. First, an offline component where we select specific profiles (called trend lines) and fit time series models to them and second, an online component where given an arbitrary query we find its relation to the trend lines in real time and then use them to generate the forecast for the given query. The trend lines are adaptively chosen based on the distribution of the queries so as to minimize the forecasting error. We illustrate our method using the LinkedIn data.