|
Abstract:
|
In this work, we are interested in detecting anomalous activity in sparse traffic networks where the network is not directly observed. Given knowledge of what the node-to-node traffic in a network should be, any activity that differs significantly from this baseline would be considered anomalous. We propose two estimators for the actual rates of the network traffic matrix. The first is a robust method based on optimization of a $\ell_1$ loss function and the second is a Bayesian hierarchical model, which uses a variational EM algorithm to simultaneously approximate the observed individual traffic between the nodes and estimate the rate matrix. We show that by warm-starting the EM at the solution of the first method, we require much fewer EM iterations. Additionally, the probabilistic framework of this method allows us to naturally perform likelihood ratio tests to detect significant deviations from the baseline network. Through simulations, we show that our methods are robust to misspecification and demonstrate that they have superior performance over existing alternatives.
|
