Malware Detection Using Nonparametric Bayesian Clustering and Classification Techniques

• Activity Number: 318
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Yimin Kao*+ and Brian J. Reich and Curtis Storlie
• Companies: Department of Statistics, North Carolina State University and North Carolina State University and Los Alamos National Laboratory
• Keywords: Classi cation ; Clustering ; Dynamic Trace ; Dirichlet Process Mixture

Abstract

Computer security requires statistical methods to quickly and accurately flag malicious software. In this talk, we propose a nonparametric Bayesian approach for clustering software and classifying software as benign and malicious. The analysis is based on the dynamic trace of instructions under first-order Markov assumption. Each row of the trace's transition matrix is modeled using the Dirichlet Process Mixture (DPM) model. The DPM clusters software within each class, and produces the Bayes factor which is used for classification. The novelty of the model is using this clustering algorithm to improve the classification accuracy. The simulation study shows that our method outperforms the Elastic Net Logistic (ENL) regression in classification performance under most of the scenarios, and that our model outperforms the Multiple Kernel Learning (MKL) method of clustering results. For the real data analysis, our method gives higher classification accuracy than the ENL method.