WITHDRAWN: Words Segmentation in Chinese Language Processing

• Activity Number: 433
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Xinxin Shu and Annie Qu and Junhui Wang and Xiaotong Shen
• Companies: Merck and University of Illinois at Urbana-Champaign and University of Illinois at Chicago and University of Minnesota
• Keywords: Language processing ; words segmentation ; cutting-plane algorithm

Abstract

The digital information in this "Big Data" era has become a essential part of modern life, from scientific research, entertainment business, product marketing to national security protection. So developing fast automatic process of information extraction becomes extremely demanding. Chinese language is the second popular language among all internet users but is still severely under studied. In the paper, we address words segmentation arisen from Chinese language processing. One major challenge is that Chinese language is a highly context-dependent language. Segmentation for Chinese documents consequently becomes crucial. We propose an optimization model with linguistically-embedded features and computational feasible losses. The proposed model is investigated through the Peking university corpus. The result shows that the proposed method delivers better performance against the current two top performers.