Feature Abstraction for Lightweight and Accurate Chinese Word Segmentation

L Tian, X Qiu, X Huang - 2013 International Conference on …, 2013 - ieeexplore.ieee.org
L Tian, X Qiu, X Huang
2013 International Conference on Asian Language Processing, 2013ieeexplore.ieee.org
Chinese word segmentation (CWS) is an important and necessary problem to analyze
Chinese texts. The state-of-art CWS systems are mostly based on sequence labeling
algorithm and use the discriminative model with millions of overlapping binary features.
However, there are few works on porting these systems to the devices with limited
computing capacity and memory. In this paper, we focus on two challenges in Chinese word
segmentation:(1) low accuracy of out-of-vocabulary word and (2) huge feature space. To …
Chinese word segmentation (CWS) is an important and necessary problem to analyze Chinese texts. The state-of-art CWS systems are mostly based on sequence labeling algorithm and use the discriminative model with millions of overlapping binary features. However, there are few works on porting these systems to the devices with limited computing capacity and memory. In this paper, we focus on two challenges in Chinese word segmentation: (1) low accuracy of out-of-vocabulary word and (2) huge feature space. To resolve these two difficult problems, we propose a method to abstract the original input on both character and feature levels. We group the "similar'' features to generate more abstract representation. Experimental results show that feature abstraction can greatly reduce the feature space with a comparable performance.
ieeexplore.ieee.org
Showing the best result for this search. See all results