Feature subset selection in conditional random fields for named entity

In the application of Conditional Random Fields (CRF), a huge number of features is typically taken into account. These models can deal with inter-dependent and correlated data with an enormous complexity. The application of feature subset selection is important to improve performance, speed and explainability. We present and compare filtering methods using information gain or X2 as well as an iterative approach for pruning features with low weights. The evaluation shows that with only 3% of the original number of features a 60% inference speed-up is possible. The F1 measure decreases only slightly.