Preventing overfitting in learning text patterns for document categorization
There is an increasing interest in categorizing texts using learning algorithms. While the majority of approaches rely on learning linear classifiers, there is also some interest in describing document categories by text patterns. We introduce a model for learning patterns for text categorization (the LPT-model) that does not rely on an attribute-value representation of documents but represents documents essentially as they are. Based on the LPT-model, we focus on learning patterns within a relatively simple pattern language. We compare different search heuristics and pruning methods known from various symbolic rule learners on a set of representative text categorization problems. The best results were obtained using the \textit{m}-estimate as search heuristics combined with the likelihood-ratio-statics for pruning. Even better results can be obtained, when replacing the likelihood-ratio-statics by a new measure for pruning; this we call \textit{l}-measure. In contrast to conventional measures for pruning, the \textit{l}-measure takes into account properties of the search space. For the entire collection see [Zbl 0977.68799].
- On the strength of hyperclique patterns for text categorization
- The use of bigrams to enhance text categorization
- Text categorization through multistrategy learning and visualization
- Meta-learning models for automatic textual document categorization
- scientific article; zbMATH DE number 1884348 (Why is no real title available?)
This page was built for publication: Preventing overfitting in learning text patterns for document categorization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2763413)
