Package de.dfki.lt.tools.tokenizer

Class Summary
AbbrevDescription AbbrevDescription extends Description.
CliticsDescription CliticsDescription extends Description.
Description Description is an abstract class that provides common methods to manage the content of description files.
FileTools FileTools provides static methods to work on files and stream.
JTok JTok is a low level tokenizer tool that recognizes paragraphs, sentences, tokens, punctuation, numbers, abbreviations, etc.
LanguageResource LanguageResource class manages the language-specific information needed by the tokenizer to process a document of that language.
NumbersDescription NumbersDescription extends Description.
PunctDescription PunctDescription extends Description.
TestJTok This is a test class for JTok.