de.dfki.lt.hog.modules.test
Class SampleTokenizer
java.lang.Object
de.dfki.lt.hog.Module
de.dfki.lt.hog.modules.test.SampleTokenizer
public class SampleTokenizer
- extends Module
Tokenizer
is a StringTokenizer
embedded
in a Memphis Module
. It is configured through a
Properties
object which sets its language,
depth, delimiters and whether this delimiters are also tokens.
It tokenizes through the process(java.lang.String)
and
processDOMTree(org.w3c.dom.Node)
methods.
- See Also:
java.Utility.StringTokenizer
Fields inherited from class de.dfki.lt.hog.Module |
CONFIG_SEARCH_PATH_PROP, LINE_SEPARATOR, MODULE_DEPTH_PROP, MODULE_LANGUAGE_PROP, MODULE_NAME_PROP, MODULE_ROOTELEMENT_PROP, moduleLog, NO, NONE, ROOT, YES |
Constructor Summary |
SampleTokenizer(java.util.Properties props)
Creates a Tokenizer module with given
Properties |
Method Summary |
java.lang.String |
process(java.lang.String processString)
Tokenizes a given String depending on configuration. |
org.w3c.dom.Node |
processDOMTree(org.w3c.dom.Node domTree)
|
Methods inherited from class de.dfki.lt.hog.Module |
getAbsolutOrRelativePath, getConfig, getFileProperty, getMoCoMan, getModuleDepth, getModuleLanguage, getModuleName, getModuleRootElement, getProperty, getProperty, init, searchConfigFile, searchConfigFile, setConfig, setMoCoMan, setModuleDepth, setModuleLanguage, setModuleName, setModuleRootElement, setProperty, shutdown, suitable |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
TOKENIZER_DELIMITERS_PROP
public static final java.lang.String TOKENIZER_DELIMITERS_PROP
- Key for delimiters in configuration.
- See Also:
- Constant Field Values
TOKENIZER_RETURN_DELIMITERS_PROP
public static final java.lang.String TOKENIZER_RETURN_DELIMITERS_PROP
- Key for return delimiters property in configuration.
- See Also:
- Constant Field Values
SampleTokenizer
public SampleTokenizer(java.util.Properties props)
- Creates a
Tokenizer
module with given
Properties
- Parameters:
props
- properties for this Tokenizer
process
public java.lang.String process(java.lang.String processString)
- Tokenizes a given
String
depending on configuration.
- Specified by:
process
in class Module
- Parameters:
processString
- String
to tokenize
- Returns:
- XML string containing tokenized document
processDOMTree
public org.w3c.dom.Node processDOMTree(org.w3c.dom.Node domTree)
- Specified by:
processDOMTree
in class Module