de.dfki.lt.hog.modules.test
Class SampleTokenizer

java.lang.Object
  extended by de.dfki.lt.hog.Module
      extended by de.dfki.lt.hog.modules.test.SampleTokenizer

public class SampleTokenizer
extends Module

Tokenizer is a StringTokenizer embedded in a Memphis Module. It is configured through a Properties object which sets its language, depth, delimiters and whether this delimiters are also tokens. It tokenizes through the process(java.lang.String) and processDOMTree(org.w3c.dom.Node) methods.

See Also:
java.Utility.StringTokenizer

Field Summary
static java.lang.String TOKENIZER_DELIMITERS_PROP
          Key for delimiters in configuration.
static java.lang.String TOKENIZER_RETURN_DELIMITERS_PROP
          Key for return delimiters property in configuration.
 
Fields inherited from class de.dfki.lt.hog.Module
CONFIG_SEARCH_PATH_PROP, LINE_SEPARATOR, MODULE_DEPTH_PROP, MODULE_LANGUAGE_PROP, MODULE_NAME_PROP, MODULE_ROOTELEMENT_PROP, moduleLog, NO, NONE, ROOT, YES
 
Constructor Summary
SampleTokenizer(java.util.Properties props)
          Creates a Tokenizer module with given Properties
 
Method Summary
 java.lang.String process(java.lang.String processString)
          Tokenizes a given String depending on configuration.
 org.w3c.dom.Node processDOMTree(org.w3c.dom.Node domTree)
           
 
Methods inherited from class de.dfki.lt.hog.Module
getAbsolutOrRelativePath, getConfig, getFileProperty, getMoCoMan, getModuleDepth, getModuleLanguage, getModuleName, getModuleRootElement, getProperty, getProperty, init, searchConfigFile, searchConfigFile, setConfig, setMoCoMan, setModuleDepth, setModuleLanguage, setModuleName, setModuleRootElement, setProperty, shutdown, suitable
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

TOKENIZER_DELIMITERS_PROP

public static final java.lang.String TOKENIZER_DELIMITERS_PROP
Key for delimiters in configuration.

See Also:
Constant Field Values

TOKENIZER_RETURN_DELIMITERS_PROP

public static final java.lang.String TOKENIZER_RETURN_DELIMITERS_PROP
Key for return delimiters property in configuration.

See Also:
Constant Field Values
Constructor Detail

SampleTokenizer

public SampleTokenizer(java.util.Properties props)
Creates a Tokenizer module with given Properties

Parameters:
props - properties for this Tokenizer
Method Detail

process

public java.lang.String process(java.lang.String processString)
Tokenizes a given String depending on configuration.

Specified by:
process in class Module
Parameters:
processString - String to tokenize
Returns:
XML string containing tokenized document

processDOMTree

public org.w3c.dom.Node processDOMTree(org.w3c.dom.Node domTree)
Specified by:
processDOMTree in class Module