Class SimpleTokenizer
java.lang.Object
org.apache.uima.analysis_component.AnalysisComponent_ImplBase
org.apache.uima.analysis_component.Annotator_ImplBase
org.apache.uima.analysis_component.CasAnnotator_ImplBase
opennlp.uima.tokenize.AbstractTokenizer
opennlp.uima.tokenize.SimpleTokenizer
- All Implemented Interfaces:
org.apache.uima.analysis_component.AnalysisComponent
An OpenNLP-based Simple Tokenizer annotator.
Mandatory parameters:
| Type | Name | Description |
|---|---|---|
| String | opennlp.uima.SentenceType |
The full name of the sentence type |
| String | opennlp.uima.TokenType |
The full name of the token type |
-
Field Summary
FieldsFields inherited from class AbstractTokenizer
context, logger, name, sentenceType, tokenType -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprotected voidpostProcessAnnotations(Span[] detectedNames, org.apache.uima.cas.text.AnnotationFS[] nameAnnotations) protected Span[]tokenize(org.apache.uima.cas.CAS cas, org.apache.uima.cas.text.AnnotationFS sentence) Methods inherited from class AbstractTokenizer
initialize, process, typeSystemInitMethods inherited from class org.apache.uima.analysis_component.CasAnnotator_ImplBase
getRequiredCasInterface, processMethods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase
getCasInstancesRequired, hasNext, nextMethods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase
batchProcessComplete, collectionProcessComplete, destroy, getContext, getLogger, getResultSpecification, reconfigure, setResultSpecification
-
Field Details
-
tokenizer
The OpenNLP simple tokenizer.
-
-
Constructor Details
-
SimpleTokenizer
public SimpleTokenizer()Initializes the current instance.Note: Use
AbstractTokenizer.initialize(UimaContext)to initialize this instance. Not use the constructor.
-
-
Method Details
-
tokenize
protected Span[] tokenize(org.apache.uima.cas.CAS cas, org.apache.uima.cas.text.AnnotationFS sentence) - Specified by:
tokenizein classAbstractTokenizer
-
postProcessAnnotations
protected void postProcessAnnotations(Span[] detectedNames, org.apache.uima.cas.text.AnnotationFS[] nameAnnotations) - Specified by:
postProcessAnnotationsin classAbstractTokenizer
-