Uses of Interface
opennlp.tools.tokenize.Detokenizer
Packages that use Detokenizer
Package
Description
Experimental package related to converting various corpora to OpenNLP Format.
Experimental package related to the
letsmt corpus format.Package related to identifying sentence boundaries.
Contains classes related to finding token or words in a string.
-
Uses of Detokenizer in opennlp.tools.formats
Methods in opennlp.tools.formats that return DetokenizerModifier and TypeMethodDescriptionprotected DetokenizerDetokenizerSampleStreamFactory.createDetokenizer(DetokenizerParameter p) -
Uses of Detokenizer in opennlp.tools.formats.convert
Fields in opennlp.tools.formats.convert declared as DetokenizerModifier and TypeFieldDescriptionprivate final DetokenizerAbstractToSentenceSampleStream.detokenizerprivate final DetokenizerNameToTokenSampleStream.detokenizerprivate final DetokenizerPOSToTokenSampleStream.detokenizerConstructors in opennlp.tools.formats.convert with parameters of type DetokenizerModifierConstructorDescription(package private)AbstractToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<T> samples, int chunkSize) NameToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<NameSample> samples, int chunkSize) Initializes aNameToSentenceSampleStream.NameToTokenSampleStream(Detokenizer detokenizer, ObjectStream<NameSample> samples) Initializes aNameToTokenSampleStream.POSToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<POSSample> samples, int chunkSize) Initializes aPOSToSentenceSampleStream.POSToTokenSampleStream(Detokenizer detokenizer, ObjectStream<POSSample> samples) Initializes aPOSToTokenSampleStream. -
Uses of Detokenizer in opennlp.tools.formats.letsmt
Fields in opennlp.tools.formats.letsmt declared as DetokenizerModifier and TypeFieldDescriptionprivate final DetokenizerDetokenizeSentenceSampleStream.detokenizerConstructors in opennlp.tools.formats.letsmt with parameters of type DetokenizerModifierConstructorDescriptionDetokenizeSentenceSampleStream(Detokenizer detokenizer, ObjectStream<SentenceSample> samples) Initializes aDetokenizeSentenceSampleStream. -
Uses of Detokenizer in opennlp.tools.sentdetect
Constructors in opennlp.tools.sentdetect with parameters of type Detokenizer -
Uses of Detokenizer in opennlp.tools.tokenize
Classes in opennlp.tools.tokenize that implement DetokenizerFields in opennlp.tools.tokenize declared as DetokenizerModifier and TypeFieldDescriptionprivate final DetokenizerDetokenizerEvaluator.detokenizerTheDetokenizerused to create the predicted tokens.Constructors in opennlp.tools.tokenize with parameters of type DetokenizerModifierConstructorDescriptionDetokenizerEvaluator(Detokenizer detokenizer, DetokenEvaluationErrorListener... listeners) Initializes the current instance with the givenDetokenizer.TokenSample(Detokenizer detokenizer, String[] tokens) Initializes ainstancevia aDetokenizer.