Class TokenSample

java.lang.Object
opennlp.tools.tokenize.TokenSample
All Implemented Interfaces:
Serializable, Sample

public class TokenSample extends Object implements Sample
A TokenSample is text with token spans.
See Also:
  • Field Details

  • Constructor Details

    • TokenSample

      public TokenSample(String text, Span[] tokenSpans)
      Initializes a instance.
      Parameters:
      text - The text which contains the tokens. Must not be null.
      tokenSpans - The spans which mark the start and end of the tokens. Must not be null.
    • TokenSample

      public TokenSample(Detokenizer detokenizer, String[] tokens)
      Initializes a instance via a Detokenizer.
      Parameters:
      detokenizer - The text which contains the tokens. Must not be null.
      tokens - The tokens to be processed. Must not be null.
  • Method Details