Class AbstractToSentenceSampleStream<T>

java.lang.Object
opennlp.tools.util.FilterObjectStream<T, SentenceSample>
opennlp.tools.formats.convert.AbstractToSentenceSampleStream<T>
All Implemented Interfaces:
AutoCloseable, ObjectStream<SentenceSample>
Direct Known Subclasses:
NameToSentenceSampleStream, POSToSentenceSampleStream

public abstract class AbstractToSentenceSampleStream<T> extends FilterObjectStream<T, SentenceSample>
  • Field Details

    • detokenizer

      private final Detokenizer detokenizer
    • chunkSize

      private final int chunkSize
  • Constructor Details

    • AbstractToSentenceSampleStream

      AbstractToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<T> samples, int chunkSize)
      Parameters:
      detokenizer - The Detokenizer to use. Must not be null.
      samples - The samples as input. Must not be null.
      chunkSize - The size of chunks. Must be equal to or greater than 0.
      Throws:
      IllegalArgumentException - Thrown if parameters are invalid.
  • Method Details

    • toSentence

      protected abstract String[] toSentence(T sample)
    • read

      public SentenceSample read() throws IOException
      Description copied from interface: ObjectStream
      Returns the next T object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.
      Returns:
      The next object or null to signal that the stream is exhausted.
      Throws:
      IOException - Thrown if there is an error during reading.