Class SentenceDetectorME

java.lang.Object
opennlp.tools.sentdetect.SentenceDetectorME
All Implemented Interfaces:
Probabilistic, SentenceDetector

public class SentenceDetectorME extends Object implements SentenceDetector, Probabilistic
A sentence detector for splitting up raw text into sentences.

A maximum entropy model is used to evaluate end-of-sentence characters in a string to determine if they signify the end of a sentence.

  • Field Details

  • Constructor Details

    • SentenceDetectorME

      public SentenceDetectorME(String language) throws IOException
      Initializes the sentence detector by downloading a default model.
      Parameters:
      language - The language of the sentence detector.
      Throws:
      IOException - Thrown if the model cannot be downloaded or saved.
    • SentenceDetectorME

      public SentenceDetectorME(SentenceModel model)
      Initializes the current instance.
      Parameters:
      model - the SentenceModel
    • SentenceDetectorME

      public SentenceDetectorME(SentenceModel model, Dictionary abbDict)
      Instantiates a SentenceDetectorME with an existing SentenceModel.
      Parameters:
      model - The SentenceModel to be used.
      abbDict - The Dictionary to be used. It must fit the language of the model. Its entries are read once, here; later changes to the abbDict instance do not affect this detector.
    • SentenceDetectorME

      @Deprecated public SentenceDetectorME(SentenceModel model, Factory factory)
      Deprecated.
      Use a SentenceDetectorFactory to extend SentenceDetector functionality.
  • Method Details

    • getAbbreviations

      private static Set<String> getAbbreviations(Dictionary abbreviations)
    • sentDetect

      public String[] sentDetect(CharSequence s)
      Detects sentences in given input CharSequence..
      Specified by:
      sentDetect in interface SentenceDetector
      Parameters:
      s - The CharSequence. to be processed.
      Returns:
      A string array containing individual sentences as elements.
    • getFirstWS

      private int getFirstWS(CharSequence s, int pos)
    • getFirstNonWS

      private int getFirstNonWS(CharSequence s, int pos)
    • sentPosDetect

      public Span[] sentPosDetect(CharSequence s)
      Detects the position of the first words of sentences in a CharSequence.
      Specified by:
      sentPosDetect in interface SentenceDetector
      Parameters:
      s - The CharSequence to be processed.
      Returns:
      An span array containing the positions of the end index of every sentence.
    • probs

      public double[] probs()
      Retrieves the probabilities of the last decoded sequence. The sequence was determined based on the previous call to sentDetect(CharSequence).
      Specified by:
      probs in interface Probabilistic
      Returns:
      An array with the same number of probabilities as tokens were sent to sentDetect(CharSequence) when it was last called. If not applicable, an empty array is returned.
    • getSentenceProbabilities

      @Deprecated(forRemoval=true, since="2.5.5") public double[] getSentenceProbabilities()
      Deprecated, for removal: This API element is subject to removal in a future version.
      Use probs() instead.
      Returns:
      The probability for each sentence returned for the most recent call to sentDetect(CharSequence). If not applicable, an empty array is returned.
    • isAcceptableBreak

      protected boolean isAcceptableBreak(CharSequence s, int fromIndex, int candidateIndex)
      Allows subclasses to check an overzealous (read: poorly trained) model from flagging obvious non-breaks as breaks based on some boolean determination of a break's acceptability.

      Note: The implementation always returns true if no abbreviation dictionary is available for the underlying model.

      Only a bounded region of s is consulted. An abbreviation occurrence starting at p with length L can veto the break only when it starts at or before the candidate and reaches it, that is when candidateIndex - L <= p <= candidateIndex. Positions outside that window are irrelevant by construction, so the decision costs time proportional to the longest dictionary entry rather than to the length of s.

      A candidateIndex greater than s.length() cannot carry an abbreviation, so the break is accepted rather than raising an IndexOutOfBoundsException. A candidateIndex equal to s.length() is evaluated like any other index.

      Parameters:
      s - the CharSequence in which the break occurred.
      fromIndex - the start of the segment currently being evaluated.
      candidateIndex - the index of the candidate sentence ending. A value below fromIndex, negative or greater than s.length() is accepted as a break.
      Returns:
      true if the break is acceptable, false otherwise.
    • isApostrophe

      private static boolean isApostrophe(char c)
      Parameters:
      c - The character to check.
      Returns:
      true if the character represents an apostrophe, false otherwise.
    • train

      public static SentenceModel train(String languageCode, ObjectStream<SentenceSample> samples, SentenceDetectorFactory sdFactory, TrainingParameters mlParams) throws IOException
      Starts a training of a SentenceModel with the given parameters.
      Parameters:
      languageCode - The ISO language code to train the model. Must not be null.
      samples - The ObjectStream of SentenceSample used as input for training.
      sdFactory - The SentenceDetectorFactory for creating related objects as defined via mlParams.
      mlParams - The TrainingParameters for the context of the training process.
      Returns:
      A valid, trained SentenceModel instance.
      Throws:
      IOException - Thrown if IO errors occurred.