Class OnePassDataIndexer

java.lang.Object
opennlp.tools.ml.model.AbstractDataIndexer
opennlp.tools.ml.model.OnePassDataIndexer
All Implemented Interfaces:
DataIndexer
Direct Known Subclasses:
OnePassRealValueDataIndexer

public class OnePassDataIndexer extends AbstractDataIndexer
A DataIndexer for maxent model data which handles cutoffs for uncommon contextual predicates and provides a unique integer index for each of the predicates.
See Also:
  • Field Details

    • logger

      private static final org.slf4j.Logger logger
  • Constructor Details

    • OnePassDataIndexer

      public OnePassDataIndexer()
  • Method Details

    • index

      public void index(ObjectStream<Event> eventStream) throws IOException
      Performs the data indexing.

      Note: Make sure the DataIndexer.init(TrainingParameters, Map) method is called first.

      Parameters:
      eventStream - A ObjectStream<Event> of events used as input.
      Throws:
      IOException - Thrown if IO errors occurred during indexing.
    • computeEventCounts

      private List<Event> computeEventCounts(ObjectStream<Event> eventStream, Map<String,Integer> predicatesInOut, int cutoff) throws IOException
      Reads events from eventStream into a linked list. The predicates associated with each event are counted and any which occur at least cutoff times are added to the predicatesInOut map along with a unique integer index.
      Parameters:
      eventStream - an EventStream value
      predicatesInOut - a TObjectIntHashMap value
      cutoff - an int value
      Returns:
      a TLinkedList value
      Throws:
      IOException