Class OnePassDataIndexer
java.lang.Object
opennlp.tools.ml.model.AbstractDataIndexer
opennlp.tools.ml.model.OnePassDataIndexer
- All Implemented Interfaces:
DataIndexer
- Direct Known Subclasses:
OnePassRealValueDataIndexer
A
DataIndexer for maxent model data which handles cutoffs for uncommon
contextual predicates and provides a unique integer index for each of the
predicates.- See Also:
-
Field Summary
FieldsFields inherited from class AbstractDataIndexer
contexts, numTimesEventsSeen, outcomeLabels, outcomeList, predCounts, predLabels, reportMap, SORT_DEFAULT, SORT_PARAM, trainingParameters -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptioncomputeEventCounts(ObjectStream<Event> eventStream, Map<String, Integer> predicatesInOut, int cutoff) Reads events from eventStream into a linked list.voidindex(ObjectStream<Event> eventStream) Performs the data indexing.Methods inherited from class AbstractDataIndexer
getContexts, getNumEvents, getNumTimesEventsSeen, getOutcomeLabels, getOutcomeList, getPredCounts, getPredLabels, getValues, index, init, sortAndMerge, toIndexedStringArray, update
-
Field Details
-
logger
private static final org.slf4j.Logger logger
-
-
Constructor Details
-
OnePassDataIndexer
public OnePassDataIndexer()
-
-
Method Details
-
index
Performs the data indexing.Note: Make sure the
DataIndexer.init(TrainingParameters, Map)method is called first.- Parameters:
eventStream- AObjectStream<Event>of events used as input.- Throws:
IOException- Thrown if IO errors occurred during indexing.
-
computeEventCounts
private List<Event> computeEventCounts(ObjectStream<Event> eventStream, Map<String, Integer> predicatesInOut, int cutoff) throws IOExceptionReads events from eventStream into a linked list. The predicates associated with each event are counted and any which occur at least cutoff times are added to the predicatesInOut map along with a unique integer index.- Parameters:
eventStream- anEventStreamvaluepredicatesInOut- aTObjectIntHashMapvaluecutoff- anintvalue- Returns:
- a
TLinkedListvalue - Throws:
IOException
-