Class AbstractDataIndexer
java.lang.Object
opennlp.tools.ml.model.AbstractDataIndexer
- All Implemented Interfaces:
DataIndexer
- Direct Known Subclasses:
OnePassDataIndexer, TwoPassDataIndexer
Abstract
DataIndexer implementation for collecting
event and context counts used in training.- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected int[][]The integer contexts associated with each unique event.private static final org.slf4j.Loggerprivate intprotected int[]The number of times an event occurred in the training data.protected String[]The names of the outcomes.protected int[]The integer outcome associated with each unique event.protected int[]The number of times each predicate occurred.protected String[]The predicate/context names.static final booleanstatic final Stringprotected TrainingParameters -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionint[][]intint[]String[]int[]int[]String[]float[][]protected List<ComparableEvent> Performs the data indexing.voidinit(TrainingParameters indexingParameters, Map<String, String> reportMap) Sets parameters used during the data indexing.protected intsortAndMerge(List<ComparableEvent> eventsToCompare, boolean sort) Sorts and uniques the array of comparable events and return the number of unique events.protected static String[]toIndexedStringArray(Map<String, Integer> labelToIndexMap) Utility method for creating aString[]from a map whose keys are labels (Strings) to be stored in the array and whose values are the indices (Integers) at which the corresponding labels should be inserted.protected static voidUpdates theMapof predicates and counter with the specified event contexts.Methods inherited from class Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface DataIndexer
index
-
Field Details
-
logger
private static final org.slf4j.Logger logger -
SORT_PARAM
- See Also:
-
SORT_DEFAULT
public static final boolean SORT_DEFAULT- See Also:
-
trainingParameters
-
reportMap
-
numEvents
private int numEvents -
contexts
protected int[][] contextsThe integer contexts associated with each unique event. -
outcomeList
protected int[] outcomeListThe integer outcome associated with each unique event. -
numTimesEventsSeen
protected int[] numTimesEventsSeenThe number of times an event occurred in the training data. -
predLabels
The predicate/context names. -
outcomeLabels
The names of the outcomes. -
predCounts
protected int[] predCountsThe number of times each predicate occurred.
-
-
Constructor Details
-
AbstractDataIndexer
public AbstractDataIndexer()
-
-
Method Details
-
init
Sets parameters used during the data indexing.- Specified by:
initin interfaceDataIndexer- Parameters:
indexingParameters- TheTrainingParametersto be used.reportMap- TheMapused for reporting.
-
getContexts
public int[][] getContexts()- Specified by:
getContextsin interfaceDataIndexer- Returns:
- Retrieves a 2-dimensional array whose first dimension is the event index and array this refers to contains the contexts for that event.
-
getNumTimesEventsSeen
public int[] getNumTimesEventsSeen()- Specified by:
getNumTimesEventsSeenin interfaceDataIndexer- Returns:
- Retrieves an array indexed by the event index indicating the number of times a particular event was seen.
-
getOutcomeList
public int[] getOutcomeList()- Specified by:
getOutcomeListin interfaceDataIndexer- Returns:
- Retrieves an array indicating the outcome index for each event.
-
getPredLabels
- Specified by:
getPredLabelsin interfaceDataIndexer- Returns:
- Retrieves an array of predicate/context names indexed by context index.
These indices are the value of the array returned by
DataIndexer.getContexts().
-
getOutcomeLabels
- Specified by:
getOutcomeLabelsin interfaceDataIndexer- Returns:
- Retrieves an array of outcome names indexed by outcome index.
-
getPredCounts
public int[] getPredCounts()- Specified by:
getPredCountsin interfaceDataIndexer- Returns:
- Retrieves an array of the count of each predicate in the events.
-
getNumEvents
public int getNumEvents()- Specified by:
getNumEventsin interfaceDataIndexer- Returns:
- Retrieves the number of total events indexed.
-
sortAndMerge
protected int sortAndMerge(List<ComparableEvent> eventsToCompare, boolean sort) throws InsufficientTrainingDataException Sorts and uniques the array of comparable events and return the number of unique events. This method will alter theeventsToComparelist.It does an in place sort, followed by an in place edit to remove duplicates.
- Parameters:
eventsToCompare- TheList<ComparableEvent>events used as input.sort- Whether to use sorting, or not.- Returns:
- The number of unique events in the specified list.
- Throws:
InsufficientTrainingDataException- Thrown if not enough events are provided- Since:
- maxent 1.2.6
-
index
protected List<ComparableEvent> index(ObjectStream<Event> events, Map<String, Integer> predicateIndex) throws IOExceptionPerforms the data indexing.Note: Make sure the
init(TrainingParameters, Map)method is called first.- Parameters:
events- AObjectStream<Event>of events used as input.predicateIndex- AMapproviding the data of a predicate index.- Throws:
IOException- Thrown if IO errors occurred during indexing.
-
update
-
toIndexedStringArray
Utility method for creating aString[]from a map whose keys are labels (Strings) to be stored in the array and whose values are the indices (Integers) at which the corresponding labels should be inserted.- Parameters:
labelToIndexMap- AMapthat holds labels to index positions.- Returns:
- The resulting
String[].
-
getValues
public float[][] getValues()- Specified by:
getValuesin interfaceDataIndexer- Returns:
- Retrieves the values associated with each event context or
nullif integer values are to be used.
-