Class GISTrainer
- All Implemented Interfaces:
Trainer, EventTrainer
The reference paper for this implementation was Adwait Ratnaparkhi's tech report at the University of Pennsylvania's Institute for Research in Cognitive Science, and is available at ftp://ftp.cis.upenn.edu/pub/ircs/tr/97-08.ps.Z.
The slack parameter used in the above implementation has been removed by default from the computation and a method for updating with Gaussian smoothing has been added per Investigating GIS and Smoothing for Maximum Entropy Taggers, Clark and Curran (2002). http://acl.ldc.upenn.edu/E/E03/E03-1071.pdf.
The slack parameter can be used by setting useSlackParameter to true.
Gaussian smoothing can be used by setting useGaussianSmoothing to true.
A Prior can be used to train models which converge to the distribution which minimizes the
relative entropy between the distribution specified by the empirical constraints of the training
data and the specified prior. By default, the uniform distribution is used as the prior.
-
Nested Class Summary
Nested Classes -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate doubleprivate int[][]Records the array of predicates seen in each event.private EvalParametersInitial probability for all outcomes.private static final booleanprivate static final Stringprivate static final doubleprivate static final Stringprivate doublestatic final doublestatic final Stringprivate static final org.slf4j.Loggerstatic final Stringprivate MutableContext[][]Stores the expected values of the features based on the current modelsprivate intNumber of outcomes.private intNumber of predicates.private int[]Records the num of times an event has been seen for each event i, in context[i].private intNumber of unique events which occurred in the event set.private MutableContext[]Stores the observed expected values of the features based on training data.private String[]Stores the String names of the outcomes.private int[]List of outcomes for each event i, in context[i].private MutableContext[]Stores the estimated parameter value of each predicate during iterationprivate String[]Stores the String names of the predicates.private PriorThis is the prior distribution that the model uses for training.private doubleprivate static final booleanprivate static final doubleprivate static final Stringprivate static final StringIf we are using smoothing, this is used as the "number" of times we want the trainer to imagine that it saw a feature that it actually didn't see.private booleanSpecified whether parameter updates should prefer a distribution of parameters which is gaussian.private booleanSpecifies whether unseen context/outcome pairs should be estimated as occur very infrequently.private float[][]The value associated with each context.Fields inherited from class AbstractEventTrainer
DATA_INDEXER_ONE_PASS_REAL_VALUE, DATA_INDEXER_ONE_PASS_VALUE, DATA_INDEXER_PARAM, DATA_INDEXER_TWO_PASS_VALUEFields inherited from class AbstractTrainer
reportMap, trainingConfiguration, trainingParametersFields inherited from interface EventTrainer
EVENT_VALUE -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptiondoTrain(DataIndexer indexer) private voidfindParameters(int iterations, double correctionConstant) private doublegaussianUpdate(int predicate, int oid, double correctionConstant) private StopCriteria<Double> getStopCriteria(TrainingConfiguration trainingConfig) Get theStopCriteriaassociated with thisTrainer.private TrainingProgressMonitorgetTrainingProgressMonitor(TrainingConfiguration trainingConfig) Get theTrainingProgressMonitorassociated with thisTrainer.voidinit(TrainingParameters trainingParameters, Map<String, String> reportMap) booleanprivate doublenextIteration(double correctionConstant, CompletionService<GISTrainer.ModelExpectationComputeTask> completionService, int iteration) voidsetGaussianSigma(double sigmaValue) Sets whether this trainer will use smoothing while training the model.voidsetSmoothing(boolean smooth) Sets whether this trainer will use smoothing while training the model.voidsetSmoothingObservation(double timesSeen) Sets whether this trainer will use smoothing while training the model.trainModel(int iterations, DataIndexer di) Trains a model using the GIS algorithm.trainModel(int iterations, DataIndexer di, int threads) Trains a model using the GIS algorithm.trainModel(int iterations, DataIndexer di, Prior modelPrior, int threads) Trains a model using the GIS algorithm.trainModel(ObjectStream<Event> eventStream) Trains a model using the GIS algorithm, assuming 100 iterations and no cutoff.trainModel(ObjectStream<Event> eventStream, int iterations, int cutoff) Trains a GIS model on the event in the specified event stream, using the specified number of iterations and the specified count cutoff.Methods inherited from class AbstractEventTrainer
getDataIndexer, train, train, validateMethods inherited from class AbstractTrainer
addToReport, getAlgorithm, getCutoff, getIterations, getTrainingConfiguration, init
-
Field Details
-
logger
private static final org.slf4j.Logger logger -
LOG_LIKELIHOOD_THRESHOLD_PARAM
- See Also:
-
LOG_LIKELIHOOD_THRESHOLD_DEFAULT
public static final double LOG_LIKELIHOOD_THRESHOLD_DEFAULT- See Also:
-
llThreshold
private double llThreshold -
useSimpleSmoothing
private boolean useSimpleSmoothingSpecifies whether unseen context/outcome pairs should be estimated as occur very infrequently. -
useGaussianSmoothing
private boolean useGaussianSmoothingSpecified whether parameter updates should prefer a distribution of parameters which is gaussian. -
sigma
private double sigma -
_smoothingObservation
private double _smoothingObservation -
numUniqueEvents
private int numUniqueEventsNumber of unique events which occurred in the event set. -
numPreds
private int numPredsNumber of predicates. -
numOutcomes
private int numOutcomesNumber of outcomes. -
contexts
private int[][] contextsRecords the array of predicates seen in each event. -
values
private float[][] valuesThe value associated with each context. If null then context values are assumes to be 1. -
outcomeList
private int[] outcomeListList of outcomes for each event i, in context[i]. -
numTimesEventsSeen
private int[] numTimesEventsSeenRecords the num of times an event has been seen for each event i, in context[i]. -
outcomeLabels
Stores the String names of the outcomes. The GIS only tracks outcomes as ints, and so this array is needed to save the model to disk and thereby allow users to know what the outcome was in human understandable terms. -
predLabels
Stores the String names of the predicates. The GIS only tracks predicates as ints, and so this array is needed to save the model to disk and thereby allow users to know what the outcome was in human understandable terms. -
observedExpects
Stores the observed expected values of the features based on training data. -
params
Stores the estimated parameter value of each predicate during iteration -
modelExpects
Stores the expected values of the features based on the current models -
prior
This is the prior distribution that the model uses for training. -
evalParams
Initial probability for all outcomes. -
MAXENT_VALUE
- See Also:
-
SMOOTHING_PARAM
If we are using smoothing, this is used as the "number" of times we want the trainer to imagine that it saw a feature that it actually didn't see. Defaulted to 0.1.- See Also:
-
SMOOTHING_DEFAULT
private static final boolean SMOOTHING_DEFAULT- See Also:
-
SMOOTHING_OBSERVATION_PARAM
- See Also:
-
SMOOTHING_OBSERVATION
private static final double SMOOTHING_OBSERVATION- See Also:
-
GAUSSIAN_SMOOTHING_PARAM
- See Also:
-
GAUSSIAN_SMOOTHING_DEFAULT
private static final boolean GAUSSIAN_SMOOTHING_DEFAULT- See Also:
-
GAUSSIAN_SMOOTHING_SIGMA_PARAM
- See Also:
-
GAUSSIAN_SMOOTHING_SIGMA_DEFAULT
private static final double GAUSSIAN_SMOOTHING_SIGMA_DEFAULT- See Also:
-
-
Constructor Details
-
GISTrainer
public GISTrainer()Initializes aGISTrainer.Note:
The resulting instance does not print progress messages about training to STDOUT.
-
-
Method Details
-
isSortAndMerge
public boolean isSortAndMerge()- Specified by:
isSortAndMergein classAbstractEventTrainer
-
init
- Specified by:
initin interfaceTrainer- Overrides:
initin classAbstractTrainer- Parameters:
trainingParameters- TheTrainingParametersto use.reportMap- TheMapinstance used as report map.
-
doTrain
- Specified by:
doTrainin classAbstractEventTrainer- Throws:
IOException
-
setSmoothing
public void setSmoothing(boolean smooth) Sets whether this trainer will use smoothing while training the model.Note:
This can improve model accuracy, though training will potentially take longer and use more memory. Model size will also be larger.- Parameters:
smooth-trueif smoothing is desired,falseif not.
-
setSmoothingObservation
public void setSmoothingObservation(double timesSeen) Sets whether this trainer will use smoothing while training the model.Note:
This can improve model accuracy, though training will potentially take longer and use more memory. Model size will also be larger.- Parameters:
timesSeen- The "number" of times we want the trainer to imagine it saw a feature that it actually didn't see
-
setGaussianSigma
public void setGaussianSigma(double sigmaValue) Sets whether this trainer will use smoothing while training the model.Note:
This can improve model accuracy, though training will potentially take longer and use more memory. Model size will also be larger.- Parameters:
sigmaValue- The Gaussian sigma value used for smoothing.
-
trainModel
Trains a model using the GIS algorithm, assuming 100 iterations and no cutoff.- Parameters:
eventStream- TheeventStreamholding the data on which this model will be trained.- Returns:
- A trained
GISModelwhich can be used immediately or saved to disk using anGISModelWriter. - Throws:
IOException
-
trainModel
public GISModel trainModel(ObjectStream<Event> eventStream, int iterations, int cutoff) throws IOException Trains a GIS model on the event in the specified event stream, using the specified number of iterations and the specified count cutoff.- Parameters:
eventStream- Astreamof all events.iterations- The number of iterations to use for GIS.cutoff- The number of times a feature must occur to be included.- Returns:
- A trained
GISModelwhich can be used immediately or saved to disk using anGISModelWriter. - Throws:
IOException
-
trainModel
Trains a model using the GIS algorithm.- Parameters:
iterations- The number of GIS iterations to perform.di- TheDataIndexerused to compress events in memory.- Returns:
- A trained
GISModelwhich can be used immediately or saved to disk using anGISModelWriter. - Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
trainModel
Trains a model using the GIS algorithm.- Parameters:
iterations- The number of GIS iterations to perform.di- TheDataIndexerused to compress events in memory.threads- The number of thread to train with. Must be greater than0.- Returns:
- A trained
GISModelwhich can be used immediately or saved to disk using anGISModelWriter. - Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
trainModel
Trains a model using the GIS algorithm.- Parameters:
iterations- The number of GIS iterations to perform.di- TheDataIndexerused to compress events in memory.modelPrior- ThePriordistribution used to train this model.- Returns:
- A trained
GISModelwhich can be used immediately or saved to disk using anGISModelWriter. - Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
findParameters
private void findParameters(int iterations, double correctionConstant) -
gaussianUpdate
private double gaussianUpdate(int predicate, int oid, double correctionConstant) -
nextIteration
private double nextIteration(double correctionConstant, CompletionService<GISTrainer.ModelExpectationComputeTask> completionService, int iteration) -
getStopCriteria
Get theStopCriteriaassociated with thisTrainer.- Parameters:
trainingConfig-TrainingConfiguration- Returns:
StopCriteria. IfTrainingConfigurationisnullorTrainingConfiguration.stopCriteria()isnull, then return the defaultStopCriteria.
-
getTrainingProgressMonitor
Get theTrainingProgressMonitorassociated with thisTrainer.- Parameters:
trainingConfig-TrainingConfiguration.- Returns:
TrainingProgressMonitor. IfTrainingConfigurationisnullorTrainingConfiguration.progMon()isnull, then return the defaultTrainingProgressMonitor.
-