Class QNMinimizer
java.lang.Object
opennlp.tools.ml.maxent.quasinewton.QNMinimizer
Implementation of the
Limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm (L-BFGS) which
supports L1-, L2-regularization and Elastic Net for solving convex optimization problems.
Usage example:
// Quadratic function f(x) = (x-1)^2 + 10
// f obtains its minimum value 10 at x = 1
Function f = new Function() {
@Override
public int getDimension() {
return 1;
}
@Override
public double valueAt(double[] x) {
return StrictMath.pow(x[0]-1, 2) + 10;
}
@Override
public double[] gradientAt(double[] x) {
return new double[] { 2*(x[0]-1) };
}
};
QNMinimizer minimizer = new QNMinimizer();
double[] x = minimizer.minimize(f);
double min = f.valueAt(x);
- See Also:
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic interfaceEvaluate the quality of training parameters.static classL2-regularized objectiveFunction.private classClass to store vectors for Hessian approximation update. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final doubleprivate intprivate QNMinimizer.Evaluatorstatic final doubleThe initial step size:1.0.private final intprivate final doublestatic final doubleThe default L1-cost value is0.0d.private final doublestatic final doubleThe default L2-cost value is0.0d.private static final org.slf4j.Loggerprivate final intstatic final intThe default number of Hessian updates to store is15.static final intThe default maximum number of function evaluations is30,000.private final intstatic final doubleThe minimum step size:1e-10.static final intBy default the number of iterations is100.static final doubleprivate QNMinimizer.UpdateInfo -
Constructor Summary
ConstructorsConstructorDescriptionQNMinimizer(double l1Cost, double l2Cost) Initializes aQNMinimizer.QNMinimizer(double l1Cost, double l2Cost, int iterations) Initializes aQNMinimizerwith L1 and L2 parameters.QNMinimizer(double l1Cost, double l2Cost, int iterations, int m, int maxFctEval) Initializes aQNMinimizer. -
Method Summary
Modifier and TypeMethodDescriptionprivate voidcomputeDirection(double[] direction) L-BFGS two-loop recursion, see Nocedal invalid input: '&' Wright 2006, Numerical Optimization, p.private voidcomputePseudoGrad(double[] x, double[] g, double[] pg) Conducts pseudo-gradient for L1-regularization.private booleandouble[]Finds the parameters that minimize the objective function.voidsetEvaluator(QNMinimizer.Evaluator evaluator)
-
Field Details
-
logger
private static final org.slf4j.Logger logger -
CONVERGE_TOLERANCE
public static final double CONVERGE_TOLERANCE- See Also:
-
REL_GRAD_NORM_TOL
public static final double REL_GRAD_NORM_TOL- See Also:
-
INITIAL_STEP_SIZE
public static final double INITIAL_STEP_SIZEThe initial step size:1.0.- See Also:
-
MIN_STEP_SIZE
public static final double MIN_STEP_SIZEThe minimum step size:1e-10.- See Also:
-
L1COST_DEFAULT
public static final double L1COST_DEFAULTThe default L1-cost value is0.0d.- See Also:
-
L2COST_DEFAULT
public static final double L2COST_DEFAULTThe default L2-cost value is0.0d.- See Also:
-
NUM_ITERATIONS_DEFAULT
public static final int NUM_ITERATIONS_DEFAULTBy default the number of iterations is100.- See Also:
-
M_DEFAULT
public static final int M_DEFAULTThe default number of Hessian updates to store is15.- See Also:
-
MAX_FCT_EVAL_DEFAULT
public static final int MAX_FCT_EVAL_DEFAULTThe default maximum number of function evaluations is30,000.- See Also:
-
l1Cost
private final double l1Cost -
l2Cost
private final double l2Cost -
iterations
private final int iterations -
m
private final int m -
maxFctEval
private final int maxFctEval -
dimension
private int dimension -
updateInfo
-
evaluator
-
-
Constructor Details
-
QNMinimizer
public QNMinimizer() -
QNMinimizer
public QNMinimizer(double l1Cost, double l2Cost) Initializes aQNMinimizer.- Parameters:
l1Cost- The L1-regularization cost. Must be equal to or greater than0.l2Cost- The L2-regularization cost. Must be equal to or greater than0.- Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
QNMinimizer
public QNMinimizer(double l1Cost, double l2Cost, int iterations) Initializes aQNMinimizerwith L1 and L2 parameters.- Parameters:
l1Cost- The L1-regularization cost. Must be equal to or greater than0.l2Cost- The L2-regularization cost. Must be equal to or greater than0.iterations- The maximum number of iterations. Must be greater than0.- Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
QNMinimizer
public QNMinimizer(double l1Cost, double l2Cost, int iterations, int m, int maxFctEval) Initializes aQNMinimizer.- Parameters:
l1Cost- The L1-regularization cost. Must be equal to or greater than0.l2Cost- The L2-regularization cost. Must be equal to or greater than0.iterations- The maximum number of iterations. Must be greater than0.m- The number of Hessian updates to store. Must be greater than0.maxFctEval- The maximum number of function evaluations. Must be greater than0.- Throws:
IllegalArgumentException- Thrown if parameters were invalid.
-
-
Method Details
-
getEvaluator
-
setEvaluator
-
minimize
-
computePseudoGrad
private void computePseudoGrad(double[] x, double[] g, double[] pg) Conducts pseudo-gradient for L1-regularization.- Parameters:
x- The current pointg- The gradient atx.pg- The pseudo-gradient atxwhich is to be computed.
-
computeDirection
private void computeDirection(double[] direction) L-BFGS two-loop recursion, see Nocedal invalid input: '&' Wright 2006, Numerical Optimization, p. 178) -
isConverged
-