com.numericalmethod.suanshu.optimization.unconstrained.steepestdescent
Class SteepestDescent

java.lang.Object
  com.numericalmethod.suanshu.optimization.unconstrained.steepestdescent.SteepestDescent

All Implemented Interfaces:

Minimizer, UnconstrainedMinimizer

Direct Known Subclasses:

ConjugateGradient, FirstOrder, GaussNewton, NewtonRaphson, Powell, QuasiNewton, Zangwill

public abstract class SteepestDescent
extends java.lang.Object
implements UnconstrainedMinimizer

A steepest descent algorithm finds the minimum by moving along the negative of the steepest g direction.

In each iteration, we compute a direction, e.g., dk = −g as in the first order approximation. We search along this direction for an improved minimizer by analytically computing αk that minimizes f(xk + αk * dk). The minimizer xk+1 = xk + αk * dk is updated. This procedure is repeated until |αk * dk| becomes sufficiently small, hence convergence.

See Also:

• "Algorithms 5.2, 5.3, 5.5. Practical Optimization: Algorithms and Engineering Applications. Andreas Antoniou and Wu-Sheng Lu."
• Wikipedia: Gradient descent

Nested Class Summary

class

SteepestDescent.LineSearch A steepest-descent method, in each iteration, searches along a direction to find the next best minimizer along a direction.

Field Summary

protected RealScalarFunction	f the function to be minimized
protected RealVectorFunction	g g: the gradient of f
protected double	tol the convergence tolerance

Constructor Summary

SteepestDescent()

Method Summary

protected abstract SteepestDescent.LineSearch	getLineSearch()
double	minimum() Get the minimum found.
Vector	search(int maxIterations, Vector... initials) Search for a minimizer from an initial guess until the increment is small enough, hence convergence.
void	solve(RealScalarFunction f, double tol)
void	solve(RealScalarFunction f, RealVectorFunction g, double tol)
void	solve(UnconstrainedProblem problem, double tol)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

f

protected RealScalarFunction f

the function to be minimized

g

protected RealVectorFunction g

g: the gradient of f

tol

protected double tol

the convergence tolerance

Constructor Detail

SteepestDescent

public SteepestDescent()

Method Detail

getLineSearch

protected abstract SteepestDescent.LineSearch getLineSearch()

solve

public void solve(RealScalarFunction f,
                  RealVectorFunction g,
                  double tol)

solve

public void solve(RealScalarFunction f,
                  double tol)

Specified by:

solve in interface UnconstrainedMinimizer

solve

public void solve(UnconstrainedProblem problem,
                  double tol)

Specified by:

solve in interface UnconstrainedMinimizer

search

public Vector search(int maxIterations,
                     Vector... initials)

Search for a minimizer from an initial guess until the increment is small enough, hence convergence.

Specified by:

search in interface Minimizer

Parameters:

maxIterations - the maximum number of iterations

initials - an initial guess of minimizer

Returns:

a minimizer

minimum

public double minimum()

Description copied from interface: Minimizer

Get the minimum found.

This is especially useful for an objective function that takes a long time to compute.

Specified by:

minimum in interface Minimizer

Returns:

the minimum found