Optimization Routines:

This is a catalog of optimization routines which I have successfully used for various projects. Please also see my MATLAB translations of various Numerical Recipes routines here, for additional optimization routines.

Levenberg-Marquardt Optimization

Non-Differentiable Optimization: The Method of Hooke and Jeeves

Here you'll find a MATLAB version of the nonlinear optimization routine developed by Hooke and Jeeves.

"Direct Search Solution of Numerical and Statistical Problems",
by R. Hooke and T. A. Jeeves,
Journal of the ACM, Vol. 8, April 1961, pp. 212-229.

This is a nice algorithm since it does not require the objective function to be differentiable ... a very nice property to have when one does not have, or cannot get, an analytic form for the objective function. The code and an example of its use (rosenbrock's banana function) are provided below:

hooke.m (the main calling function)
best_nearby.m (a helpful auxiliary function)
banana.m (Rosenbrock's "banana" function to minimize)

Fibonacci Searching

Here are MATLAB version of the famous Fibonacci search routine. This is the optimal search strategy to use for continuous functions when one wants to minimize the number of function evaluations required to find the optimum of a unimodal (a single global optimum only) function. This routine is considered better than the more recognized golden-ratio search. For this search method it is assumed that a minimum is between two limits and we provide a discrete grid of values one of which corresponds to the minimum of our unimodal function over this grid. For more detail on this search method see

Optimum Seeking Methods, by Wilde, D. J.,
Prentice-Hall, Inc., Englewood Cliffs, New Jersey (1964).

Minimization by the method of Trisection: MINX

Here you'll find a FORTRAN version of the very cute trisection algorithm for optimization. This is not the most efficient method for this type of optimization routine. A much better method would be to use the golden ratio/golden section search for a "bracketing" type algorithm (found here). The MINIX algorithm is describe in the following reference

Algorithm 7, MINIX by J. Boothroyd in Comp. Bulletin, Issue 104,
year 1965, pages 104-108.

sminx.f (single precision version of minx)
dminx.f (double precision version of minx)
TestSMinx.f (tesing driver for sminx.f)
TestDMinx.f (tesing driver for dminx.f)

John Weatherwax

Last modified: Mon Apr 16 10:18:28 EDT 2007