# Optimization Routines:

This is a catalog of optimization
routines which I have successfully used for various projects.
Please also see my MATLAB translations of various Numerical Recipes routines
here,
for additional optimization routines.
## Levenberg-Marquardt Optimization

## Non-Differentiable Optimization: The Method of Hooke and Jeeves

Here you'll find a MATLAB version of the nonlinear optimization routine
developed by Hooke and Jeeves.

"Direct Search Solution of Numerical and Statistical Problems",

by R. Hooke and T. A. Jeeves,

Journal of the ACM, Vol. 8, April 1961, pp. 212-229.

This is a nice algorithm since it does not require the objective
function to be differentiable ... a very nice property to
have when one does not have, or cannot get, an analytic form for
the objective function. The code and an example of its use
(rosenbrock's banana function) are provided below:

## Fibonacci Searching

Here are MATLAB version of the famous Fibonacci search routine.
This is the *optimal* search strategy to use for continuous
functions when one wants to minimize the number of function
evaluations required to find the optimum of a unimodal (a single
global optimum only) function. This routine is considered better
than the more recognized golden-ratio search. For this search
method it is assumed that a minimum is between two limits and we
provide a discrete grid of values one of which corresponds to the
minimum of our unimodal function over this grid. For more detail
on this search method see

Optimum Seeking Methods, by Wilde, D. J.,

Prentice-Hall, Inc., Englewood Cliffs, New Jersey (1964).
## Minimization by the method of Trisection: MINX

Here you'll find a FORTRAN version of the very cute trisection
algorithm for optimization. This is not the most efficient method
for this type of optimization routine. A much better method would
be to use the golden ratio/golden section search for a
"bracketing" type algorithm (found
here). The MINIX
algorithm is describe in the following reference

Algorithm 7, MINIX by J. Boothroyd in Comp. Bulletin, Issue 104,

year 1965, pages 104-108.

John Weatherwax
Last modified: Mon Apr 16 10:18:28 EDT 2007