rostyq/levenberg-marquardt

Provides abstractions to run Levenberg-Marquardt optimization

levenberg-marquardt

Solver for nonlinear least squares problems

The implementation is a port of the classic MINPACK implementation of the Levenberg-Marquardt (LM) algorithm. This version of the algorithm is sometimes referred to as exact LM.

All current unit tests indicate that we achieved identical output (on a floating-point level) to the MINPACK implementation, especially for rank deficient unstable problems. This was mainly useful for testing. The Fortran algorithm was extended with NaN and inf handling, similar to what lmfit does.

The crate offers a feature called minpack-compat which sets floating-point constants to the ones used by MINPACK and removes the termination criterion of "zero residuals". This is necessary for identical output to MINPACK but generally not recommended.

Usage

See the docs for detailed information.

impl LeastSquaresProblem<f64> for Problem {
    // define this trait for the problem you want to solve
}
let problem = Problem::new(initial_params);
let (problem, report) = LevenbergMarquardt::new().minimize(problem);
assert!(report.termination.was_successful());

References

Software:

• The MINPACK Fortran implementation.
• A C version/update, lmfit.
• A Python implementation in pwkit.

One original reference for the algorithm seems to be

Moré J.J. (1978) The Levenberg-Marquardt algorithm: Implementation and theory. In: Watson G.A. (eds) Numerical Analysis. Lecture Notes in Mathematics, vol 630. Springer, Berlin, Heidelberg.

by one of the authors of MINPACK.

The algorithm is also described in the form as implemented by this crate in the book "Numerical Optimization" by Nocedal and Wright, chapters 4 and 10.