realife-brahmin/wsu-convex-nonlinear-nonsmooth-analysis-and-optimization

Convex, Nonsmooth, Nonlinear Optimization Solver and Problems

MATH 564: Convex and Nonlinear Optimization and MATH 565: Nonsmooth Analysis and Optimization

Prof. Thomas Asaki | Fall 2023 and Spring 2024

Julia implementations for the two courses at Washington State University, Pullman.

This repo contains a solver in julia , and different types of optimization problems to solve for.

Projects as part of the two courses:

• Parameter Estimation (Estimating DampedSHM parameters)
• Functional Estimation (Estimating a function which minimizes drag force)
• Signal Denosing (Fitting a curve to data, choosing between abiding smoothness of curve or accuracy of fitting)
• Neural Network Training (Detecting Liver Disease in a dataset of patients based on 10 features)
• Minimum Time Trajectory (for a 2D Speed Matrix, and given points A and B on the plane, compute the trajectory minimizing the time of travel)
• Final Project (Computing the optimal placement of a receiver, given the location of several transmitters)
• Derivative Free Optimization Methods (implementation of Genetic Algorithm and Nelder Mead Simplex Method)
• Equality Constrained Quadratic Programming (detecting location of fire based on some boundary constraints which can be expressed as an ECQP)
• Active Set Quadratic Programming (finding optimal locations for substations in various communities to minimize electric transmission losses)

LineSearch/TrustRegion Methods Used:

• Gradient Descent
• Quasi Newton BFGS
• Projected Gradient Conjugate Gradient Method
• Note: Conjugate Gradient Descent Method hasn't been supported in a while, and has issues.
• Note: SR1 based Trust Region Method implementation has been indefinitely paused, in favour of more urgent tasks.

Heuristic Methods Available:

• Genetic Algorithm
• Nelder Mead Simplex Algorithm

Space Sampling Methods Available:

• Halton Sequence Sampling
• Latin Hypercube Sampling

Step-selection Algorithms Used:

• Strong Wolfe + Bisection Interpolation
• Armijo + Backtracking (support removed, always use Strong Wolfe henceforth

wsu-convex-nonlinear-nonsmooth-analysis-and-optimization's Issues

`createInitialSimplexFromOnePoint` in two places, make sure it is only in one from now on

Currently it is both in its own script of the same name and inside nelderMead.jl. The NM one only uses haltonSequence. I think it would be good to move the non-NM function into spaceSample.jl with a slightly different name.

Scripts for non-working algorithms can be archived.

Algos like Conjugate Gradient and Trust Region, which really need a re-write based on my Dict() based data structures can be shelved. I'll get back to rewriting them after the semester perhaps, or never, but at least my src/ will be less cluttered.

Think about whether `pr` should be a `Dict` instead of the current `NamedTuple`.

It does get annoying sometimes. Especially when repeated solver runs are required, or post instantiation mutation is required.

Remover `logs.txt` every time a new run is made

logs.txt always written in, never reset, causing it to become bloated with continuous testing

Forgot to doc all the functions used in `transmissionLines` objfun

Insert `xopt` and `fopt` (both a single value each) inside `res` output by each `optimizeXYZ` solver.

It's annoying for me to always having to extract them from xvals and fvals

ASQP solver currently does not count `fevals`, `lagrevals`.

This can be fixed after the semester ends?

High time `pDict` becomes equal to `params`, instead of having the latter as a field.

Currently pDict = (params=params, data=data)
Want: pDict = params