Configuration questions

[rust-cv]

Configuration questions

[vadixidav]

@jannschu do you think you could give some advice on this?

My naive guess is that perhaps the trust region is too small and it is choosing to bias more towards gradient descent than Gauss-Newton. I haven't been using this library recently, but my guess would be to increase the trust region size. When the damping factor is zero, Leverberg-Marquardt behaves like Gauss-Newton, and when it is high it behaves like gradient descent. My guess is that the damping factor is not getting set low enough at some point, but I don't know how to configure the logic for converging and backing off the damping factor. I defer to @jannschu, as they know much more about how the current implementation works (they ported it from MINPACK).

[jannschu]

Is this problem a 3D version of the beacon finding problem shown in the video? Is the source code already somewhere?

The tiny step sizes might indicate a wrong Jacobian. How do you compute it? If your problem is reasonably fast to to port to Python or Matlab you could check if the reference implementation from MINPACK gives different results. Should that be the case I would consider this a bug.

If given a correct solution as initial parameters the algorithm should terminate immediately since the gradient should be zero.

To answer your question regarding ignoring ftol and gtol: You may set them to zero, but the algorithm will still terminate if the predicted or actual reduction is below the the machine precision.

[ChristopherRabotin]

I don't know what is the beacon searching problem. The problem I'm solving is orbital targeting for spacecraft maneuver design. For the time being, the gradient is computed by finite differencing. I know it to be correct because the same gradient is computed for the Newton Raphson method, and there are at least a dozen validation scenarios of that implementation compared to a leading NASA software that solves the same problem.

…

On Mon, Jan 3, 2022, 09:15 jannschu ***@***.***> wrote: Is this problem a 3D version of the beacon finding problem shown in the video? Is the source code already somewhere? The tiny step sizes might indicate a wrong Jacobian. How do you compute it? If your problem is reasonably fast to to port to Python or Matlab you could check if the reference implementation from MINPACK gives different results. Should that be the case I would consider this a bug. If given a correct solution as initial parameters the algorithm should terminate immediately since the gradient should be zero. To answer your question regarding ignoring ftol and gtol: You may set them to zero, but the algorithm will still terminate if the predicted or actual reduction is below the the machine precision. — Reply to this email directly, view it on GitHub <#14 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2AYOYUHVABFYGKZCRTUUFLIHANCNFSM5KMFXILA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jannschu]

You said a good description of the problem you are solving is given at the start of the linked YouTube video. There I can only find a problem where the location of a transmitter shall be found through distances to beacons. The other page only seems to contain a high level list of things you want to control.

I also looked in the nyx_space repository, but I could not find the code you are working on.

In order to proceed I would suggest to provide a minimal working example and if possible a comparison with MINPACK. A description of the least-squares problem would also be helpful.

It converges on the correct solution in 8 evaluations if I provide it with the exact solution of the Gauss Newton algorithm.

This is, as I said, quite suspicious. Other basic things to check would be that the gradient for the correct x is computed, that is in the right form (matches residuals), and that it was not multiplied with -1. More generally, your optimization problem must be correctly mapped to the form this library expects. But it seems you already spent quite some time on that.

It also seems you want to find a root. Modeling this as a least squares problem might give you local minima.

Since there is no description of the least-squares description of the problem you want to solve (unless I spent hours digging into orbital targeting, which is new to me) , there is no minimal code for me to work with, there is no comparison with other LM implementations where it does work, and your issue only contains examples of iterations which are meaningless to me, and that I answered the question on xtol and ftol it seems that was the best we can do here. Sorry, I feel your frustrations :-(

[ChristopherRabotin]

Ah, yes, now I see. Sorry, I'm still on vacation and my brain isn't working, so I had forgotten that I added a link to that video. But yes, the problem is analogous in theory. I try to find a solution to my control where the targets are met: I'm formulating that as a root finding problem. The residuals will go to zero when the control is approaching the solution. Is there another way to formulate this problem without manually rewriting Lagrange multipliers and such for every new input problem? The code currently has a whole new implementation to integrate with your library but I've been wanting to consolidate it with the Newton Raphson implementation since the residuals and Jacobian are identical. The LM version is available here: https://gitlab.com/nyx-space/nyx/-/blob/128-finite-burn-targeting/src/md/opti/minimize_lm.rs (128-finite-burn-targeting branch). I'll work on this probably next week, and that will help me write a simple reproducible example. I must say that your api is great: using a trait is by far the cleanest method and I wish many other libs did the same (instead of assuming the dimensions of the function to minimize).

…

On Tue, Jan 4, 2022, 08:47 jannschu ***@***.***> wrote: You said a good description of the problem you are solving is given at the start of the linked YouTube video. There I can only find a problem where the location of a transmitter shall be found through distances to beacons. The other page only seems to contain a high level list of things you want to control. I also looked in the nyx_space repository, but I could not find the code you are working on. In order to proceed I would suggest to provide a minimal working example and if possible a comparison with MINPACK. A description of the least-squares problem would also be helpful. It converges on the correct solution in 8 evaluations if I provide it with the exact solution of the Gauss Newton algorithm. This is, as I said, quite suspicious. Other basic things to check would be that the gradient for the correct x is computed, that is in the right form (matches residuals), and that it was not multiplied with -1. More generally, your optimization problem must be correctly mapped to the form this library expects. But it seems you already spent quite some time on that. It also seems you want to find a root. Modeling this as a least squares problem might give you local minima. Since there is no description of the least-squares description of the problem you want to solve (unless I spent hours digging into orbital targeting, which is new to me) , there is no minimal code for me to work with, there is no comparison with other LM implementations where it does work, and your issue only contains examples of iterations which are meaningless to me, and that I answered the question on xtol and ftol it seems that was the best we can do here. Sorry, I feel your frustrations :-( — Reply to this email directly, view it on GitHub <#14 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2E5SCRWPM3FZRH4JPLUUKQXRANCNFSM5KMFXILA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>