scipy >= 1.8.1 causes a segfault

[petrelharp]

scipy >= 1.8.1 causes a segfault

[silastittes]

method='BFGS' (instead of L-BFGS-B) runs with scipy==1.8.1 fwiw.

[petrelharp]

But, we can ignore this for now because we have a requirements file that pins scipy earlier than this.

[silastittes]

I've narrowed the segfault down to this line

product-space-FEM/product_fem/inverse_problems.py

Line 148 in daf7a45

results = opt.minimize(fun, m0.array(), args, method, jac, callback=callback, options=options)

There was an edit to scipy minimize for v1.8.1 here
scipy/scipy@1a0ac2d

[silastittes]

This might be relevant but didn't read closely
scipy/scipy#14915

[silastittes]

Or this one scipy/scipy#5102

[petrelharp]

Or this one scipy/scipy#5102

That one looks promising. However, we're only using 125 parameters in that example, so I don't think it can be it? Oh, that's also with scipy 0.12.1.

[petrelharp]

ps. I got the number of parameters by sticking in

        print(f"Optimizing using {method} on {len(m0.array())} parameters with arguments {args} and options {options}.")

just before the call to opt.minimize.

[petrelharp]

We're not doing constrained optimization, so no need to use L-BFGS-B instead of BFGS!

[silastittes]

Awesome! I’ll play around with the versions for Python and scipy in the conda yml a bit more and make sure that switch clears up the segfault for other versions. Want me to make a small PR with the tiny updates after that?

[petrelharp]

Sure! I notice that BFGS doesn't have the ftol and maxcor options, but probably it has an option that's similar to ftol at least; perhaps we should stick that in.

[silastittes]

Cool. Happy to modify that. Does fixing the method to be BFGS seem like the right move, or should it keep option to use other methods?

[petrelharp]

I think we might want to try other ones out. Maybe we should throw an error if it's "L-BFGS-B", though?