The function _select_initial_step() may output nan

[rtqichen]

The function _select_initial_step() may output nan

[patrick-kidger]

@rtqichen fixed with the latest merge.

[000Justin000]

I just found the same problem, LOL

[rtqichen]

Hmm yeah that would work temporarily. I didn't think too much about the effect of this on tuples, but maybe removing the elements where f(y0,t0) is returning 0 would make more sense. I'll keep this in mind, thanks for reporting it.

[DrChainsaw]

I also ran into the same issue when I wanted to see how batch norm with running mean and avg would work inside an OdeFunc. Not that I expected it to work well, but as it is now, this kills the model as the first step size evaluation inserts inf in the running mean and var.

d1 typically has a 0 tensor in it because one the the elements of the tuple is the time gradient (which is often zero as func does seldom depend on t).

Isn't the intention of the code to use max norm for d0, d1 and d2? From the reference document: "Do one function evaluation f(x0, y0) at the initial point. It is in any case needed for the first RK step. Then put d0 = ||y0|| and d1 = ||f(x0, y0)||, where the norm is that of (4.11)"?

If so, the statments
dn = tuple(...

can be replaced by
dn = max(...

while de-tuplifying the remaining code dealing with them.

Or maybe just switch "tuple" and "_norm" for mean norm instead which is what 4.11 above is (not that I think it matters much).