lift pytorch restrictive version constraints

[drzraf]

lift pytorch restrictive version constraints

[mittagessen]

Yes, then that seems like the best compromise for now.

Ok, it's pinned now. I'll tag a new release once a couple of more bug fixes accumulate.

[mittagessen]

That's not going to happen. We've arrived at the current pinned dependencies because the random library combinations when not doing so regularly broke functionality. In particular, different pytorch versions can cause enormous (10x) differences in memory consumption and training divergence. You'll see that some dependencies are much less restricted. Those are for packages whose maintainers do not randomly break stuff.

pytorch 2.4.0 is out and I wonder any would keep Kraken from working but a recent commit 51d593a downgraded from 2.2.0 to 2.1.0 :|

Lightning officially only supports up to pytorch 2.2. But that one isn't on conda-forge yet. I can't maintain and keep track of different dependency sets for each packaging system.

[drzraf]

FYI I got it to work full-pip (no conda), using

• torch 2.4.0.dev20240510+cu121
• pytorch-lightning 1.1.7
• lightning 2.2.4
• coremltools 6.3.0 (7.x didn't work due to missing libcoremlpython.so in the wheel)
• numpy 1.26.4

It's indeed a very sensitive matters of version combinations (eg: shapely version breaking inference in non-obvious ways)

[Sadly, no model fits onto 2GB GPU... yet 😢]

[bertsky]

The problem with Torch 2.1 is that it still uses pkg_resources.packaging, which broke with setuptools v70.

What do you recommend we do?

[mittagessen]

They've pushed pytorch 2.3 onto conda-forge so I can bump up the version as soon as I've verified that everything works.

[mittagessen]

Hm, the issue is there isn't a lightning release that works with pytorch 2.3 yet and conda-forge never got a pytorch 2.2 release. I can pin setuptools below v70 if that helps.

[bertsky]

I can pin setuptools below v70 if that helps.

Yes, then that seems like the best compromise for now.