Conda environment

[jaroslawjanas]

Conda environment

[jaroslawjanas]

@jaroslawjanas

Have you tried python=3.11 ?

Can we adapt the file?

Oh yeah. I've been on 3.11 for a week now. No issues.
It's a matter of changing python=3.8 to python=3.11
I can make a PR for it if we are all in agreement.

[darodi]

• fixed by #524

[darodi]

Hello @jaroslawjanas

We already have binaries, docker, flatpak/appImage and pip requirements/venv.
I don't really see the point, but why not.
You should maybe add 7zip. For kindlegen, it will be harder I think 😉

[axu2]

As an aside, Python 3.11 really speeds things up.

The Faster CPython Project is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See Faster CPython for details.

https://www.python.org/downloads/release/python-3110/

[jaroslawjanas]

@axu2
So we should really go with 3.11 for the default Python version?
We could just remove python= completely and let it install the latest but I think that's considered bad practice by the Anaconda community since it doesn't ensure reproducibility and compatibility.
I think 3.11.* is reasonable. It works for me so I see no issue. But the PR #520 is merged so it might require another one to bump the version. I can make one but I want to make sure we reach some consensus here so that we don't end up with a ton of PRs.

[darodi]

What is the best practice with conda to manage other dependencies?
For example qt5, 7zip, ...
Do you need an environment.yml for each platform ? (windows, linux, macOs)

[jaroslawjanas]

What is the best practice with conda to manage other dependencies? For example qt5, 7zip, ... Do you need an environment.yml for each platform? (windows, linux, macOs)

I think the best way to think about Anaconda (conda for short) is as an extension of pip.

And yes there do seem to be some ports of 7zip in conda-forge which is the channel for community-maintained packages.
But it's not something I'd rely on. There is no documentation and not sure what the maintenance is like. I am also not sure how it would integrate into kcc. I am also not sure if we should consider 7zip as a Python dependency.

As for qt5, there is already PyQt5>=5.6.0 in environment.yml if that's what you mean?

We don't need an environment.yml for each platform, the same as we don't need requirements.txt for each platform.
Where I see environment.yml to be the most useful is in a Windows environment, where Anaconda is used 99% of the time. It makes the setup more straightforward. Could be probably added to wiki in the future.

I don't think we need any of the non-python dependencies on Windows (with the exception of 7zip maybe?).
At least I didn't run into any issues so far.

Linux users are likely to use virtualenv together with pip install -r requirements.txt, though conda should work just as well.

[axu2]

We use Python 3.11 in docker for KCC so...

[darodi]

I'm not a conda user, but from what I've read:

https://scribe.esmailelbob.xyz/analytics-vidhya/understand-conda-and-pip-9e5c67da47cc

Difference b/w Conda and Pip
Pip installs Python packages whereas conda installs packages which may contain software written in any language.

If you are using conda without external dependencies other than python packages,
I must admit I don't really understand and see the point "conda vs pyenv/pip/venv".

https://imgs.xkcd.com/comics/standards.png 😁

[jaroslawjanas]

I'm not a conda user, but from what I've read:

https://scribe.esmailelbob.xyz/analytics-vidhya/understand-conda-and-pip-9e5c67da47cc

Difference b/w Conda and Pip
Pip installs Python packages whereas conda installs packages which may contain software written in any language.

If you are using conda without external dependencies other than python packages, I must admit I don't really understand and see the point "conda vs pyenv/pip/venv".

https://imgs.xkcd.com/comics/standards.png 😁

What they mean by "[...] install packages which may contain software written in any languages" is that they are packages that are actually compiled binaries that are then used by Python.

Anaconda downloads .conda files which I think are containers with all the information necessary for a particular package to work properly. I actually don't know the inner workings of those files, but there is some good info here.

In other words "Anaconda provides its own distribution of packages [...]. The packages in Anaconda are pre-compiled and optimized for various platforms, making installation easier and potentially faster."
Though obviously, depending on the specific package you might get little to no improvement.
How significant is this? I don't know. I'd actually have to do proper testing to see if there is any actual improvement in this specific scenario.

As to address "conda vs pyenv/pip/venv". Anaconda is the most popular Python environment manager on Windows so it'd be nice to facilitate that anyway? Makes for a single command setup which might be more convenient for some. Though you can still do it with requirements.txt but it requires an extra command, but then you don't end up with conda packages and miss out on any of the potential optimisations I mentioned above.

Oh, and we get to dictate the default Python version which seems to be a good thing seeing there is performance to be gained there.

I am not a know-it-all so if someone knows something I don't then I am all ears. Python did change a fair bit these past few years so maybe what I wrote above has changed.

[darodi]

@jaroslawjanas

Have you tried
python=3.11
?

Can we adapt the file?