erdc channel's "stevedore" clobbers "six"

[holoviz-topics]

erdc channel's "stevedore" clobbers "six"

[jbednar]

@dharhas, is there any reason stevedore bundles six and argparse instead of just depending on them? That's an unusual arrangement and to me seems safe only if it bundled them as submodules within its own module, not at the top level where there will be conflicts with other packages.

[ceball]

The packaging of stevedore on conda-forge looks less problematic, so maybe it would be enough to just pin stevedore's version to be higher than the stevedore on erdc's channel. I've done that in #10; please tell me if that's going to cause a problem (with quest?) I'm unaware of.

If everyone were using conda 4.4, we could instead specify to install only gssha, filigree, libfiligree from the erdc channel.

[dharhas]

I've removed stevedore from the erdc channel. I think it is an artifact of a time before conda-forge was available. I'm a little confused why a lower version of stevedore is preferred over a higher version but I guess channel has priority? Our policy is to only maintain packages in erdc that cannot be put in an community channel like conda-forge.

[dharhas]

@jbednar no idea. Stevedore and pbr (python build reasonableness) are part of openshift and I've noticed that they have their own particular way of doing things that don't always match the wider community. Back in the day I picked stevedore as a framework for a plugin architecture in quest and pbr to deal with versioning. If I had to do it over, I would probably go with versioneer for versions and I'm not sure what else is out there for plugins.

[ceball]

I'm a little confused why a lower version of stevedore is preferred over a higher version but I guess channel has priority?

I think so, yes.

[jbednar]

In that case, presumably we could have avoided the issue by bumping the erdc channel to be down below conda-forge?

[ceball]

Depends. It was the first thing I considered, but I took the conservative option of avoiding only the one package known to cause a problem.

By 'depends', I mean: (a) is there any package already on conda-forge that's also on erdc, where we need the package to come from erdc? (b) what if in the future someone puts one of the must-come-from-erdc packages onto conda-forge?

However, Dharhas has clarified that the "policy is to only maintain packages in erdc that cannot be put in an community channel like conda-forge", so maybe the erdc channel should be put at a lower priority than conda-forge, and we should just see what happens?

And also, maybe various packages should be removed from the erdc channel that are on conda-forge? There appear to be quite a few. However, I don't know why those packages exist - there might be a good reason.

[jbednar]

I think we should pin versions very specifically as the final deliverable from this phase, and then relax them when we resume development in the next phase. So at this point it shouldn't matter; there shouldn't be any significant ambiguity. Eventually, my guess is to put the erdc channel lower in priority, because new versions of things are likely to appear on conda-forge, and we'll then realize that and be forced to adapt accordingly.

[dharhas]

In general, the erdc channel is only for stuff not available in other channels, so it should probably be last in priority.

[ceball]

I think we should pin versions very specifically

@jbednar, so just to be clear, I should pin all packages in environment.yml to exact versions currently being used?

And then later on, we'll also make a conda package where we don't do such pinning.

[jbednar]

If you're volunteering, then yes, please do! I was just saying abstractly that someone should do it. :-) We should make a note (e.g. a comment on that line of the file) for any of those pins that are actually known to be required, so that we don't mistakenly delete those later when we remove the rest of the pinning.

[dharhas]

And also, maybe various packages should be removed from the erdc channel that are on conda-forge? There appear to be quite a few. However, I don't know why those packages exist - there might be a good reason.

There are some packages in the erdc channel that were created by the hashdist/conda portion of this project in 2017. The reason as far as I understand, is that they have compilation options suited for hpc that were not accepted by conda-forge. Those need to stay. None are packages EarthSim uses. Any older ones can be removed if causing problems.

[ceball]

Thanks for explaining that.

Anyway, the original issue is resolved.