Docker latest tag not updated?

[gregerspoulsen]

Docker latest tag not updated?

[gregerspoulsen]

Btw. dockerhub has great integration with github for automatically publishing images. I would be happy to help here, but it isn't entirely clear to me how the release flow is around crossbar.

[oberstet]

sure, just some notes that first pop to my mind. in general, the time spent with CI/CD issues of all kinds can be substantial. recently, I had this thought: I love Python (spent many years in c++ before), and I still do! But there are a couple of "issues" that really come and bite once a project has grown to the size/complexity of crossbar with Python. I really feel the price I paid cumulatively over the years on those - and I don't think they ever will disappear:( anyways, wining over.

Obviously when they break everything fails

yes indeed, this has gotten quite complex in itself. the reasons for failing in this first area are:

• complex functional tests, including tests that run multiple nodes and clients => the tests and test drivers need more love, in principle their approach makes sense and works
• past massive code refactorings while things were still settling in the crossbar code base itself
• doing the former with a language lacking strict static types and interfaces => we have expanded our use of python type hints and internal interfaces (ABC types) .. which helps a bit, but not fundamentally

I get the impression you have experienced many issues downstream even when they pass, can you elaborate on that?

• high number of dependencies in crossbar (~100), some of which (eg ethereum related, but also others) have (still) a permanent code churn / change
• python dependency management in general (setuptools/pip), and in particular: dependencies that insist on upper bounds in their deps + confluence in our dependency tree where such a dep is an indirect dep in 2 different paths in our dep graph (DAG) but with different dep version (and thus potentially different upper bounds)
• the challenge of making this work not only on debian/x86-64 (the "easiest" platform), but also arm64, or clang, or windows, or ..
• the complexity due to different "binary packaged formats" for crossbar: Docker, PyInstaller, snaps

[oberstet]

pls see https://hub.docker.com/r/crossbario/crossbar/tags?page=1&name=23.1.2

[gregerspoulsen]

Yes, I noticed a new version has been released on dockerhub, but the latest tag points to a release two years ago: https://hub.docker.com/r/crossbario/crossbar/tags?page=1&name=latest

I think it is generally accepted practice that the latest tag points to the most recent release, so I was a bit surprised when I noticed I was running a two year old release. If it's really intentional I think it should be mentioned somewhere in the documentation which specifically instructs to pull without specifying a tag.

Thanks for your time :)

[oberstet]

I would be happy to help here, but it isn't entirely clear to me how the release flow is around crossbar.

oh, that would be very welcome! helping with build automation=)

there actually is another repo with docker files (and pyinstaller and snaps), which could be OSS'ed. this even includes arm64 etc

BUT: the actual time sucker is making sure those keep working at all. this project, due to amount and characteristics of deps and python building, and due to complexity, can be a black hole. you've been warned;)

if you are still willing to help, I could also give you access to https://github.com/typedefint/crossbar-binaries and you could have a look first and then you can see. it would be incredible welcome and helpful, but it is really a nerve wrecking time sucker;)

[gregerspoulsen]

I noticed you have workflows in this repo that installs crossbar and executes tests. Obviously when they break everything fails, but from the way you write I get the impression you have experienced many issues downstream even when they pass, can you elaborate on that?