Global mode does not work correctly

[crazy-max]

Global mode does not work correctly

[danielgrabowski]

Any news on this issue?

[crazy-max]

@danielgrabowski I'm going to start working on it but I'll need to do a large refactoring beforehand. Keep you in touch.

[moschlar]

@crazy-max @djmaze It seems that compose file syntax support for jobs only recently landed in Docker 23 (moby/moby#41895 (comment))

docker/cli#2907

[danielgrabowski]

I have the same problem. If I start a global service on 9 nodes running every 1m I can see that every minute it starts on a different set of nodes (usually 2 or 3 out of 9 more less randomly).

[crazy-max]

Hi guys, I did some tests on my side and it appears that this could be due to the way the tasks on the Swarm side are managed. Maybe linked to a GC issue with task reaper. Any ideas @dperny?

[djmaze]

So, in spite of job support finally getting into Docker (yeah!), this project will probably still be useful as long as there is no cron scheduler in Swarm. @crazy-max Do you plan on migrating swarm-cronjob to the new model?

That should then probably fix this issue as well.

[crazy-max]

@djmaze Yes I'm aware of this feature. That looks promising and I think it will benefit for the global mode support (and fix this issue).

[djmaze]

Is anyone aware if Docker CE 20.10 will be released in the near future? That will contain the new job support which blocks this issue.

[mostafa-asg]

I have the same issue.
Cluster: 1 manager node with 2 worker nodes.
Docker version: 20.10.6
I ran busybox date example in global mode.

date_test.0.p88rnuivp25o@swarm_test1    | Sun May 23 18:48:44 UTC 2021
date_test.0.66loc48rvrgd@swarm_test3    | Sun May 23 18:50:00 UTC 2021
date_test.0.d3j43d50ls90@swarm_test2    | Sun May 23 18:51:00 UTC 2021
date_test.0.49dmp9uvt1mt@swarm_test3    | Sun May 23 18:52:00 UTC 2021
date_test.0.bxnixn2wpetx@swarm_test3    | Sun May 23 18:48:47 UTC 2021
date_test.0.xvocor1a123r@swarm_test3    | Sun May 23 18:51:01 UTC 2021
date_test.0.flw796nt9mmy@swarm_test2    | Sun May 23 18:48:32 UTC 2021
date_test.0.km0tkw07iqww@swarm_test2    | Sun May 23 18:49:00 UTC 2021

swarm_test1 only appeared once.

[Luis97lol]

Any news on this issue? Mode global only runs on the node that swarm-cronjob is deployed. Other nodes never ran

[Luis97lol]

docker swarm ca --rotate

This command apparently solves the problem