mysql source worker %s has already started with source %s, but get a request with source %s

[GMHDBJD]

mysql source worker %s has already started with source %s, but get a request with source %s

[GMHDBJD]

reproduced

cluster: dm-worker1(172.16.6.1), dm-worker2(172.16.6.2), dm-worker3(172.16.6.3)
operate-source create source1
operate-source create source2
transfer-source: dm-worker1(free), dm-worker2(source1), dm-worker3(source2)
start-relay for source1 and source2
start-task for source1 and source2
upgrade from v2.0.6 to v2.0.7

worker2 error

{
            "result": true,
            "msg": "no sub task started",
            "sourceStatus": {
                "source": "mysql-replica-02",
                "worker": "dm-172.16.6.2-6262",
                "result": {
                    "isCanceled": false,
                    "errors": [
                        {
                            "ErrCode": 40071,
                            "ErrClass": "dm-worker",
                            "ErrScope": "internal",
                            "ErrLevel": "high",
                            "Message": "mysql source worker dm-172.16.6.2-6262 has already started with source mysql-replica-01, but get a request with source mysql-replica-02",
                            "RawCause": "",
                            "Workaround": "Please try restart this DM-worker"
                        }
                    ],
                    "detail": null
                },
                "relayStatus": {
                    "masterBinlog": "(mysql-bin.003385, 292551706)",
                    "masterBinlogGtid": "eca09ad9-1cf8-11ec-9b6e-0242ac110002:1-4444683058",
                    "relaySubDir": "eca09ad9-1cf8-11ec-9b6e-0242ac110002.000001",
                    "relayBinlog": "(mysql-bin.003385, 292551706)",
                    "relayBinlogGtid": "eca09ad9-1cf8-11ec-9b6e-0242ac110002:1-4444683058",
                    "relayCatchUpMaster": true,
                    "stage": "Running",
                    "result": null
                }
            },
            "subTaskStatus": [
            ]
        }

root cause

dm-worker1(free), dm-worker2(source1), dm-worker3(source2)
tiup upgrade will restart worker in lexicographic order
restart dm-worker1: dm-worker1(free), dm-worker2(source1), dm-worker3(source2)
restart dm-worker2: dm-worker1(source1), dm-worker2(free), dm-worker3(source2)
restart dm-worker3: dm-worker1(source1), dm-worker2(source2), dm-worker3(free)
now dm-worker2 start-relay for source1 but bounded to source2, but we did not handle this situation well.

[lance6716]

as we discussed, since a "relay bound" worker should not be treat as a free worker, I plan to add "relay" WorkerStage to (offline, free, bound). WorkerStage is not persisted so there should be no compatibility problem.