[Mod] Combat Extended causes desync

[zetrith]

[Mod] Combat Extended causes desync

[Dunkhan]

Yes. It is going to take much longer than expected however as there is extensive use of random numbers from three different random number libraries in this mod. I have to make sure that all clients roll the same result on every random roll, and that is complicated.

[Dunkhan]

I looked into this. I am having trouble working out how the sync values are calculated and without knowing that there is not much I can do. Is there any documentation of how this mod calculates sync?

What I found so far is the following:

• The mismatch is in the syncmap (as the error says). One value is different.
• The desync occurs when the player uses the UI button to reload a pawn's weapon.
• A pawn reloading automatically when picking up an empty weapon does not cause a desync.
• The class that deals with job tracking, JobTrackerStart, throws a warning when any job is spawned from the interface: Log.Warning($"Started a job {newJob} on pawn {__instance.pawn} from the interface!"); This could be related.

[kruemelkeksfan]

In case you need any support with CE compatibility, the mod has a pretty active discord:
https://discordapp.com/invite/SGQKESU

[Dunkhan]

Thanks for the tip, but I was assuming none of them really knew how to do it. I spoke to Dub from Dubs Mods because he already did the compatibility. He was really helpful.
If I succeed in implementing it for CE I will write a short guide in the wiki here about what I did with copy paste-able code examples so that others wont have the same issues.

[kruemelkeksfan]

Oh, my bad. I didn't recognize your one of the CE-devs. Knowing this, your post suddenly makes more sense :O

[Dunkhan]

I am not really. I made a fork, but I have never contributed. That should change tomorrow though.

[Dunkhan]

Seems like I cant edit the wiki here. I wrote a guide on my own fork but I doubt many people will see it there.
https://github.com/Dunkhan/Multiplayer/wiki/Guide-to-making-UI-buttons-compatible

[kruemelkeksfan]

Did you/could you add these changes to your CE-fork and maybe publish a release?

[Dunkhan]

I could, I got the reload button working. There are a number of other buttons that need to have the same treatment before it can be released, there are still desyncs. I am confident I can get it done now though and NoImageAvailable says he will include it in the main mod when it is done.

[kruemelkeksfan]

Nice, thank you a lot for your effort :)

[xetra11]

is this about getting CE ready for the multiplayer mod?

[xetra11]

Ah I see. Ok good luck on that @Dunkhan. I will only play Rimworld MP only with CE :)
Thanks for your effort!

[kruemelkeksfan]

How is it going along? Do you need help with anything?

[Dunkhan]

I have worked out how to synchronise the UI elements as mentioned above, but there is still an issue that shooting instantly desyncs the game. At first I thought this was due to random calls (and these may still be an issue later) but even without those the issue persists. I worked out how to debug properly (guide coming soon) and I have narrowed it down to the following:

When a pawn shoots, the target has a different height (vertically above the map) on the arbiter than on the clients. Given the CE calculates ballistic projectile curves (a bit overkill IMO, I never noticed it as a player), different heights mean different collision checks. So the bullet always hits something different on the arbiter than the client.

Debugging is still ongoing and unless you are a really good C# coder you probably can't help much at this stage. Once I have eliminated the main desyncs though i will need a lot of testing to be done. I will post here once I get to that stage.

[AlphaAtlas]

Would this be helpful at this point?

https://github.com/Pecius/UnofficialMultiplayerAPI/

[kruemelkeksfan]

Did you push your progress somewhere? I finally might have the time and motivation to look into it and decide, whether I could be able to help out. I wouldn't call myself a "really good C# coder", but honestly, who would?

[Dunkhan]

This is the branch where I was committing: https://github.com/Dunkhan/CombatExtended/commits/ZetrithMultiplayerCompatibility

It is way out of date, sorry about that. Also I removed a bunch of stuff in this commit Dunkhan/CombatExtended@32ced7f which needs to be reverted once the problem is found. It seems like the random stuff I removed was not the cause of the issue so it should be fine to put it back in.

The first and last commits are both important progress towards compatibility I think, although the first could be replaced with use of the new API.

[kruemelkeksfan]

Did you synchronize any buttons other than the reload button? I imagine the loadouts for example being a pain to synchronize...from your perspective would it be better to start over with the new Multiplayer API or would it not be worth the time?

[Dunkhan]

I think starting over with the API is probably a good plan. I am happy to do it myself though. The issue is not with the buttons. Something unidentified is causing desyncs, and until that is resolved I didn't think it was worth doing all the basic grunt work. On the other hand it is not impossible that something in the button code is triggered without the user clicking them... maybe?

Either way, the game desyncs when a character fires a weapon. I could not figure out why.

[kruemelkeksfan]

I just took a look at your commit "32ced7fb97d5b2b51cf518f4aea28d4a3de8043f". It seems, that there is still a random number generated in "DamageWorker_AddInjuryCE.cs" line 139. Did it just desync on hits or on missed shots, too? I can only imagine a slipped through random value causing the desync, since everything else should function deterministically. Even if there was an automatic button click, it should click the same button on the arbiter and all clients, unless a random gen is involved, right?

There are also more calls to methods with the word "Random" in their name, but without checking I assume, that you defused all of those? I am talking about "CE_Utility.RandomElement()", "source.TryRandomElementByWeight()", "pawn.health.hediffSet.GetRandomNotMissingPart()", "CE_Utility.RandomInRange()" and "region.RandomCell".

[Dunkhan]

That is totally it. I don't know how I missed that. Not every shot was desyncing and it seemed that injuries resulting from shots were different on each client. I think there is a very high chance that was it.

Do we have a confirmed way to get all those random elements working again? I know some random calls are compatible with multiplayer, and although some of the ones I removed are unnecessary, a lot of them are not.

[kruemelkeksfan]

Nice, now I feel a bit useful at least :)

Can you do it like in the "Fields"-Section here: https://github.com/Parexy/Multiplayer/wiki/API?

A cheap workaround could also be to disable the RNGs and replace them with something deterministic, but still practically random. So in the injury-example you could e.g. use the final height of the bullet and do a modulo operation on that, to determine, which organ is hit. I don't know whether this is really possible and it might degrade the gameplay a bit, but it could be a relatively easy way of doing it.

[Dunkhan]

This can be closed. It is not an open issue that needs to be dealt with by this project. CE compatibility is something I will add to the CE issue tracker if I start working on it again. That will most likely not happen until they are done with the 1.1 update.