Non-deterministic, incorrect computation by FTM with a large number of cores

[julien-tierny]

Non-deterministic, incorrect computation by FTM with a large number of cores

[julien-tierny]

In particular, when looking at the two files in ParaView, an obvious difference is that the global min-max pair is missing with 24 cores (see the above screenshot)
But that's not the only missing pair.
The diagram on the left has 345,452 cells (information tab).
The one on the right has 276,343.

[julien-tierny]

Another observation: when my 24core system is busy (for instance, building some code in the background with a high number of threads), the computation is correct (as if, in that context, a racing condition was not happening due to the activity of the system).

[julien-tierny]

I confirm that #849 seems to do the trick (at least on one of my systems)

[julien-tierny]

I have the impression that performances drastically degraded on my 24 core system though.
running with 48threads is now as slow as 1 thread, with a very very long execution of leafGrowth JT :

• 1 threads, leafGrowth JT 1.858s
• 48 threads, leafGrowth JT 6.299s

[CharlesGueunet]

The changes in #849 should only impact the trunk, and not the leafGrowth, do you have any other change ?

[julien-tierny]

actually, we tried to revert this commit with @pierre-guillou and it turns out the problem was already there in the first place.
maybe the problem for this dataset was there was just too many threads and too much contention...