Issues when running tests in parallel

[Tymon13]

Issues when running tests in parallel

[gcampbell-msft]

@Tymon13 Thanks for posting the issue. It sounds like, at the very least, we need to make sure that our docs are up-to-date with our actual implementation. Additionally, we need to continue investigating our testing implementation.

@xisui-MSFT FYI.

[gcampbell-msft]

@Tymon13 Could you update to the most recent pre-release version and confirm whether these findings are still accurate? We recently merged #3577 and I'm curious about what affects it has on this issue. Thanks.

[gcampbell-msft]

Either way, we have multiple reasons for running the tests like we do. Specifically, without the allowParallelJobs setting enabled, we want to run each test individually so that the Test Explorer can correctly track progress (assuming the test explorer integration is turned on). With the allowParallelJobs setting enabled, we run in parallel, but the Test Explorer may not track progress as well.

With these things in mind, I think this is simply a documentation bug, but I'd like to get your feedback in order to confirm this.