wonderbird/testprocesswrapper

Launch and control dotnet processes wrapped into the coverlet code coverage analyzer.

My Organizations

• Small Coding Dojo - Exercises performed by a group of software crafting enthusiasts.
• Wonderbird's Katas - Exercises I perform alone.

Link Collections

• wonderbird.github.io - My blog and link collection (GitHub Repo: wonderbird.github.io).
• Software Crafting - Links related to Software Crafting.
• Software Architecture - Links related to Software Architecture.
• Gists - My GitHub Gists

testprocesswrapper's Issues

🐞 ExceptionReporter does not work under windows server 2019 🐞

ExceptionReporter does not work under windows server 2019

Problem

Under macOS Catalina 10.05 and Ubuntu 20 LTS, the tests are working as expected.
Look at this Pull Request

The worst part is, that the exception seems to cause the test host to not shut down.

That means the dotnet test process will run indefinitely until manually killed.

This is pretty bad if you want to execute the test in pipelines.

Question

Should I ditch the support for windows inside the test pipelines, or what do you suggest?

Additional Information

For more information please look into the full logs

Starting: RemoteControlledProcess.Unit.Tests
[xUnit.net 00:00:00.81] RemoteControlledProcess.Unit.Tests.ExceptionReporterExtensionTests.Write_CalledInCatchBlock_WrittenMessagesMatch(invocationIndex: 0, expectedMessageRegex: "Unhandled exception in .*RemoteControlledProcess.U"...) [FAIL]
[xUnit.net 00:00:00.81] Assert.Matches() Failure:
[xUnit.net 00:00:00.83] RemoteControlledProcess.Unit.Tests.ExceptionReporterExtensionTests.Log_CalledInCatchBlock_LogsCriticalMessageWithExpectedRegex [FAIL]
Test Run Failed.
[xUnit.net 00:00:00.81] Regex: Unhandled exception in .*RemoteControlledProcess.Unit.Tests/ExceptionReporterExtensionTests.cs:[0-9]+
[xUnit.net 00:00:00.81] Value: Unhandled exception in D:\a\RemoteControlledProcess\RemoteControlledProcess\RemoteControlledProcess.Unit.Tests\ExceptionReporterExtensionTests.cs:22
[xUnit.net 00:00:00.81] Stack Trace:
[xUnit.net 00:00:00.81] D:\a\RemoteControlledProcess\RemoteControlledProcess\RemoteControlledProcess.Unit.Tests\ExceptionReporterExtensionTests.cs(32,0): at RemoteControlledProcess.Unit.Tests.ExceptionReporterExtensionTests.Write_CalledInCatchBlock_WrittenMessagesMatch(Int32 invocationIndex, String expectedMessageRegex)
[xUnit.net 00:00:00.83] Assert.Matches() Failure:
[xUnit.net 00:00:00.83] Regex: Unhandled exception in .*RemoteControlledProcess.Unit.Tests/ExceptionReporterExtensionTests.cs:[0-9]+
[xUnit.net 00:00:00.83] Value: Unhandled exception in D:\a\RemoteControlledProcess\RemoteControlledProcess\RemoteControlledProcess.Unit.Tests\ExceptionReporterExtensionTests.cs:43
[xUnit.net 00:00:00.83] Stack Trace:
[xUnit.net 00:00:00.83] D:\a\RemoteControlledProcess\RemoteControlledProcess\RemoteControlledProcess.Unit.Tests\ExceptionReporterExtensionTests.cs(56,0): at RemoteControlledProcess.Unit.Tests.ExceptionReporterExtensionTests.Log_CalledInCatchBlock_LogsCriticalMessageWithExpectedRegex()

A bug inside unit when running code under windows server 2019

Fix: CA1848 - Enable high performance logging

While migrating to .net 6, the compiler issued the code analysis warnings

• CA1848 "Use the LoggerMessage delegates",
• CA2254 "Template should be a static expression".

This issue addresses the corresponding TODO comments in the code. Please fix them.

Bug: Allow selecting "Release" or "Debug" configuration of wrapped test process

Steps to reproduce the issue

1. Compile the process under test with "release" configuration
2. Run tests using TestProcessWrapper for this process

Expected behavior

The test should run the process from the corresponding "bin/Release/net7.0" directory

Observed behavior

The test tries to start the process from the "bin/Debug/net7.0" directory
The test run fails, because the Debug directory does not exist

References

The detailed logs of my build action shows the error:

...
2023-05-16T03:47:33.0342273Z   malimo -> /home/runner/work/malimo/malimo/malimo/bin/Release/net7.0/malimo.dll
...
2023-05-16T03:47:41.6956624Z    System.ComponentModel.Win32Exception : An error occurred trying to start process '/usr/share/dotnet/dotnet' with working directory '/home/runner/work/malimo/malimo/malimo/bin/Debug/net7.0'. No such file or directory

This is the corresponding GitHub action result

Bug: Tests hang when coverlet is enabled and arguments are specified for process under test

Steps to reproduce the issue

1. Create TestProcessWrapper for a process under test (e.g. short lived application in this repo)
2. Enable coverlet for the wrapper
3. Specify a command line argument
4. Start the process using TestProcessWrapper.Start()

Expected behavior

• The TestProcessWrapper.Start() call returns after less than a second
• The process runs as expected
• Coverage is reported in the captured output

Actual behavior

• The TestProcessWrapper.Start() call never returns

Hints

• When starting the process, TestProcessWrapper reports the following command line (malimo.dll is the process under test):

coverlet "." --target "dotnet" --targetargs "malimo.dll" --output /Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/TestResults/malimo.48f6e798-70a1-443d-94bd-a503d7dfc022.xml --format cobertura --dry-run --file=/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/source/Testfile.md --target-dir=/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/target

• The coverlet command line documentation says, that the arguments for the process under test must be given via the --targetargs argument to coverlet. This works:

coverlet "." --target "dotnet" --targetargs "malimo.dll --dry-run --file=/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/source/Testfile.md --target-dir=/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/target" --output /Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/TestResults/malimo.48f6e798-70a1-443d-94bd-a503d7dfc022.xml --format cobertura
Debug: Process ID 15160
Information: Target folder: '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/target'
Information: File '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/source/Testfile.md' contains
Information: - 'noun-starship-3799189.png'
Information: - 'noun-island-1479438.png'
Debug: Would move '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/source/noun-starship-3799189.png' to '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/target'
Debug: Would move '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/source/noun-island-1479438.png' to '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/bin/Debug/net7.0/data/test-20230517T234022-c1235e05-6d5a-402f-8dca-ab2d16a8f236/target'

Calculating coverage result...
  Generating report '/Users/stefan/source/wonderbird/malimo/malimo.Acceptance.Tests/TestResults/malimo.48f6e798-70a1-443d-94bd-a503d7dfc022.xml'
+--------+--------+--------+--------+
| Module | Line   | Branch | Method |
+--------+--------+--------+--------+
| malimo | 83.59% | 55.55% | 95.65% |
+--------+--------+--------+--------+

+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 83.59% | 55.55% | 95.65% |
+---------+--------+--------+--------+
| Average | 83.59% | 55.55% | 95.65% |
+---------+--------+--------+--------+