Junit5 nested test classes run twice

[mas-chen]

Junit5 nested test classes run twice

[marcphilipp]

@mas-chen Please provide a complete reproducer, i.e. including the project's POM etc.

[mas-chen]

Sure here is my reproducer test. https://github.com/mas-chen/flink-connector-kafka/blob/b93fc74029a7c88c61ed04dc820fc9f59d554448/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/MasonTest.java

Surefire is configured as part of an external project (see parent pom in the root pom)

[marcphilipp]

Thanks!

I debugged the execution. In this case, Maven Surefire sends three separate discovery/execution requests (order is unclear to me, might be dependent on file system):

1. ClassSelector [className = 'org.apache.flink.streaming.connectors.kafka.MasonTest$MasonTest1']
2. ClassSelector [className = 'org.apache.flink.streaming.connectors.kafka.MasonTest']
3. ClassSelector [className = 'org.apache.flink.streaming.connectors.kafka.MasonTest$MasonTest2']

(1) will cause MasonTest1 to be executed, (3) will cause MasonTest2 to be executed, while (2) will cause both to be executed. That explains the double execution. IIRC Surefire scans the test classes dir (target/test-classes) for classes matching MasonTest* (supplied via -Dtest). Thus it finds MasonTest.class but also MasonTest$MasonTest1.class and MasonTest$MasonTest2.class. Interestingly, this does not happen when running all tests (without passing -Dtest). In any case, I think Surefire should filter out nested classes when the enclosing class passes the filter to avoid this. Please raise an issue with the Maven Surefire project.

[mas-chen]

@marcphilipp thanks for the pointers, this helps a lot! I'll try an exclude filter on the nested classes