testsmells/testsmelldetector

A tool to detect test smells in Java projects that utilize JUnit as the testing framework

testsmelldetector's Issues

Verbose test smell threshold set to 123?

Hi! Thanks for the tool!
I was looking at the threshold of Verbose Test, and it's currently set to 123 statements. I think this is an error, it seems waaay to high, and also it's kind of a magic number (first 3 numbers of the keyboard). Is it a leftover? What's the correct threshold?

TestSmellDetector/src/main/java/testsmell/smell/VerboseTest.java

Line 61 in 08be07d

final int MAX_STATEMENTS = 123;

Linux support?

When I try to run TestSmellDetector on a UNIX-like system the following error is printed out:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 1
        at testsmell.TestFile.getTagName(TestFile.java:44)
        at Main.main(Main.java:91)

This error references the following line of code:

    public String getTagName(){
        return testFilePath.split("\\\\")[4];
    }

Which explicitly deals with backslashes (used on windows path). So assuming TestSmellDetector is Windows-only: Is Linux support planned?

Add comment in source pointing to the detected smell

The first time I ran the tool, one of smell it flagged was "Assertion Roulette", but when I went into the test file, it was not clear to me whether it flagged a false positive or if I just couldn't detect the smell myself. It would be super helpful if I had some indication (like an inserted comment) of exactly what the tool was thinking was a smell.

Improve documentation

https://testsmells.org/pages/testsmells.html#ExceptionHandling

Does not define:

Exception Catching Throwing
Print Statement
Verbose Test

Inputs cannot be fed in batches

Currently if I have 10 test codes, there are 10 input files needed containing the test and production code paths and can be executed only one at a time. Also, there are 10 separate output files generated as a result of this. It would be helpful if the inputs can be passed in batches or can be done for all test files where it automatically identifies and associates itself to the source codes for the same.

Does not work on macOS (or Linux)

It'll throw a java.lang.ArrayIndexOutOfBoundsException: 4 on ClassEntity.getAppName() as it expects a Windows path format, instead of using an universal solution.

Maven / Gradle plugin?

It would be awesome to have integration with build tools such as the ones mentioned, which could then be parameterized to even fail the build in case of too many test smells, occurrence of some specific test smell, etc.

Mention the location of the test smell in the test files

update readme

Check the test smell called "Exception Catching Throwing". I think there is duplicate.

Test Smell Tool Detect Test level file or Functional Level ?

Hi Folks
My Thesis is based on test smells I need some help that Test Smell tool detect are based on Test level files (Test case file has many test cases) or Function Level files (Test case file contain different function to be tested) and I want's to know that Test smell Tool give results on the basis of ( single test case file that contain all result of test cases) or ( test case file give different results of different test cases ) ?

If test case file give results of different test cases, then which method are used to separate the different function from Test cases other wise it's time taking to remove test cases from every file and makes copies, every copies contain 1 test case.

My Thesis work is Test smell tools detect at functional Level.

@shehan
@giograno
@KhalidSAlmalki

Once found "true", never goes "false"

Run this using attached file on ftp-server 1.0.6. One example - Print Statement - returns True correctly for clienttests/SymbolicLinkTest.java, but remains True even right after that with no print statement - clienttests/ListPassiveTest.java. This behavior is observed on many code smell categories, indicating that clearing of status is not properly done.

Output_TestSmellDetection_1647781715304.csv

Detected smells are wrong

In the latest version of this tool, whether it is release v2.1 or on the latest commit on main, the reported smells tend to be innexact.

When using the tool, we noted inconsistencies in the reported smells. The results would usually look like this:

File	Smell 1	Smell 2	Smell 3
test1.java	false	true	false
test2.java	true	true	false
test3.java	true	true	false
test4.java	true	true	false

This is a sample table, displaying why we thought there was an issue, not representing real data.

When a smell gets detected in a file, it systematically gets detected in the next file, which is particularly shocking when a project has more than 20 test files. For over 500 of projects, the mean number of detected smells was around 8, which is a lot compared to what has been observed in other studies.

To obtain such result we input a .csv file mapping each test file to its production file (when possible), which means several files will be processed in a row.

I plan to do a pull request fixing this issue. More information on the technical details will be available there.

[AssertionRoulette] Handle Assertj's methods `Descriptable.as` and `AbstractAssert.withFailMessage`

As discussed in a stackoverflow question. AssertJ's assertions differ from builtin Junit's assertions as they don't have third or second parameters for descriptions. Instead, they offer the Descriptable.as and AbstractAssert.withFailMessage methods.

On the other hand, TSDetect's AssertionRoulette implementation ( at /src/main/java/testsmell/smell/AssertionRoulette.java) handles an homonymous assertion from Junit 4 (assertThat) which is deprecated. For projects using AssertJ, this could raise the false impression that test methods using Descriptable.as or AbstractAssert.withFailMessage are description-less.

Integration of TS-Detect with IDE

Integrating TS-Detect with IDE in the form of a plugin would be a great step forward, making it easier and convenient for the developers to use the tool.

TS-Detect was not giving output on Mac

Mac OS: 12.3.1
Chip: Apple M1 (not intel)
Run method attempted: Docker
Test project; https://github.com/Addltariq10/mockito

I pick up test files and its production files in the csv, using the terminal, i executed the java -jar pathname.jar pathname.csv but it didnt give output in csv, although the same file worked fine on another windows laptop,