racodond/sonar-gherkin-plugin

SonarQube Cucumber Gherkin Analyzer

SonarQube Cucumber Gherkin Analyzer

Disclaimer

I don't want to keep maintaining this plugin. Feel free to ping me if you want to take over.

Description

This SonarQube plugin analyzes Cucumber Gherkin feature files and:

• Computes metrics: lines of code, number of scenarios, etc.
• Checks various guidelines to find out potential bugs and code smells through more than 40 checks
• Provides the ability to write your own checks

Usage

1. Download and install SonarQube
2. Install the Cucumber Gherkin plugin by a direct download. The latest version is compatible with SonarQube 6.7+.
3. Install your favorite scanner (SonarQube Scanner, Maven, Ant, etc.)
4. Analyze your code

Maven

It is likely that your feature files are not located in source code directories but in test directories. By default, SonarQube doesn't analyze those test directories. Thus, you have to explicitly tell SonarQube to also analyze the test directories containing your feature files.

Let's say that the structure of your project is:

pom.xml
src
  |-- main
        |-- java
        |-- resources
  |-- test
        |-- java
        |-- resources
              |-- features
                    |-- my-feature.feature
                    |-- my-other-feature.feature
                                   
                

In you POM file, you would need to add:

<properties>
  <sonar.sources>pom.xml,src/main/java,src/main/resources,src/test/resources/features</sonar.sources>
</properties>  

Custom Checks

You're thinking of new valuable rules? Version 1.0 or greater provides an API to write your own custom checks. A sample plugin with detailed explanations is available here. If your custom rules may benefit the community, feel free to create a pull request in order to make the rule available in the Cucumber Gherkin analyzer.

You're thinking of new rules that may benefit the community but don't have the time or the skills to write them? Feel free to create an issue for your rules to be taken under consideration.

Metrics

Statements

Number of steps.

Functions

Number of scenarios and scenario outlines.

Classes

Number of features.

Available Rules

• "FIXME" tags should be handled
• "TODO" tags should be handled
• All comments should be formatted consistently
• And and But prefixes should be used instead of redundant Given/When/Then prefixes
• Byte Order Mark (BOM) should not be used for UTF-8 files
• Common Given steps should be added to Background
• Duplicated steps should be removed
• End-line characters should be consistent
• Examples data tables should contain data at least two data rows
• Features should be written in the same language
• Features should have a description
• Features should have a name
• Features should have a unique name
• Features should not contain too many scenarios
• Features that do not define any scenario should be removed
• File names should comply with a naming convention
• Files should contain an empty new line at the end
• Files that do not define any feature should be removed
• Given steps should follow a regular expression
• Given/When/Then steps should be defined in the right order
• Lines should not end with trailing whitespaces
• Missing data table columns should be added
• Non-Given steps should be moved out of Background
• Only tags from the whitelist should be used
• Regular expression on comment
• Scenarios should define at least one of each Given/When/Then step type
• Scenarios should have a name
• Scenarios should have a unique name
• Scenarios should not contain too many steps
• Scenarios that do not define any step should be removed
• Source code should be properly indented
• Spelling mistakes should be fixed
• Star (*) step prefixes should not be used
• Step sentences should not be too long
• Steps of unknown type should not be used
• Tabulation characters should not be used
• Tags should be defined at the right level
• Tags should comply with a naming convention
• Tags should not be set on Examples
• Then steps should follow a regular expression
• There should be one single When step per scenario
• Unused variables should be removed
• Useless tags should be removed
• When steps should follow a regular expression
• Wording should remain at business level

sonar-gherkin-plugin's Issues

add sonarlint supported config prop

https://github.com/SonarSource/sonarlint-core/blob/master/its/plugins/javascript-custom-rules/pom.xml#L60

Make the plugin compatible with SonarQube 6.7

Improve localization of spelling mistakes, non-business words, patterns found in regular expression

Only highlight par of the string that should be fixed

New rule: Useless tags should be removed

Tags in Scenario, Scenario Outlines, Examples that are the same as on Features

New rule: Wording should remain at business level

No words targeting the web interface for example (click, checkbox, ...)

New rule: Features should have a description

Not activated by default

New rule: Scenarios should have a name

Support scenarios with two 'Examples' sections

`CRITICAL SonarQube violation: Parse error at line 97 column 5:

@mytag
Examples:
| unitName | unitCode |
| myUnit | myCode |

Examples:
^
| unitName | unitCode |
| myUnit1 | myCode1 |
`

Because we use tag for some examples, we need to provide two sections of Example. It works pretty nice with Cucumber - at least this is what we verified using cucumber report so I think that the problem is with the parser that does not support such case.

Implements i18n

Gherkin contains many lang ;-)

in this file

https://github.com/racodond/sonar-gherkin-plugin/blob/master/gherkin-frontend/src/main/java/org/sonar/gherkin/parser/GherkinLexicalGrammar.java

it will be useful to use this JSON as a keywords definition from this

https://github.com/cucumber/gherkin-java/blob/master/src/main/resources/gherkin/gherkin-languages.json

New rule: Tags should be defined at the right level

If a tag is defined in all scenarios, it should be defined at feature level instead

New rule: Missing data table column should be added

Scenario Outline: my scenario
  Given I am a customer
  When I add <abc> products to my cart
  Then I should see <abc> products in my cart

  Examples:
    | number |
    | 0      | 
    | 1      |

'abc' is not missing

New rule: Non-Given steps should be moved out of Background

Goal is to set the context only

Allow Feature, Scenario, etc. with no name

Improve description sentence regex

For instance, feature description should allow sentences starting with 'Given'.

New rule: Common Given steps should be added to Background

When all scenarios start with the same step

False positive when a variable is used inside a table

This gherkin code

    Scenario Outline: Create banana with 2 different dates, valid combinations of rateDifference
        Given I have a valid "postBananas.json" json body
        And I have 2 identical elements of "interestRateChanges"
        And I set these elements to:
            | "interestRateChanges[0].date"           | "2025-06-10" |
            | "interestRateChanges[1].date"           | "2025-06-15" |
            | "interestRateChanges[0].rateDifference" | <value0>     |
            | "interestRateChanges[1].rateDifference" | <value1>     |
        When a POST is performed on the endpoint "bananas"
        Then the response code is "201"
        And I get the created banana ID
        And I perform a GET on bananas endpoint with the created banana ID
        And the response code is "200"
        And the response body has these elements:
            | "interestRateChanges[0].date"           | "2025-06-10" |
            | "interestRateChanges[1].date"           | "2025-06-15" |
            | "interestRateChanges[0].rateDifference" | <value0>     |
            | "interestRateChanges[1].rateDifference" | <value1>     |
        Examples:
            | value0 | value1 |
            | 492.0  | -0.01  |
            | -0.01  | -0.012 |

gives the error message "Remove the following unused variables: value0, value1" even though the two variables are indeed used inside other tables.

New rule: Spelling mistakes should be fixed

New rule: Features should have a name

false-positive when pystring contains '<...>'

The rule missing-data-table-column should ignore "pystrings (Doc Strings)" in a scenario (see screenshot, source file https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/integration-tests/features/test_execution_statistics.feature)

Make rule "Source should be properly indented" support i18n

Tight to English keywords for now: checkNameIndentation should be refactored to test one single whitespace after prefix.

New rule: Steps of unknown type should not be used

New rule: Features should be written in the same language

New rule: Scenarios that do not define any step should be removed

New rule: Regular expression on When steps

Not activated by default
Default regex: ^[A-Z].+$

New rule: There should be one single When step per scenario

Feature files not picked up

Hi,
the plugin is not picking up the feature files regardless of me forcefully setting both the language and the folders to include. I'm using maven 3.3.9, maven-sonar plugin 3.2, sonar 6.2 and 1.3 version of your plugin.

Any ideas what I'm doing wrong?

Thanks

Check that .feature files exist in the project

As a Test Manager I am interesting in 1. checking the quality of the tests being written which this helps with and 2. I want to insure my teams are using cucumber in the first place so a rule that checks that feature files exist would be great

Make rule "Spelling mistakes should be fixed" support i18n

Not only on english we write feature

it will be usefull to add to this class support of all language

https://github.com/racodond/sonar-gherkin-plugin/blob/master/gherkin-checks/src/main/java/org/sonar/gherkin/checks/SpellingCheck.java

in our proof of concept we use method getLanguage in Feature file
and after that get JLanguageTool class by this lang-code

 String featureLanguage = feature.getLanguage();
    if ("en".equals(featureLanguage)) {
      featureLanguage = "en-US";
    }

    Language language = Languages.getLanguageForShortName(featureLanguage);
    JLanguageTool langTool = new JLanguageTool(language);

New rule: Unused variable should be removed

Scenario Outline: my scenario
  Given I am a customer
  When I add <number> products to my cart
  Then I should see <number> products in my cart

  Examples:
    | number | type  |
    | 0      | type1 |
    | 1      | type2 |

'type' is not used

Metrics in SonarQube 6.6

Thanks for making this plugin available.
Probably more a question based on my inexperience (and outdated docs?):

I was wondering, how to use the metrics mentioned in the Readme, I'm not seeing them in sonarqube 6.6, JS project.
Only the defaults in /admin/custom_metrics.

Improve Indentation rule

one space only before names

New rule: Scenarios should have a unique name

New rules: Check tense of each kind of step

FP on rule unused-variable: Variable in doc string not recognized

A variable in a doc string (multiline string) is not recognized and the bug "Remove the following unused variable" is issued.

Example code for reproducing this bug:

Feature: CRUD operations on mocked banana prices

    Background:
        Given that i have a banana WS in "local" environmnet

    Scenario Outline: 4. Verify error messages for updates with invalid values
        When I perform a PUT request for "mock/bananaprices/12345"
            """
            {
                "code":"79365",
                "price":<value>
            }
            """
        Then I receive the response code 400
        Examples:
            | value |
            |       |
            | aaa   |
            | "aaa" |
            | false |

Spelling check: add the ability to ignore some rules

New rule: Tags should not be set on Examples

Put tags on Scenario Outlines instead

New rule: All features should be written in the same language

please provide stand alone maven example to use it

Hi,
Can you please provide a standalone maven example? Or may be provide link to the source code to the demo project?
Thanks much.

Plugin not working when used with SonarCloud

Hello,

So I wanted to use the plugin with SonarCloud, so I added the following in the pom.xml file of my Java project.

<dependency> <groupId>com.racodond.sonarqube.plugin.gherkin</groupId> <artifactId>sonar-gherkin-plugin</artifactId> <version>1.7</version> </dependency>

<sonar.sources>pom.xml,src/main/java,src/test/resources/features</sonar.sources> (properties section).

I ran the command, "mvn clean install sonar:sonar". The command ran successfully, but the Gherkin plugin did not run or find any issues.

Is there anything that I'm missing?

New rule: Given/When/Then steps should be defined in the right order

Compliant:

Given I am a customer
When I add a product to my cart
Then I should see the product in my cart

Noncompliant:

Given I am a customer
When I add a product to my cart
Then I should see the product in my cart
When I proceed to the order                            # Noncompliant
Then I should see the payment mode choice
...

New rule: Scenarios should define at least one of each Given/When/Then step type

False positive on Common Given steps should be added to Background

Background doesn't support examples, so rule shouldn't apply to steps that require parameters.

New rule: And and But prefixes should be used instead of redundant Given/When/Then prefixes

Compliant:

Given I am a customer
And I have a product in my cart
...

Noncompliant:

Given I am a customer
Given I have a product in my cart
...

New rule: Step sentences should not be too long

100 characters

Support ``` as DocString prefix/suffix

On top of """

New rule: Features should have a unique name

New rule: Regular expression on Then steps

Not activated by default
Default regex: ^[A-Z].+$

Can be customized for instance as "^I should [.+]$

Change 'allowed-tags' rule parameter from a comma-separated list to a regular expression

@racodond We use SpecFlow and want to use references in feature tags to synchronize TFS work items or test cases (https://www.youtube.com/watch?v=uhZH81sMMfs).
I implemented a plugin already with the rule "allowed-composite-tag" (see https://github.com/Bertk/sonar-gherkin-custom-rules-plugin/).
Can this rule be added to the SonarQube Gherkin plugin?
The current solution of a separate plugin with only one single rule is somehow overdone.

see #58

New rule: Regular expression on Given steps

Not activated by default
Default regex: ^[A-Z].+$

New rule: Duplicated steps should be removed