koreyoshi-wang/allure-csharp

Allure integrations for C# test frameworks

Allure c# Integrations

Allure.Commons

.Net implementation of Allure java-commons.

Can be targeted either by legacy .net 4.5+ or .net standard 2.* projects.

Use this library to create custom Allure adapters for .Net test frameworks.

Configuration

Allure lifecycle is configured via json file with default name allureConfig.json. NuGet package installs allureConfig.Template.json which you can use as an example. There are 2 ways to specify config file location:

• set ALLURE_CONFIG environment variable to the full path of json config file. This option is preferable for .net core projects which utilize nuget libraries directly from nuget packages folder. See this example of setting it via code: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.Commons.NetCore.Tests/AllureConfigTests.cs#L13-L15

• place allureConfig.json to the location of Allure.Commons.dll. This option can be used with .net classic projects which copy all referenced package libraries into binary folder. Do not forget to set 'Copy to Output Directory' property to 'Copy always' or 'Copy if newer' in your test project or set it in .csproj:

<ItemGroup>
<None Update="allureConfig.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>

Allure lifecycle will start with default configuration settings if allureConfig.json is not found.

Raw json configuration can be accessed from AllureLifeCycle.Instance.JsonConfiguration to extend configuration by adapters. See extension example here: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.SpecFlowPlugin/PluginHelper.cs#L20-L29

Base configuration params are stored in AllureLifeCycle.Instance.Configuration

Allure configuration section is used to setup output directory and link patterns, e.g.:

{
  "allure": {
    "directory": "allure-results", // optional, default value is "allure-results"
    "title": "custom run title", // optional
    "links": //optional 
    [
      "https://example.org/{link}",
      "https://example.org/{issue}",
      "https://example.org/{tms}"
    ]
  }
}

All Link pattern placeholders will be replaced with URL value of corresponding link type, e.g.

link(type: "issue", url: "BUG-01") => https://example.org/BUG-01

AllureLifecycle

AllureLifecycle class provides methods for test engine events processing.

Use AllureLifecycle.Instance property to access.

Fixture Events

• StartBeforeFixture
• StartAfterFixture
• UpdateFixture
• StopFixture

Testcase Events

• StartTestCase
• UpdateTestCase
• StopTestCase
• WriteTestCase

Step Events

• StartStep
• UpdateStep
• StopStep

Attachment Events

• AddAttachment - adds attachment to the current lifecycle executable item
• AddScreenDiff - adds needed artifacts to the test case with given uuid to be used with screen-diff-plugin

Utility Methods

• CleanupResultDirectory - can be used in test run setup to clean old result files

Troubleshooting

...

SpecFlow Adapter

Currently supports SpecFlow v2.1 - 3.1.*

See Allure report generated from https://github.com/allure-framework/allure-csharp/tree/master/Allure.Features

Please use corresponding NuGet package version.

Installation

Install Specflow.Allure nuget package according to your Specflow version.

Configuration

For Specflow 3 please add or update the following section in your specflow.json:

"stepAssemblies": [
 { "assembly": "Allure.SpecFlowPlugin" }
]

The plugin uses Allure.Commons json configuration extended with custom sections.

Custom host name

In case if you want to customize host name which is displayed in Allure Timeline section, please configure allure.title property in json configuraion file.

If you use NUnit

Default value for allure.directory in allureConfig.json is "allure-results", default working directory in NUnit 3.* is the working directory of console runner. If you don't want to place allure results into NUnit default working folder please either set absolute path in allure.config or set working directory for NUnit in your test setup, e.g.:

[OneTimeSetUp]
public void Init()
{
   Environment.CurrentDirectory = Path.GetDirectoryName(GetType().Assembly.Location);
}

Usage

Just run your SpecFlow scenarios and find allure-results folder ready to generate Allure report.

Features

Grouping

You can structure your scenarios in 3 Allure hierarchies using feature and scenario tags. Please read report structure Allure documentation section for additional details. Hierarchies consist of the following levels:

Suites

• Parent Suite
• • Suite
• • • Sub Suite

Behaviors

• Epic
• • Feature
• • • Story

Packages

• Package
• • Class
• • • Method

The plugin uses allure:grouping configuration section to parse tags with the regular expression. If the expression contains the group, it will be used as hierarchy level name otherwise entire match will be used. E.g:

^mytag.* : any tags starting with @mytag will be used for grouping.

^type:(ui|api) : @ui or @api tags from regex pattern will be used for grouping.

Check this config example as a starting point.

Links

You can add links to your scenarios using tags. Tag and link patterns can be configured in allureConfig.json

{
  "allure": {
    "links": [
      "https://myissuetracker.org/{issue}",
      "https://mytestmanagementsystem.org?test={tms}"
    ]
  },
  "specflow": {
    "links": {
      "link": "^link:(.+)",
      "issue": "^\\d+",
      "tms": "^tms:(\\d+)"
    }
  }
}

The following scenario

@123456 @tms:42 @link:http://example.org 
Scenario: I do like click on links in Allure report 

will have three links in Allure report: 123456, 42 and http://example.org.

In case there are links, which are generated during tests, then they can be added in code via AllureLifecycle:

AllureLifecycle.UpdateTestCase(testResult.uuid, tc =>
            {
                tc.links.Add(new Link()
                {
                    name = "Example link",
                    url = "http://example.com"
                });
            });

This will show for scenario link with Text: Example link; and url: "http://example.com".

Severity

You can add Severity for your scenarios using tags. It can be configured in allureConfig.json

 "labels": {
      "severity": "^(normal|blocker|critical|minor|trivial)"
    },

The following scenario

@critical
Scenario: ....

will set current scenario severity in Allure report as Blocker

Tables conversion

Table arguments in SpecFlow steps can be converted either to step csv-attacments or step parameters in the Allure report. The conversion is configurable in specflow:stepArguments config section. With specflow:stepArguments:convertToParameters set to true the following table arguments will be represented as parameters:

• one row tables
• two column tables with the headers matching both specflow:stepArguments:paramNameRegex and specflow:stepArguments:paramValueRegex regular expressions.

SpecFlow	Allure

Attachments

You can add attachments to an Allure report from your step bindings:

using Allure.Commons;
...
AllureLifecycle.AddAttachment(path, "Attachment Title");
// or
AllureLifecycle.AddAttachment("Attachment Title", "application/txt", "path");
// where "application/txt" - type of your attachment