troyschmidt/runtsqlttests-azdo

A pipeline task to run tSQLt tests and extract its results and Code Coverage

Run tSQLt Unit Tests

What's new

Version 0.2.1

• Added the option to run only a Test Class or Test Case (Issue #6)
• Minor improvements on the task UI

Description

With this build task, besides running tSQLt Unit Tests on a SQL Database, you'll also be able to:

• Export the test results to the build summary;
• Export the code coverage report to the build summary.

Originally, there is no build task that does all those 3 tasks. Thinking from an automation and engineering perspective, once you run your unit tests, it would be good to have its results published somewhere to inspect and adapt your tests.

This task relies on 3 components:

SQLCover

Link: https://github.com/GoEddie/SQLCover

This is an Open Source component written by Ed Elliot to extract code coverage from a tSQLt test execution.

OpenCover to Cobertura Converter

Link: https://github.com/danielpalme/OpenCoverToCoberturaConverter

An Open Source tool written by Daniel Palme to convert OpenCover code coverage reports to Cobertura, a coverage report supported by Azure DevOps. See more here.

Report Generator

Link: https://github.com/danielpalme/ReportGenerator

Daniel Palme also wrote a tool to convert XML Code Coverage reports into human readable reports in various formats.

Flow

Basically, this extension performs the following tasks, in order:

• Run tSQLt using SQLCover;
• Exports the test results using TSQL Script;
• Save Stored Procedures source files for generating the Code Coverage report;
• Converts the Open Cover XML report into human readable report using Report Generator;

Preview

This extension is still in preview to gather information on using, feedback and tracking initial issues.

How to use it

TL;DR

If you just want your test (with or without Code Coverage) and you will not customize anything right now, just follow these steps:

• Add the Run tSQLt Unit Tests task;
  • Add your connection string to the SQL Server Connection String field;
  • Check Control Options/Continue on error;
• Add the Publish Test Results task;
• If you're enabling Code Coverage
  • Add the Publish Code Coverage Results task;
  • Change Code coverage tool to Cobertura;
  • Change Summary file to $(System.DefaultWorkingDirectory)/tSQLt-out/Cobertura.xml;
  • Change Report directory to $(System.DefaultWorkingDirectory)/tSQLt-out/AzurePipelines

Add the Run tSQLt Unit Tests build task to your pipeline

Search for tsqlt and make sure to add the task to your pipeline:

General configuration

The only important thing here is to highlight that the working directory for the task will be the build sources directory. You can change this but it's recommended to keep it as it is.

Database Information

Here, two required information:

• SQL Server Connection String: Make sure to test your connection string before using it here. A good source for connection strings is the Connection Strings website.

• SQL Connection Timeout: It's recommended to use 60 (seconds) as a timeout for your connection. Be careful not to add an amount of time here that could make your build stuck on your queue.

Test Execution

You'll have default information filled here that is supposed to make it work by default with the requirements of the Publish Test Results task. You can also customize it.

• The root folder to store all the output for this task: You must be aware that all the output from this task will be dropped under working $(System.DefaultWorkingDirectory)/tSQLt-out (or whatever you configure here).

• The name for the Test Results XML report: This is the file that the Publish Test Results task will use to publish the results to the summary. Its default value is already discoverable by the Publish Test Results task's default configuration.

Code Coverage

By default, Code Coverage is disabled. If you want the task to also generate Cobertura code coverage reports, you must enable it. Remember that all those paths are relative to working directory\root folder

• Folder to store the Open Cover generated source: It will be useful to show detailed code coverage report.
• The name for the Cobertura Code Coverage XML report: You'll use this format to Publish Code Coverage report later.
• HTML reports output: You'll need to publish this folder as an artifact so it will be shown as part of the build summary.

Control options

VERY IMPORTANT

If any of your tests fail, the task will also fail. Because of that, you must mark the Continue on error option so your pipeline will publish test results and code coverage reports, if enabled.

Add the Publish Test Results to the pipeline

To have the test results available on the pipeline summary, you must publish it. To do that, you just need to use the Publish Test Results task:

Configure important parameters

• Test result format: The test results tSQLt provides is JUnit based.
• Test results file: The file name you configured on The name for the Test Results XML report.
• Search folder: It is the combination of the working directory and the root output. If you left everything default, it should be $(Build.SourcesDirectory)/out

Add the Publish Code Coverage Results task (if Code Coverage Enabled)

If you enabled Code Coverage on the Tests task, you'll need to publish its results to the pipeline summary.

• Code coverage tool: You must select Cobertura as this task produces the results using this format.
• Summary file: The combination of working directory, root output folder and the file name you configured on The name for the Cobertura Code Coverage XML report.
• Report directory: The directory containing the HTML reports output.