reportportal/client-net

Report Portal API client for .NET

API client for Report Portal

Provides an ability to interact with Report Portal API in .NET/C#. Supports starting/finishing launches/tests, sending logs.

Setup

Install ReportPortal.Client NuGet package.

PS> Install-Package ReportPortal.Client

Usage

The main entry point to start interact with API is ReportPortal.Client.Service class. It requires uri, project name and uuid. Uuid value is specific for an user and it can be obtained on User Profile page.

var service = new ReportPortal.Client.Service(
new Uri("https://demo.reportportal.com"), "my_project", "my_uuid");

Starting new launch:

var launch = await service.Launch.StartAsync(new StartLaunchRequest
    {
        Name = "LaunchName",
        Description = "LaunchDescription"
    });

To start test item we need to use the LaunchUuid received from the previous step:

var test = await service.TestItem.StartAsync(new StartTestItemRequest
    {
        LaunchUuid = launch.Uuid,
        Name = "Test1",
        Type = TestItemType.Test
    });

To send log item the TestItemUuid is used which was received from the previous step:

var log = await service.LogItem.CreateAsync(new CreateLogItemRequest
    {
        TestItemUuid = test.Uuid,
        Text = "My log",
        Level = LogLevel.Debug
    }); 

Finishing the test:

await Service.TestItem.FinishAsync(test.Uuid, new FinishTestItemRequest
    {
        Status = Status.Passed
    });

Finishing the launch:

await Service.Launch.FinishAsync(launch.Uuid, new FinishLaunchRequest());

client-net's Issues

Reorganize constructors of Service class to remove IWebProxy

This is because of IWebProxy is a part of HttpHandler, not HttpClient.

Users will be able to provide IWebProxy through HttpHandler.

Introduce net5.0 as target framework

With

• System.Text.Json as serializer

It allows:

• Less memory allocation with better performance

To be carefully tested:

• Shared package is not broken
• Agents, which targets on older framework, can work in cooperation with client/shared targeted to net5.0

Benefits of new json serializer with code gen

Make it possible:

• Find out all usages of Serializer in upstream projects
• All upstream projects should use own implementation of serializer
  • NUnit 3
  • XUnit
  • VSTest
• Hide serializer here at all

Supporting batch log item request

RP API is able to receive several log messages in single request.

We want to support it.

Unit tests should be independent on real hosted api

We started writing tests, which interact with real environment, just to make sure that http/serialization/whatever stuff is working somehow. And it would helpful.

We need isolate tests, and at the same time we would have ability to verify all stuff.

Move RetryWithExponentialBackoffHttpClientHandler delegating handler to Shared package

This library should not retry http failed requests, it's should be done by Shared library.

v5: start_time was changed to startTime

the same for end_time

Remove "Newtonsoft.json" dependency

Client uses Newtonsoft.json library to serialize/deserialize objects to/from json format. It would be better if Client will use standard/native possibilities to do it without 3rd party libraries.

Define api interface for service class to make it testable

Implement netstandard to support dotnet core

Support V2 async controllers for posting launch/test/log

Docs about async reporting: https://reportportal.io/docs/dev-guides/AsynchronousReporting

Await all async operations rather than returning Task

Right now

Task<T> SomeMEthod()
{
  return someTask<T>;
}

would be better to await task

async Task<T> SomeMEthod()
{
  return await someTask;
}

In this case stacktrace is better (tracking all nested methods)

Remove net6.0 as target framework

Since we introduced System.Text.Json as a dependency, we don't need supporting of net6.0, netstandard2.0 covers it already.

Add support of custom defect types

Type property from Issue class has IssueType type (see https://github.com/reportportal/client-net/blob/master/src/ReportPortal.Client/Models/TestItem.cs). As a result, only default defect types (PB001, AB001, etc.) are parsed correctly and Type can't be set to a custom defect type.

Consuming GitHub NuGet packages requires authentication?! Move back to MyGet for dev channel

Or support both repositories for dev channel

Use http compression for requests

This allows to be more tolerant for network usage.

Change http to https in package meta

Would be nive to see here https, improves confidence level.

How to use API to get a list of unfinished launches?

I am running many tests in parallel and then merging them into a single launch after the fact. How can I use the API to get a list of launches that are unfinished?

FilterOperation values are not the same in requests from Report Portal UI

In request from Report Portal UI we have !has filter:

Example:

/api/v1/{project}/launch?page.page=1&page.size=50&page.sort=startTime%2Cnumber%2CDESC&filter.%21has.compositeAttribute=Android%2CiOS

Report Portal UI version - 5.7.2

Inside of nuget ReportPortal.Client we have only has filter and there are no !has filter:

https://github.com/reportportal/client-net/blob/3.0.6/src/ReportPortal.Client/Abstractions/Filtering/Filter.cs#L34

public enum FilterOperation
{
[DataMember(Name = "eq")] Equals,
[DataMember(Name = "ne")] NotEquals,
[DataMember(Name = "cnt")] Contains,
[DataMember(Name = "!cnt")] NotContains,
[DataMember(Name = "ex")] Exists,
[DataMember(Name = "in")] In,
[DataMember(Name = "!in")] NotIn,
[DataMember(Name = "gt")] GreaterThan,
[DataMember(Name = "gte")] GreaterThanOrEquals,
[DataMember(Name = "lt")] LowerThan,
[DataMember(Name = "lte")] LowerThanOrEquals,
[DataMember(Name = "btw")] Between,
[DataMember(Name = "size")] Size,
[DataMember(Name = "has")] Has,
}

ReportPortal.Client version - 3.0.6.
Newer version(3.1.0) has the same FilterOperation enum values.

Remove "RestSharp" dependency

Asynchronous API calls to add log items

Make stacktrace of rp service exception shorter

We can easily remove these lines.

To make it just remove async keyword from methods and return Task object directly.

Benefits:

• Short stacktrace without loosing useful info
• Increased performance (ha :))

v5: support launch/test attributes

API does not have the ability to get the list of Debug Launches

As of now i see API does not have the ability to get the List of Debug Launches.
This is useful if we want to delete old debug launches.
Tried to implement in the following in Sevice.Launch.cs in my local and it does give the debug launch list

  public LaunchesContainer GetDebugLaunches(FilterOption filterOption = null)
        {
            var request = new RestRequest(Project + "/launch/mode");
            if (filterOption != null)
            {
                foreach (var p in filterOption.ConvertToDictionary())
                {
                    request.AddParameter(p.Key, p.Value);
                }
            }
            var response = _restClient.ExecuteWithErrorHandling(request);
            return JsonConvert.DeserializeObject<LaunchesContainer>(JObject.Parse(response.Content).ToString());
        }

Extend exception for keeping internal info from http response

client-net/src/ReportPortal.Client/ReportPortalException.cs

Line 9 in e438e2c

{

Like http status code, body, request method

Retry amount threshold clarification

Hello,

Let me first say I've been enjoying the integration of ReportPortal with my NUnit tests. Currently, I've been playing with the retry feature and after reading the doc I found an ambiguous hint as to how the retries are processed:

If there were a few invocations of the one test case... (https://reportportal.io/docs/View-launches>retried-test-case-retry-)

Can we clarify how many "few" is please?

Contract based testing for integration tests

Problem: to execute tests we need real hosted RP API. It fails sporadically.

Solution: https://pact.io is a good candidate for this purpose.

EndTime throws AgrumentNullException for IN_PROGRESS launches

var launch = await service.GetLaunchAsync(id); //get launch which is in progress
var time = launch.EndTime; // throws exception

Update readme with simple example how to use this library

Examples would be

• Reporting several test items with logs to the server
• Get the latest launch and fetch all test items

Pack PDB files in nuget package for debugging purposes

Or upload snupkg separate package to nuget symbol server.

Make StartTime/FinishTime for Start/Finish launch/test/log items as DateTime.UtcNow by default

It also simplifies usage in readme file

Using FinishTestItemRequest.Issue.ExternalSystemIssues with Azure integration fails the report

STR:

1. Perform azure integration
   

2. Add external issues using

https://github.com/reportportal/example-net-nunit/blob/master/src/Example/ReportPortalCustomization/Customization.cs

[Extension]
public class ReportPortalListener : ITestEventListener

BeforeTestFinished += BeforeTestFinishedHandler;    

client-net/test/ReportPortal.Client.IntegrationTests/TestItem/TestItemFixture.cs

Line 585 in 2dc1bcd

//ExternalSystemIssues = new List<ExternalSystemIssue>

private void BeforeTestFinishedHandler(object sender, TestItemFinishedEventArgs e)
{
    e.FinishTestItemRequest.Issue = new()
    {
        Type = _reportParameters.KnownIssueResolutionId,
        Comment = value,
        ExternalSystemIssues = new List<ExternalSystemIssue>
        {
            new ExternalSystemIssue
            {
                TicketId = "47029",
                Url = "https://dev.azure.com/IN**********ve/azure-web/_workitems/edit/47029"
            }
        }
};
}

Actual result:
BeforeTestFinishedHandler executes without exception
But on the ReportPortal UI, the test and the tests run remain not finished

Extra:
When I do not set ExternalSystemIssues - it works.

Versions:

<PackageReference Include="NUnit" Version="3.13.2" />
<PackageReference Include="NUnit.Engine" Version="3.12.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.0.0" />
<PackageReference Include="ReportPortal.NUnit" Version="4.2.0" />
<PackageReference Include="ReportPortal.Shared" Version="3.1.3" />

ExternalSystemIssue.SubmitDate throws exception on debug

Linking the issue from the UI works as expected:

v5: Support 2 strong typed ids in client

To use api user needs 2 ids: id and uuid.

When starting new launch, client should provide 2 ids in model implicitly.

string - uuid   (guid)     (start launch)
long - id

RetryHandler doesn't work properly

HttpClient creates single instance of this handler to send requests. It means that inner counter is shared for all requests, that's definitely is unexpected behavior.

It seems enum UserFilterCondition duplicates enum FilterOperation

The remote certificate is invalid because of errors in the certificate

In case of self-signed certificate (using VsTest runner)

System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: UntrustedRoot

Split methods by separate resources according root endpoint

Currently we have:

service.GetLaunches()
service.StartTestItem()

Class Service has many methods. It would be better to organize them:

service.LaunchResource.GetLaunches()

Make all Async methods cancellable

interface ILaunchResource
Task<MessageResponse> AnalyzeAsync(AnalyzeLaunchRequest request, CancellationToken cancellationToken = default);

CAUTION:
It's not a breaking change on compilation level, but a breaking change at binary level.

Move trimming of long values to Shared package

client-net/src/ReportPortal.Client/Abstractions/Requests/StartLaunchRequest.cs

Line 19 in 6d692f5

public string Name { get { return _name; } set { _name = StringTrimmer.Trim(value, 256); } }

and make it configurable?

How to send files / pictures to the report portal?

Include README file in nuget package

https://devblogs.microsoft.com/nuget/add-a-readme-to-your-nuget-package/

nested step status not correctly displayed on UI

I'm using net-client to log data into rp. When sending FinishAsync request for a NestedStep I expect that in RP the nested step will be marked as failed if it has a log of LogLevel.Fatal in it, but they are always passed unless I provide the Status in the FinishTestItemRequest. Am I doing something wrong?

Follow client-java and move away from GPL license?

Currently the license of this client library obligates any consumers of the library to also be released under the GPL license. This isn't acceptable for essentially any commercial company.

Could this client lib have it's license changed, or if not be dual licensed?

I'm hopeful this is possible as this year the java lib made exactly this change:
reportportal/client-java@ec8b7d1

Fix CI build

https://github.com/reportportal/client-net/actions/runs/4246478938/jobs/7383343432

The issue is dotnet/sdk#30624

Client doesn't throw an exception in case of invalid UUID or Url

Ability to update test item status

Please, add to the Report Portal client an ability to update test item status. It is possible to do it with Report Portal API.

test-item-controller (Test Item Controller)
/v1/{projectName}/item/{itemId}/update

{
"attributes": [
{
"key": "string",
"value": "string"
}
],
"description": "string",
"status": "string"
}

v5: nested steps: log messages can be tree structured

Fetch UUID for authentication in tests project

We are pointing to demo.reportportal.io where UUID is refreshing daily. We know user/pass, I hope it's possible to get UUID each time when we execute tests to keep our CI build green.

Incorrect Request. [Field 'attributes[].value' should have size from '1' to '128'.]

This error occurs when we send long attribute value. Here we need just trim it as we do for launch/item name.

It seems http retry doesn't work properly

Improve ReportPortalException to be structured object

Now we have this error in output

InternalServerError (500) POST http://localhost:8080/api/v1/demo/item)
 ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://localhost:8080/api/v1/demo/item
 ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [Ambiguous handler methods mapped for '/v1/demo/item': {public com.epam.ta.reportportal.demodata.model.DemoDataRs com.epam.ta.reportportal.demodata.DemoDataController.generate(java.lang.String,com.epam.ta.reportportal.demodata.model.DemoDataRq,com.epam.ta.reportportal.commons.ReportPortalUser), public com.epam.ta.reportportal.ws.model.EntryCreatedAsyncRS com.epam.ta.reportportal.ws.controller.TestItemController.startRootItem(java.lang.String,com.epam.ta.reportportal.commons.ReportPortalUser,com.epam.ta.reportportal.ws.model.StartTestItemRQ)}]"}

We see some structured information in http response (errorCode and message). To simplify output, we can try to deserialize response, and throw "good" exception which will keep all information inside appropriate object.

Main goal is simplifying output.