microsoft/kiota-http-dotnet

Kiota http provider implementation for dotnet with HttpClient

Kiota Http Library for dotnet

The Kiota HTTP Library for dotnet is the dotnet HTTP library implementation with HttpClient.

A Kiota generated project will need a reference to a HTTP package to make HTTP requests to an API endpoint.

Read more about Kiota here.

Using the Kiota Http Library for dotnet

dotnet add package Microsoft.Kiota.Http.HttpClientLibrary --prerelease

Debugging

If you are using Visual Studio Code as your IDE, the launch.json file already contains the configuration to build and test the library. Otherwise, you can open the Microsoft.Kiota.Http.HttpClientLibrary.sln with Visual Studio.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

kiota-http-dotnet's Issues

Defaulting to HTTP/2.0 breaks consumers using .NET Framework

#231 introduces a breaking change for consumers that use .NET Framework, as HTTP/2.0 is not a supported version in HttpRequestMessage.

Minimal repro with <TargetFramework>net472</TargetFramework>:

var adapter = new HttpClientRequestAdapter(new AnonymousAuthenticationProvider());
await adapter.SendNoContentAsync(new RequestInformation()
{
    URI = new Uri("http://example.com")
});

This works as expected on Microsoft.Kiota.Http.HttpClientLibrary version 1.3.8, but throws the following exception on 1.3.9 and 1.3.10:

Unhandled Exception: System.ArgumentException: Only HTTP/1.0 and HTTP/1.1 version requests are currently supported.
Parameter name: value
   at System.Net.HttpWebRequest.set_ProtocolVersion(Version value)
   at System.Net.Http.HttpClientHandler.CreateAndPrepareWebRequest(HttpRequestMessage request)
   at System.Net.Http.HttpClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)

UriReplacementHandler issues

UriReplacementHandler is hard to use because it differs from the other handlers in the following ways:

1. It isn't added by default.
2. It doesn't inspect HttpRequestMessage instances for options, instead only respecting the option give to the handler upon construction. This means that every time a different option is needed, the consumer of the middleware stack needs to be rebuilt with new middlewares.

Are these intentional? The docs seem to imply that all handlers can be used in the same way and should work out of the box, not giving any mention of needing to add handlers manually. If these choices are intentional, then maybe the docs could be made clearer to help readers better understand how to use this handler. It was only by reading through source code that I realized that this handler functions differently.

Is there any way to get response body when status code is unsuccessful?

I generated a client with Kiota, based on an OpenAPI spec, which does not specify error codes, hence errorMapping is empty in Kiota-generated client. The ApiException that Kiota throws contains just the status code and response headers. Is there any way to read the actual response body (as string) without modifying the Kiota-generated client code?

How do I react to unexpected status codes?

I originally posted this to the main https://github.com/microsoft/kiota repo, but I figured I should also ask here. Close whichever one you feel like doesn't make sense.

Original post

Rename ThrowFailedResponse to ThrowIfFailedResponse

This method name confused me. How do folks feel like about renaming it to ThrowIfFailedResponse?

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Lines 88 to 94 in 20633e2

	var responseHandler = GetResponseHandler(requestInfo);
	if(responseHandler == null)
	{
	try {
	await ThrowFailedResponse(response, errorMapping, span);
	if(shouldReturnNull(response)) return default;
	var rootNode = await GetRootParseNode(response);

Add static analyzers dotnet Abstraction libraries

Add static anaylzers for Core and Abstraction libraries and cleanup the readmes to reflect this

This will mainly involves setting up Sonar cloud for the abstraction libs and fixing validation pipelines once the abstraction libs are moved to a separate repo.

AB#10960

KiotaClientFactory doesn't build a httpclient that will work because it is missing auth middleware

kiota-http-dotnet/src/KiotaClientFactory.cs

Line 36 in eb76ee8

return new List<DelegatingHandler>

This issue is related to this ADR microsoft/kiota#3941

We need to think about building an AuthHandler middleware that can use our Kiota Auth Providers so that we can have native HttpClient experiences. Otherwise, we can't satisfy this ADR microsoft/kiota#3200

ApiException should return mapped error response if it parses successfully

Current Behavior

At the moment, when an HTTP request receives an error response, the response is parsed using a ParsableFactory<IParsable> that's passed in as a dictionary of HTTP Status Code -> Factory.

At the moment, when the error response is successfully parsed, it's used to set an activity tag for telemetry, then checked whether it's an exception or not.

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Lines 410 to 423 in 3c35d1c

	var result = rootNode?.GetObjectValue(errorFactory);
	SetResponseType(result, activityForAttributes);
	spanForDeserialization?.Dispose();
	if(result is not Exception ex)
	throw new ApiException($"The server returned an unexpected status code and the error registered for this code failed to deserialize: {statusCodeAsString}")
	{
	ResponseStatusCode = statusCodeAsInt,
	ResponseHeaders = responseHeadersDictionary
	};
	if(result is ApiException apiEx)
	{
	apiEx.ResponseStatusCode = statusCodeAsInt;
	apiEx.ResponseHeaders = responseHeadersDictionary;
	}

Desired Behavior

It would be useful for the parsed response to be bubbled up to the caller in the thrown ApiException so that this can be accessed in the calling code.

API Changes

This would depend on an API change in Microsoft.Kiota.Abstractions in ApiException to accommodate the error response value.

public class ApiException : Exception
{
    public ApiException() : base()
    public ApiException(string message) : base(message)
    public ApiException(string message, Exception innerException) : base(message, innerException)

    public int ResponseStatusCode { get; set; }
    public IDictionary<string, IEnumerable<string>> ResponseHeaders { get; set; }
+   public object? ResponseObject { get; set; }
}

Package should be using a license expression

Summary

I wish for the nuget packages to have the licence expression property set correctly

Details

The licence expression property should be set to the correct licence type I.e. MIT as this will enable analysis of licences in use to occur in external tools & the license type will be shown in Nuget etc.

ConfigureAwait(false) calls missing

ConfigureAwait(false) is missing from a number of async calls across the HttpClientLibrary project, which may cause issues with consuming projects that have a synchronization context e.g. ASP.NET projects.

map XXX error status code

related to microsoft/kiota#4025
add a 3rd case in the error handling of the request adapter for XXX (must be the 3rd in the order, first sepcific code, then 4XX or 5XX depending on the first number, then XXX)

System.Text.Json dependency constraint for .NET 8 scenarios

Hey, I use Microsoft.Graph package for the dotnet/Scaffolding project. For the new .NET 8 release of the scaffolding package, the dependency constraint this project has on System.Text.Json being [6.0, 8.0) is causing issues since it causes the default .NET 8 project's dependency on System.Text.Json to downgrade causing an error (dependency downgrade). Unless you have a workaround you can suggest.

Streams returned from the HttpClientRequestAdapter cannot be used

Streams returned from HttpClientRequestAdapter methods like SendPrimitiveAsync<Stream> cannot be used. The methods return disposed streams. I believe the culprit is the new DrainAsync method.

Suggestion: KiotaClientFactory should allow for setting handler options

My use case is a custom user agent. Using the provided factory (in C#), I need to set request options on every call to pass the UserAgentHandlerOptions.

Ideally, I could set the options when the default handlers are created. For example:

static IList<DelegatingHandler> CreateDefaultHandlers(UserAgentHandlerOption? userAgentHandlerOption = null)
{
  return new List<DelegatingHandler>
  {
    //add the default middlewares as they are ready
    new RetryHandler(),
    new RedirectHandler(),
    new ParametersNameDecodingHandler(),
    new UserAgentHandler(userAgentHandlerOption),  // <-- this
    new HeadersInspectionHandler(),
  };
}

Obviously, having a parameter for every handler becomes unwieldy, so perhaps there is a better approach.

I thought about enumerating the resulting list, but the base DelegatingHandler class does not understand the Kiota options:

var handlers = CreateDefaultHandlers();
foreach (var handler in handlers)
{
  if (handler is UserAgentHandler userAgentHandler)
  {
    userAgentHandler.Options = userAgentHandlerOption;  // <--  compile error
  }
}

All comments appreciated. (I'm happy to submit a PR once we have a solid approach.)

Blazor WASM client unable to send web requests through HttpClient BaseAddress

This issue occurs for Blazor WASM applications consuming an OpenAPI spec that does not supply a base URL. Blazor WASM does not support the System.Net.Http.HttpClientHandler class, so those apps must supply an HttpClient to the HttpClientRequestAdapter to work.

The issue is that the BaseAddress property on the supplied HttpClient object does not infer that as the HttpClientRequestAdapter's BaseUrl, and it must be set separately for requests to be sent appropriately.

How to reproduce:

1. Create a Blazor WASM application with .NET 7 using the empty template (For the sake of example, named "KiotaTestApp.FrontEnd")

2. Generate a client from an API spec that does not supply a Base URL. E.g. (kiota generate -d http://<url>/swagger/v1/swagger.json -n KiotaTestApp.FrontEnd.Client -o ./Client -l CSharp -c ApiClient

   Note: The dotnet-aspnet-codegenerator tool can be used to scaffold an API with an OpenAPI spec that reproduces the above scenario.

3. Note the following warning that appears in the console output:

   warn: Kiota.Builder.KiotaBuilder[0]
         OpenAPI warning: #/ - A servers entry (v3) or host + basePath + schemes properties (v2) was not present in the OpenAPI description. The root URL will need to be set manually with the request adapter.
   warn: Kiota.Builder.KiotaBuilder[0]
         No server url found in the OpenAPI document. The base url will need to be set when using the client.
   

4. Add the Kiota API client service setup in a Blazor WASM application's Program.cs file like so:

   // ...
   var authProvider = new AnonymousAuthenticationProvider();
   var client = new HttpClient { BaseAddress = new Uri("http://<url>") };
   var requestAdapter = new HttpClientRequestAdapter(authProvider, httpClient: client );
   builder.Services.AddScoped<ApiClient>(_ => new ApiClient(requestAdapter));
   // ...

5. In the Index.razor (or any other page) inject the client and create a method that sends an HTTP request with it

6. Launch the application, and view that requests are not sent to the HttpClient's BaseAddress

Workaround

Explicitly setting the BaseUrl after adding an HttpClient applies the parameter value, allowing the client to send requests.

// ...
var authProvider = new AnonymousAuthenticationProvider();
var client = new HttpClient { BaseAddress = new Uri("http://<url>") };
var requestAdapter = new HttpClientRequestAdapter(authProvider, httpClient: client );
requestAdapter.BaseUrl = "http://<url>";
builder.Services.AddScoped<ApiClient>(_ => new ApiClient(requestAdapter));
// ...

KiotaClientFactory is missing key helper methods in C#

Maybe I am missing something but this is painful....

A key scenario is that someone wants to use Kiota "out of the box" but with one more MessageHandler. e.g. CompressionHandler because it isn't there by default yet.

To customize the message handling pipeline it is necessary to pass in a "custom" HttpClient. e.g.

var requestAdapter = new HttpClientRequestAdapter(new AnonymousAuthenticationProvider(), httpClient:client);

You can create your own HttpClient by doing

var client = KiotaClientFactory.Create();

This is great if you want to set custom headers that are sent with every request. Having said that, if you set an Accept header it will get overridden by RequestBuilders. That's not ideal.
However, if you want to add a message handler, you need to use the overload that accepts a messagehandler pipeline. I'm not sure why we dropped this terminology for "chain" as pipeline has been the term used since HttpClient was originally created.

You can create a customized set of handlers by calling the CreateDefaultHandlers method and adjusting it. e.g.

       var handlers = KiotaClientFactory.CreateDefaultHandlers();
        handlers.Insert(0, new CompressionHandler());

However, there is no obvious way to use that list. I tried this

var pipeline = KiotaClientFactory.ChainHandlersCollectionAndGetFirstLink(KiotaClientFactory.GetDefaultHttpMessageHandler(), handlers);

but unfortunately, we made ChainHandlersCollectionAndGetFirstLink accept "params". I had to Bing how to use a list to provide "params". So in the end, this works.

        var handlers = KiotaClientFactory.CreateDefaultHandlers();
        handlers.Insert(0, new CompressionHandler());
        var pipeline = KiotaClientFactory.ChainHandlersCollectionAndGetFirstLink(KiotaClientFactory.GetDefaultHttpMessageHandler(), handlers.ToArray());
        var client = KiotaClientFactory.Create(pipeline);
        var requestAdapter = new HttpClientRequestAdapter(new AnonymousAuthenticationProvider(), httpClient:client);

I would like us to consider adding an overload to Create() that accepts a list of handlers and optionally a non-default finalhandler. This would allow a developer to do the following.

        var handlers = KiotaClientFactory.CreateDefaultHandlers();
        handlers.Insert(0, new CompressionHandler());
        var client = KiotaClientFactory.Create(handlers);
        var requestAdapter = new HttpClientRequestAdapter(new AnonymousAuthenticationProvider(), httpClient:client);

I don't see a whole lot of value making our customers convert the list to a pipeline. We can do that as an implementation detail.

SocketsHttpHandler as final handler

I noticed in #237 the default handler on .NET Framework was changed to WinHttpHandler.

Should the default on .NET (Core) be SocketsHttpHandler rather than HttpClientHandler?

It was changed to the default handler in .NET Core 2.1 and the docs say

A significant performance improvement when compared with the previous implementation.

Pass HttpCompletionOption.ResponseHeadersRead to HttpClient for Stream responses

Related to microsoftgraph/msgraph-sdk-dotnet#1954

For calls to SendPrimitiveAsync<Stream> we should pass the HttpCompletionOption.ResponseHeadersRead to the HttpClient to avoid the loading of the entire stream buffer into memory as the requested stream may be unnecessarily large.
This could be done by passing the optional parameter once the return type is determined to be that of a Stream.
https://github.com/microsoft/kiota-http-dotnet/blob/main/src/HttpClientRequestAdapter.cs#L410

We can avoid this for other scenarios as typed return are not expected to run into this issue as the size is not expected to grow as much compared to an unknown Stream size.

Null reference exception when template is null

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Line 72 in b80e07c

var telemetryPathValue = queryParametersCleanupRegex.Replace(decodedUriTemplate!, string.Empty);

A requestInfo object may be created with just the raw url and thus have no template. The line above results in throwing and error if not provided

Microsoft.Graph.Models.ODataErrors.ODataError: Signature generation failed on client with:Cannot access a disposed object.

Using this through <PackageReference Include="Microsoft.Graph.Core" Version="3.1.10" />

Microsoft.Graph.Models.ODataErrors.ODataError: Signature generation failed on client with:Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeNCryptKeyHandle'., unable to proceed sending token request
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.ThrowIfFailedResponse(HttpResponseMessage response, Dictionary`2 errorMapping, Activity activityForAttributes, CancellationToken cancellationToken)
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.SendPrimitiveAsync[ModelType](RequestInformation requestInfo, Dictionary`2 errorMapping, CancellationToken cancellationToken)
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.SendPrimitiveAsync[ModelType](RequestInformation requestInfo, Dictionary`2 errorMapping, CancellationToken cancellationToken)
   at Microsoft.Graph.Users.Item.Photo.Value.ContentRequestBuilder.PutAsync(Stream body, Action`1 requestConfiguration, CancellationToken cancellationToken)

Activity source memory leak

When the HttpClientRequestAdapter is resolved from a service provider using either transient or scoped lifetime and passing a HttpClient then it leaks memory. The leak is caused by the creation of an ActivitySource every time the HttpClientRequestAdapter is constructed but it's never disposed unless the HttpClientRequestAdapter created the HttpClient.

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Line 55 in eb76ee8

activitySource = new(obsOptions.TracerInstrumentationName);

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Lines 604 to 608 in eb76ee8

	if(createdClient)
	{
	activitySource?.Dispose();
	client?.Dispose();
	}

This problem was introduced in commit e44fbfb#diff-d9ae4f1159bbe7195bce06dbc06baeb882e3f3d4fcf280c3d31523bf8c0dada7

In a very simple test constructing 1 million HttpClientRequestAdapter objects which can easily happen in the use-case I have results in the following memory usage pattern. Once the test completes we have a large number of objects in gen2 of the GC.

    public static void DotPeekActivitySourceLeak()
    {
        HttpClient httpClient = new HttpClient();
        for (var i = 0; i < 1000000; i++)
        {
            using var adapter = new HttpClientRequestAdapter(new AnonymousAuthenticationProvider(), httpClient: httpClient);
        }
    }

Updating the HttpClientRequestAdapter constructor to register the ActivitySource in the same way as the HttpClient middleware implementations, since it's using the same ActivitySource name eliminates this leak.

activitySource = ActivitySourceRegistry.DefaultInstance.GetOrCreateActivitySource(obsOptions.TracerInstrumentationName);

parameters names decoding handler also decodes values

originally reported in microsoft/kiota-http-python#179
We add those cases as unit tests and correct the implementation so it doesn't impact query parameters values

Configurable defaults for `ObservabilityOptions` in http middleware

The default telemetry configuration results in creating and an ActivitySource and Activity for each request, and then disposing those objects after a response is returned. ActivitySource.Dispose calls SynchronizedList.Remove (https://github.com/dotnet/runtime/blob/main/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/ActivitySource.cs#L299) as shown in the Azure Profiler output where the lock is present (https://github.com/dotnet/runtime/blob/main/src/libraries/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/ActivitySource.cs#L402).

As we are doing this in each of the client middleware irrespective of the whether the telemetry is being listened to (e.g.

kiota-http-dotnet/src/Middleware/CompressionHandler.cs

Line 38 in 7048ed2

activitySource = new ActivitySource(obsOptions.TracerInstrumentationName);

) , the use of the SynchronizedList in the underlying telemetry implementation can be perfomance bottleneck.

A temporary workaround is to add middleware to remove the default ObservabilityOptions created from the request. But the action below should probably be configurable and "opt-in"

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Line 547 in 7048ed2

message.Options.Set(new HttpRequestOptionsKey<IRequestOption>(typeof(ObservabilityOptions).FullName!), obsOptions);

cc @baywet

RetryHandler should throw an APIException with retry error nested in the innerException.

Sourced from microsoftgraph/msgraph-sdk-dotnet#2444

In the event that the API fails in making a request due to a large number of retries, the retryHandler should return an ApiError for a consistent experience and have the AggregateException with the listed errors as the InnerException.

ensure 3XX responses without location header do not throw

follow up to microsoft/kiota#4367
and microsoft/kiota#4190

Implement a unit test with a 304 response, no location header, check the request adapter returns null and doesn't throw.
Most likely this line will need to be adapted.

kiota-http-dotnet/src/HttpClientRequestAdapter.cs

Line 385 in d960c5f

if(response.IsSuccessStatusCode) return;

RetryHandler should add more information on failures

Sourced from microsoftgraph/msgraph-beta-sdk-dotnet#816

The default retry handler simply throws an InvalidOperationException exception when it reaches the maximum retry count. The error message is simply set to: «Too many retries performed.» and the inner exception simply states: «More than {retryCount} retries encountered while sending the request.»
There is no way to know why retries have been performed. What were the error codes and error messages?

The retry handler should return an AggregateException (or a specific exception like TooManyRetriesException) with one exception per retry specifying the HTTP status and the error code returned by the server.

Leverage Framework dependencies rather than explicit where possible

Is your feature request related to a problem? Please describe.
I want to minimise dependencies in my project by utilising framework dependencies wherever possible

Describe the solution you'd like
I want the package to not have an explicit dependency on System.Diagnostics.DiagnosticSource & System.Text.Json on net 6+.

Describe alternatives you've considered
Accept the additional dependency

Additional context
n/a

Kiota HttpAdapter does not support Http/2

Currently Http/2 is used widely, like Microsoft Graph support Http/2 in Sep 2023.
In previously Microsoft Graph core SDK it will use Http/2 by default, but since 5.x which use Kiota to send http request, it only support Http1.1, and can't change it.
Should use http 2.0 by default, and it can fall back to Http 1.1 if server side does not support it.

Reference:
MC447341 – Beginning September 15, 2023, the Microsoft Graph service (graph.microsoft.com) will enable HTTP version 2 (HTTP/2) in addition to HTTP/1.1. for API requests.

microsoftgraph/msgraph-sdk-dotnet-core#357