Giter Site home page Giter Site logo

serilog-contrib / serilog-enrichers-clientinfo Goto Github PK

View Code? Open in Web Editor NEW
82.0 4.0 19.0 95 KB

Enrich logs with client IP, correlation id and HTTP request headers.

License: MIT License

C# 100.00%
serilog serilog-enrichers asp-net-core asp-net-mvc enrich-logs enrichers

serilog-enrichers-clientinfo's Introduction

serilog-enrichers-clientinfo NuGet

Enrich logs with client IP, Correlation Id and HTTP request headers.

Install the Serilog.Enrichers.ClientInfo NuGet package

Install-Package Serilog.Enrichers.ClientInfo

or

dotnet add package Serilog.Enrichers.ClientInfo

Apply the enricher to your LoggerConfiguration in code:

Log.Logger = new LoggerConfiguration()
    .Enrich.WithClientIp()
    .Enrich.WithCorrelationId()
    .Enrich.WithRequestHeader("Header-Name1")
    // ...other configuration...
    .CreateLogger();

or in appsettings.json file:

{
  "Serilog": {
    "MinimumLevel": "Debug",
    "Using":  [ "Serilog.Enrichers.ClientInfo" ],
    "Enrich": [
      "WithClientIp",
      "WithCorrelationId",
      {
          "Name": "WithRequestHeader",
          "Args": { "headerName": "User-Agent"}
      }
    ],
    "WriteTo": [
      { "Name": "Console" }
    ]
  }
}

ClientIp

For ClientIp enricher you can configure the x-forwarded-for header if the proxy server uses a different header to forward the IP address.

Log.Logger = new LoggerConfiguration()
    .Enrich.WithClientIp(headerName: "CF-Connecting-IP")
    ...

or

{
  "Serilog": {
    "MinimumLevel": "Debug",
    "Using":  [ "Serilog.Enrichers.ClientInfo" ],
    "Enrich": [
      {
        "Name": "WithClientIp",
        "Args": {
          "headerName": "CF-Connecting-IP"
        }
      }
    ],
  }
}

CorrelationId

For CorrelationId enricher you can:

  • Configure the header name and default header name is x-correlation-id
  • Set value for correlation id when the header is not available in request header collection and the default value is false
Log.Logger = new LoggerConfiguration()
    .Enrich.WithCorrelationId(headerName: "correlation-id", addValueIfHeaderAbsence: true)
    ...

or

{
  "Serilog": {
    "MinimumLevel": "Debug",
    "Using":  [ "Serilog.Enrichers.ClientInfo" ],
    "Enrich": [
      {
        "Name": "WithCorrelationId",
        "Args": {
          "headerName": "correlation-id"
          "addValueIfHeaderAbsence": true
        }
      }
    ],
  }
}

RequestHeader

You can use multiple WithRequestHeader to log different request headers. WithRequestHeader accepts two parameters; The first parameter headerName is the header name to log and the second parameter is propertyName which is the log property name.

Log.Logger = new LoggerConfiguration()
    .Enrich.WithRequestHeader(headerName: "header-name-1")
    .Enrich.WithRequestHeader(headerName: "header-name-2", propertyName: "SomeHeaderName")
    ...

or

{
  "Serilog": {
    "MinimumLevel": "Debug",
    "Using":  [ "Serilog.Enrichers.ClientInfo" ],
    "Enrich": [
      {
        "Name": "WithRequestHeader",
        "Args": {
          "headerName": "User-Agent"
        }
      },
      {
        "Name": "WithRequestHeader",
        "Args": {
          "headerName": "Connection"
        }
      },
      {
        "Name": "WithRequestHeader",
        "Args": {
          "headerName": "Content-Length",
          "propertyName": "RequestLength"
        }
      }
    ],
  }
}

Note

To include logged headers in OutputTemplate, the header name without - should be used if you haven't set the log property name. For example, if the header name is User-Agent, you should use UserAgent.

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .Enrich.WithRequestHeader("User-Agent")
    .WriteTo.Console(outputTemplate: "[{Timestamp:HH:mm:ss}] {Level:u3} {UserAgent} {Message:lj}{NewLine}{Exception}")

Installing into an ASP.NET Core Web Application

You need to register the IHttpContextAccessor singleton so the enrichers have access to the requests HttpContext to extract client IP and client agent. This is what your Startup class should contain in order for this enricher to work as expected:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Serilog;

namespace MyWebApp
{
    public class Startup
    {
        public Startup()
        {
            Log.Logger = new LoggerConfiguration()
                .MinimumLevel.Debug()
                .WriteTo.Console(outputTemplate: "[{Timestamp:HH:mm:ss}] {Level:u3} CLient IP: {ClientIp} Correlation Id: {CorrelationId} header-name: {headername} {Message:lj}{NewLine}{Exception}")
                .Enrich.WithClientIp()
                .Enrich.WithCorrelationId()
                .Enrich.WithRequestHeader("header-name")
                .Enrich.WithRequestHeader("another-header-name", "SomePropertyName")
                .CreateLogger();
        }

        // This method gets called by the runtime. Use this method to add services to the container.
        // For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940
        public void ConfigureServices(IServiceCollection services)
        {
            // ...
            services.AddHttpContextAccessor();
            // ...
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            // ...
            loggerFactory.AddSerilog();
            // ...
        }
    }
}

serilog-enrichers-clientinfo's People

Contributors

atesoglu avatar augustoproiete avatar crtaylor243 avatar dependabot[bot] avatar edmacdonald avatar kahbazi avatar klinki avatar mesmailpour-spectra avatar mo-esmp avatar pergardebrink avatar shoboske avatar spottedmahn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

serilog-enrichers-clientinfo's Issues

ClientIp enricher should have a way to not read X-Forwarded-For

Instead of directly reading the X-Forwarded-Header by default, it's much better to use the ForwardedHeaders middleware (https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer?view=aspnetcore-7.0) and then let that middleware set the HttpContext.Connection.RemoteIpAddress so we only log that information instead.

If, for some reason someone don't want to use that, then the ClientIp enricher could read this by header. Since there's a danger of IP spoofing (which this enricher does not have any protection against), it's better to let the ForwardedHeader middleware and configuration take care of this (the KnownNetworks and KnownProxies configuration).

I can implement a suggestion for this (by making it possible to set a configuration to disable reading headers or similar), but I'd personally prefer if the default would be changed to not read X-Forwarded-For (which would be a breaking change).
Maybe something for a 3.0 release?

Client Origin and Referer

I needed those two properties in enriched.

Do you mind if I make PR to your library with those 2?

Serilog deployed w/ full framework package

When I install the NuGet package on a full framework project it is including Serilog.dll (v2.4.0.0)

image

Obviously, this enricher shouldn't be deploying Serilog, just taking a dependency on it as it causes problems like this: MissingMethodException Serilog.Context.LogContext.Push


Seems to be related: Including assembly files

<files>
  <file src="src\Serilog.Enrichers.ClientInfo\bin\$target$\net452\*.dll" target="lib/net452" />
  <file src="src\Serilog.Enrichers.ClientInfo\bin\$target$\netstandard2.0\*.dll" target="lib/netstandard2.0" />
  <file src="src\Serilog.Enrichers.ClientInfo\bin\$target$\netstandard2.1\*.dll" target="lib/netstandard2.1" />
</files>  

If you follow the conventions described in Creating a Package, you do not have to explicitly specify a list of files in the .nuspec file. The nuget pack command automatically picks up the necessary files.

Was there a reason not to use the automatic behavior?


I see it puts Serilog.dll in the bin directory on net452:

image

I don't see it in the netstandard folders. I guess msbuild behaves differently? ๐Ÿค”

X-forwarded-for and CF-Connecting-IP

Sorry in advance if I wrote something wrong but... is it possible to use both X-forwarded-for and CF-Connecting-IP or others?
something like

"Args": {
          "xForwardHeaderName": [ "CF-Connecting-IP", "X-Forwarded-For", "X-Real-IP" ]
        }

ClientAgent replacement not working

Hi this is my serilog configuration in app.settings.json


{
  "Serilog": {
    "Using": [
      "Serilog.Sinks.Console",
      "Serilog.Sinks.File",
      "Serilog.Sinks.Seq",
      "Serilog.Enrichers.ClientInfo",
      "Serilog.Enrichers.CorrelationId",
      "Serilog.Enrichers.ExceptionStackTraceHash",
      "Serilog.Enrichers.Environment",
      "Serilog.Enrichers.Memory"
    ],
    "MinimumLevel": {
      "Default": "Verbose",
      "Override": {
        "Microsoft.AspNetCore": "Warning",
        "Microsoft": "Information",
        "Microsoft.EntityFrameworkCore.Database.Command": "Error",
        "Microsoft.AspNetCore.SignalR": "Information",
        "Microsoft.AspNetCore.Http.Connections": "Information"
      }
    },
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [correlationId: {CorrelationId} machineName:{MachineName} environmentName:{EnvironmentName} environmentUserName:{EnvironmentUserName} clientIp:{ClientIp} requestHeader:{RequestHeader} processId:{ProcessId} processName:{ProcessName} memory:{MemoryUsage} level:{Level: u3}] ({SourceContext}) {Message} {NewLine} {Exception} {NewLine}",
          "theme": "Serilog.Sinks.SystemConsole.Themes.Literate",
          "restrictedToMinimumLevel": "Information"
        }
      },
      {
        "Name": "Seq",
        "Args": {
          "serverUrl": "http://localhost:5341/",
          "apiKey": "none",
          "restrictedToMinimumLevel": "Information"
        }
      },
      {
        "Name": "File",
        "Args": {
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [correlationId: {CorrelationId} machineName:{MachineName} environmentName:{EnvironmentName} environmentUserName:{EnvironmentUserName} clientIp:{ClientIp} requestHeader:{RequestHeader} processId:{ProcessId} processName:{ProcessName} memory:{MemoryUsage} level:{Level: u3}] ({SourceContext}) {Message} {NewLine} {Exception} {NewLine}",
          "path": "C:\\Users\\Lenovo\\AppData\\Local\\Logs\\Cloud.log",
          "rollingInterval": "Day",
          "restrictedToMinimumLevel": "Verbose"
        }
      }
    ],
    "Enrich": [
      "FromLogContext",
      "WithMachineName",
      {
        "Name": "WithClientIp",
        "Args": {
          "headerName": "CF-Connecting-IP"
        }
      },
      {
        "Name": "WithRequestHeader",
        "Args": {
          "headerName": "User-Agent"
        }
      },
      "WithMessageTemplate",
      "WithEnvironmentUserName",
      "WithMemoryUsage",
      "WithEnvironmentName",
      "WithExceptionDetails",
      "ExceptionStackTraceHash",
      "WithExceptionProperties",
      "WithProcessName",
      "WithProcessId"
    ]
  }
}

I have updated the client agent to request header according to the enricher package change. But now, im not able to display the client agent details in log.

image
image

Please help me to fix the issue

Wrong Serilog version in csproj?

According to the nuspec, the Serilog dependency is 2.7.1:

<group targetFramework="netstandard2.0">
	<dependency id="Microsoft.AspNetCore.Http" version="2.1.1" />
	<dependency id="Serilog" version="2.7.1" />
</group>

but the csproj is referencing 2.9.0:

<ItemGroup Condition=" '$(TargetFramework)' == 'netstandard2.0'">
	<PackageReference Include="Microsoft.AspNetCore.Http" Version="2.1.1" />
	<PackageReference Include="Serilog" Version="2.9.0" />
</ItemGroup>

ASP.NET Core enriches wrong IP when proxy is used

Hello,
first of all, thank you for great enricher :)

I have a small problem with it - it doesn't work correctly for ASP.NET Core when proxy is used.
It works fine for full ASP.NET framework. It is caused by handling those cases differently with #if NETFULL macro.

Is there any reason for it? Couldn't also .NET Core version read X-Forwarded-For headers?

(Also, when used on Azure, azure adds also port number, but that's whole different issue)

#if NETFULL
        private string GetIpAddress()
        {
            var ipAddress = _contextAccessor.HttpContext.Request.ServerVariables["HTTP_X_FORWARDED_FOR"];

            if (!string.IsNullOrEmpty(ipAddress))
            {
                var addresses = ipAddress.Split(',');
                if (addresses.Length != 0)
                    return addresses[0];
            }

            return _contextAccessor.HttpContext.Request.ServerVariables["REMOTE_ADDR"];
        }

#else
     private string GetIpAddress()
     {
        return _contextAccessor.HttpContext.Connection.RemoteIpAddress.ToString();
     }
#endif

XML Documentation not included in nuget package.

I see XML documentation in the source here, but it doesn't show up in the published nuget packages.

Should be as simple as adding this to the csproj file:

<GenerateDocumentationFile>True</GenerateDocumentationFile>

Passing correlation id to HttpClient

Is there any suggestion about how can I get and pass the current correlation id through HttpClient's from HttpClientFactory?
My objective is to propagate the current value to subsequent microservices.

CorrelationId not working with .NET 8

dotnet new webapi -o ApiTest -f net8.0
using Serilog;
using Serilog.Events;
using Serilog.Exceptions;
using Serilog.Formatting.Json;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

// SERILOG ================================================================================================

builder.Services.AddSerilog((services, logger) => logger
    .MinimumLevel.Information()
    .MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
    .Enrich.FromLogContext()
    .Enrich.WithThreadId()
    .Enrich.WithMachineName()
    .Enrich.WithEnvironmentUserName()
    .Enrich.WithEnvironmentName()
    .Enrich.WithMachineName()
    .Enrich.WithProcessId()
    .Enrich.WithProcessName()
    .Enrich.WithExceptionDetails()
    .Enrich.WithClientIp()
    .Enrich.WithCorrelationId(addValueIfHeaderAbsence: true)
    .WriteTo.File(formatter: new JsonFormatter(), "log.json")
    .WriteTo.Console()
    .WriteTo.OpenTelemetry(options =>
    {
        options.Endpoint = "http://host.docker.internal:4317";
        options.ResourceAttributes = new Dictionary<string, object>
        {
            ["service.name"] = "basket-api"
        };
    })
);

// ========================================================================================================

var app = builder.Build();

// SERILOG ================================================================================================
app.UseSerilogRequestLogging();
// ========================================================================================================

if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.MapGet("/", () => "TEST API!");

app.Run();

No correlationId on json:

{
    "Timestamp": "2024-05-13T04:58:01.2798131-03:00",
    "Level": "Information",
    "MessageTemplate": "HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms",
    "TraceId": "b16858179222d2ad68f368015c24c708",
    "SpanId": "dd04f5cda4b0ad0f",
    "Properties": {
        "RequestMethod": "GET",
        "RequestPath": "/",
        "StatusCode": 200,
        "Elapsed": 13.5145,
        "SourceContext": "Serilog.AspNetCore.RequestLoggingMiddleware",
        "RequestId": "0HN3J49KKL45L:00000001",
        "ConnectionId": "0HN3J49KKL45L",
        "ThreadId": 15,
        "MachineName": "DESKTOP",
        "EnvironmentUserName": "DESKTOP\\tfsan",
        "EnvironmentName": "Development",
        "ProcessId": 21532,
        "ProcessName": "ApiTest"

    },
    "Renderings": {
        "Elapsed": [
            {
                "Format": "0.0000",
                "Rendering": "13.5145"
            }
        ]
    }
}

ASP.NET Core 2.1 Support?

On startup of our 2.1 app we're getting:

FileLoadException: Could not load file or assembly 'Microsoft.AspNetCore.Http, Version=2.2.2.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The located assembly's manifest definition does not match the assembly reference.

loggerConfiguration..Enrich.WithClientIp()

PS, thanks for creating this enricher! ๐Ÿค

2.0.3 no longer working with Blazor WASM

As of 2.0.3, building a Blazor WASM app with Serilog.Enrichers.ClientInfo referenced results in the following build error:

error NETSDK1082: There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'browser-wasm'.

Downgrading the package to 2.0.1 works correctly.

I've tested the build process on Linux, Windows, and in docker, with the same results.

I'm wondering if it has anything to do with the <Project Sdk="Microsoft.NET.Sdk.BlazorWebAssembly"> required in WASM apps, vs <Project Sdk="Microsoft.NET.Sdk.Web"> in your sample app.

Enrichers are not thread-safe

The enrichers provided by ClientInfo are not thread-safe. This is a problem when concurrent tasks are trying to log inside the scope of a request. For example in the following situation:

var httpClient = _httpClientFactory.CreateClient(HttpClientName);

var task1 = Task.Run(async () =>
{
    var response = await httpClient.GetAsync("/api/call1");
    return await response.Content.ReadAsStringAsync();
});

var task2 = Task.Run(async () =>
{
    var response = await httpClient.GetAsync($"/api/call2");
    return await response.Content.ReadAsStringAsync();
});

await Task.WhenAll(task1, task2);

Because the HttpClients are logging they both use a logger that is trying to enrich the event with some client info (e.g. using ClientIpEnricher). If the property (e.g. Serilog_ClientIp) is not yet available on the HttpContext.Items then both instances will try to write it concurrently, which may cause various exceptions to be thrown depending on when and how the conflicting access occurs.

In theory this problem should only occur if the property isn't already on the HttpContext.Items during the concurrent phase. In other words, if the first logs of the request are written in a concurrent situation then it might occur. This means a workaround fix would be to write a log message before going concurrent (e.g. with _logger.Information("workaround fix") before the tasks in the snippet above). In my own tests this indeed seems to work.

I think there is an (implicit) expectation that Serilog enrichers are thread-safe (as mentioned here: serilog/serilog#1144) so that's why I report this as a bug.

Exception thrown in xUnit test

First off, I am testing with Visual Studio 2022, this doesn't happen with Visual Studio 2019.

However, when running a xUnit test, clientinfo generates an exception that ClientIp and ClientAgent are null. I have tried to add them as headers but it did not make any difference. My testing code:

    public class UnitTest1
    {
        [Fact]
        public async Task Test1()
        {
            // Arrange
            var factory = new CustomWebApplicationFactory<EcommerceWebAPI.Startup>();
            var client = factory.CreateClient();

            // Act
            var request = new HttpRequestMessage(HttpMethod.Post, "/api/Users/AuthenticateUser")
            {
                Content = new StringContent("{\"Username\":\"[email protected]\",\"Password\":\"wrongPassword\"}",
                    Encoding.UTF8,
                    "application/json")
            };
            request.Headers.Add("X-Real-IP", "84.247.85.224");
            request.Headers.Add("ClientIp", "127.0.0.1");
            request.Headers.Add("ClientAgent",
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0");

            var response = await client.SendAsync(request);
            var responseStatusCode = (int)response.StatusCode;
            var apiResponse = await response.Content.ReadAsStringAsync();
            dynamic thing = JObject.Parse(apiResponse);

            // Assert
            Assert.Equal(400, responseStatusCode);
            Assert.Equal("Account is Suspended", (string)thing.message);
        }
    }

What I did to the EcommerceWebAPI appsettings.json file get around this issue:

"Serilog": {
    "Using": [ "Serilog.Exceptions", "Serilog", "Serilog.Sinks.Seq" ],
    "MinimumLevel": {
      "Default": "Verbose",
      "Override": {
        "System": "Information",
        "Microsoft": "Information",
        "Microsoft.EntityFrameworkCore": "Debug",
        "Microsoft.AspNetCore": "Debug"
      }
    },
    "WriteTo": [
      {
        "Name": "Seq",
        "Args": {
          "serverUrl": "http://localhost:5341",
          "apiKey": "HAQif5iJVjIPDlM60yTL",
          "restrictedToMinimumLevel": "Verbose"
        }
      }
    ],
    //"Enrich": [ "WithClientIp", "WithClientAgent", "WithEnvironmentName", "WithMachineName", "WithEnvironmentUserName", "FromLogContext", "WithExceptionDetails" ],
    "Enrich": [ "WithEnvironmentName", "WithMachineName", "WithEnvironmentUserName", "FromLogContext", "WithExceptionDetails" ]
  }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.