Giter Site home page Giter Site logo

bleacherreport / plug_logger_json Goto Github PK

View Code? Open in Web Editor NEW
132.0 15.0 33.0 89 KB

Elixir Plug that formats http request logs as json

Home Page: https://dev.bleacherreport.com/elixir-phoenix-centralized-http-logging-aa50efe3105b#.k00ri7og7

License: Apache License 2.0

Elixir 100.00%
elixir plug json logging logger

plug_logger_json's Introduction

PlugLoggerJson

Hex pm Build Status License

A comprehensive JSON logger Plug.

Dependencies

  • Plug
  • Poison

Elixir & Erlang Support

The support policy is to support the last 2 major versions of Erlang and the three last minor versions of Elixir.

Installation

  1. add plug_logger_json to your list of dependencies in mix.exs:

    def deps do
      [{:plug_logger_json, "~> 0.7.0"}]
    end
  2. ensure plug_logger_json is started before your application (Skip if using Elixir 1.4 or greater):

    def application do
      [applications: [:plug_logger_json]]
    end
  3. Replace Plug.Logger with either:

    • Plug.LoggerJSON, log: Logger.level,
    • Plug.LoggerJSON, log: Logger.level, extra_attributes_fn: &MyPlug.extra_attributes/1 in your plug pipeline (in endpoint.ex for Phoenix apps),

Recommended Setup

Configure plug_logger_json

Add to your config/config.exs or config/env_name.exs if you want to filter params or headers or suppress any logged keys:

config :plug_logger_json,
  filtered_keys: ["password", "authorization"],
  suppressed_keys: ["api_version", "log_type"]

Configure the logger (console)

In your config/config.exs or config/env_name.exs:

config :logger, :console,
  format: "$message\n",
  level: :info, # You may want to make this an env variable to change verbosity of the logs
  metadata: [:request_id]

Configure the logger (file)

Do the following:

  • update deps in mix.exs with the following:

    def deps do
     [{:logger_file_backend, "~> 0.0.10"}]
    end
  • add to your config/config.exs or config/env_name.exs:

    config :logger,
      format: "$message\n",
      backends: [{LoggerFileBackend, :log_file}, :console]
    
    config :logger, :log_file,
      format: "$message\n",
      level: :info,
      metadata: [:request_id],
      path: "log/my_pipeline.log"
  • ensure you are using Plug.Parsers (Phoenix adds this to endpoint.ex by default) to parse params as well as request body:

    plug Plug.Parsers,
      parsers: [:urlencoded, :multipart, :json],
      pass: ["*/*"],
      json_decoder: Poison

Error Logging

In router.ex of your Phoenix project or in your plug pipeline:

  • add require Logger,

  • add use Plug.ErrorHandler,

  • add the following two private functions:

    defp handle_errors(%Plug.Conn{status: 500} = conn, %{kind: kind, reason: reason, stack: stacktrace}) do
      Plug.LoggerJSON.log_error(kind, reason, stacktrace)
      send_resp(conn, 500, Poison.encode!(%{errors: %{detail: "Internal server error"}}))
    end
    
    defp handle_errors(_, _), do: nil

Extra Attributes

Additional data can be logged alongside the request by specifying a function to call which returns a map:

def extra_attributes(conn) do
  map = %{
    "user_id" => get_in(conn.assigns, [:user, :user_id]),
    "other_id" => get_in(conn.private, [:private_resource, :id]),
    "should_not_appear" => conn.private[:does_not_exist]
  }

  map
  |> Enum.filter(&(&1 !== nil))
  |> Enum.into(%{})
end

plug Plug.LoggerJSON,
  log: Logger.level(),
  extra_attributes_fn: &MyPlug.extra_attributes/1

In this example, the :user_id is retrieved from conn.assigns.user.user_id and added to the log if it exists. In the example, any values that are nil are filtered from the map. It is a requirement that the value is serialiazable as JSON by the Poison library, otherwise an error will be raised when attempting to encode the value.

Log Verbosity

LoggerJSON plug supports two levels of logging:

  • info / error will log:

    • api_version,
    • date_time,
    • duration,
    • log_type,
    • method,
    • path,
    • request_id,
    • status
  • warn / debug will log everything from info and:

    • client_ip,
    • client_version,
    • params / request_body.

The above are default. It is possible to override them by setting a include_debug_logging option to:

  • false โ€“ means the extra debug fields (client_ip, client_version, and params) WILL NOT get logged.
  • true โ€“ means the extra fields WILL get logged.
  • Not setting this option will keep the defaults above.

Example:

plug Plug.LoggerJSON,
  log: Logger.level,
  include_debug_logging: true

Contributing

Before submitting your pull request, please run:

  • mix credo --strict,
  • mix coveralls,
  • mix dialyzer,
  • update changelog.

Please squash your pull request's commits into a single commit with a message and detailed description explaining the commit.

plug_logger_json's People

Contributors

bgmarx avatar crowdhailer avatar davich avatar gazler avatar gmile avatar johnkelly avatar scrogson avatar shanethehat avatar slashdotdash avatar trumant avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

plug_logger_json's Issues

Support different log levels

Support different log levels. Make a debug level that is verbose (logs headers & params) and trim the info level a bit further (remove params).

Provide config for json library

Phoenix now uses Jason by default. Would be great if there was a config and one could pass in Phoenix.json_library().

I'd be happy to make a PR.

RoadMap 0.2

  • Support different log levels. Make a debug level that is verbose (logs headers & params) and trim the info level a bit further (remove params).
  • Log exceptions in JSON tagged with the request id (see support_errors branch for first attempt)

For those thinking of using 0.1, plug_logger_json is used in very high traffic services on production without issue and sends those logs to an ELK stack.

Future Improvements:

  • Find a way to make the fields logged more customizable.

Filtering nested fields

I'm using the JSONAPI spec with phoenix and plug. I've really enjoyed using this plugin. The one downside is that with JSONAPI, nested fields are pretty common and I couldn't get this logger to filter on nested fields. Eg. {"data": {"attributes": {"password": "thing"}}}

I did get nested filtering working on a fork here, though it broke Dializer. This actually surprised me a bit as the spec specifies a tuple with two strings, I was pretty sure maps were being passed in in the tests. Anyway, I thought I'd check to see if you'd be interested in merging the changes before I spent time fixing Dializer.

Let me know, and thanks for sharing this sweet library!

Remove duplicate error logging

I haven't been able to figure out how to shut off the default non JSON error logging, so that the error is only logged in the JSON format instead of the default + JSON formats.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.