ankane/notable

Track notable requests and background jobs

Notable

🌟 🌟 🌟

Notable tracks notable requests and background jobs and stores them in your database. What makes a request or job notable? There are a number of default situations, but ultimately you decide what interests you.

By default, Notable tracks:

• errors
• 404s
• slow requests and jobs
• timeouts
• validation failures
• CSRF failures
• unpermitted parameters
• blocked and throttled requests

You can track custom notes as well.

🍊 Battle-tested at Instacart

Installation

Add this line to your application’s Gemfile:

gem "notable"

And run:

rails generate notable:requests
rails generate notable:jobs
rails db:migrate

To explore the data, check out Blazer.

How It Works

A Notable::Request is created for:

• errors
• 404s
• slow requests
• timeouts from Slowpoke
• validation failures
• CSRF failures
• unpermitted parameters
• blocked and throttled requests from Rack Attack

A Notable::Job is created for:

• errors
• slow jobs
• validation failures

Create a custom note inside a request or job with:

Notable.track("Note Type", "Optional extra info")

Customization

Disable tracking in certain environments

Notable.enabled = Rails.env.production?

Requests

Set slow threshold

Notable.slow_request_threshold = 5.seconds

Custom user method

Notable.user_method = lambda do |env|
  env["warden"].try(:user) || env["action_controller.instance"].try(:current_visit)
end

Custom track method

Notable.track_request_method = lambda do |data, env|
  Notable::Request.create!(data)
end

Skip tracking CSRF failures

skip_before_action :track_unverified_request

Anonymize IP addresses

Notable.mask_ips = true

Jobs

Set slow threshold

Notable.slow_job_threshold = 60.seconds

To set a threshold for a specific job, use:

class CustomJob < ApplicationJob
  def notable_slow_job_threshold
    5.minutes
  end
end

Custom track method

Notable.track_job_method = lambda do |data|
  Notable::Job.create!(data)
end

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/ankane/notable.git
cd notable
bundle install
bundle exec rake test

notable's Issues

User not being associated to Notable::Request

Is there a reason that the current_user is not being associated to the Notable::Request? I'm using Devise and the latest version of notable.

Ruby 2.7 - Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

With Ruby 2.7 there is a problem using the belongs_to association:

/Users/laykou/.rvm/gems/ruby-2.7.1/gems/notable-0.3.0/app/models/notable/request.rb:5: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/Users/laykou/.rvm/gems/ruby-2.7.1/gems/activerecord-6.0.3.1/lib/active_record/associations.rb:1657: warning: The called method `belongs_to' is defined here

Consider request belongs_to true_user ?

Would you be up to a PR to add a 'belongs_to :true_user' to notable requests For when we use an impersonating/masquerading strategy like https://github.com/ankane/pretender ?

(Or alternatively, authenticated_user or virtual_user vs current_user, naming up for grabs)

Could not find table 'blazer_connections'

In an attempt to access my admin, I received this error:

"ActiveRecord::StatementInvalid at /
Could not find table 'blazer_connections'"

Here were the codes I used:

gem 'notable'
rails generate notable:requests
rails generate notable:jobs # optional
rake db:migrate

Thanks!

PG::UndefinedTable: ERROR: relation "notable_requests"

When the gem tries to save a notable_request, it creates the error:

PG::UndefinedTable: ERROR: relation "notable_requests" 

NotableJob blank despite using Sidekiq

Do I have to manually turn on NotableJobs as the table is empty?

I'm using Sidekiq Pro, and have tried testing it by creating errors including validation issues.

NoMethodError in NotableWeb::HomeController#index

When visiting "http://localhost:3000/notable/", I get the following error:

undefined method `map' for <ActionController::Parameters {} permitted: true>:ActionController::Parameters Did you mean? tap

The error occurs on:
notable_web (0.0.1) app/controllers/notable_web/home_controller.rb:8:in `index'

relation "requests" does not exist

#6 When we run "rake db:migrate", we get the error:

    PG::UndefinedTable: ERROR:  relation "requests" does not exist
: CREATE TABLE "notable_requests" ("id" serial primary key, "note_type" character varying , "note" text , "user_id" integer , "user_type" character varying , "action" text , "status" integer , "url" text , "request_id" character varying , "ip" character varying , "user_agent" text , "referrer" text , "params" text , "request_time" decimal , "created_at" timestamp , CONSTRAINT fk_notable_requests_user_id FOREIGN KEY ("user_id") REFERENCES "users" ("id"), CONSTRAINT fk_notable_requests_request_id FOREIGN KEY ("request_id") REFERENCES "requests" ("id"))
/Users/andrewkelley/Documents/websites/frontend/db/migrate/20170724194710_create_notable_requests.rb:3:in `change'
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "requests" does not exist
: CREATE TABLE "notable_requests" ("id" serial primary key, "note_type" character varying , "note" text , "user_id" integer , "user_type" character varying , "action" text , "status" integer , "url" text , "request_id" character varying , "ip" character varying , "user_agent" text , "referrer" text , "params" text , "request_time" decimal , "created_at" timestamp , CONSTRAINT fk_notable_requests_user_id FOREIGN KEY ("user_id") REFERENCES "users" ("id"), CONSTRAINT fk_notable_requests_request_id FOREIGN KEY ("request_id") REFERENCES "requests" ("id"))
/Users/andrewkelley/Documents/websites/frontend/db/migrate/20170724194710_create_notable_requests.rb:3:in `change'
PG::UndefinedTable: ERROR:  relation "requests" does not exist
/Users/andrewkelley/Documents/websites/frontend/db/migrate/20170724194710_create_notable_requests.rb:3:in `change'

The issue is being caused by the migration with the line "t.string :request_id"

Mongoid support

I have Rails 4.1.8 with Mongoid 4.0.2. I get this when I rackup:
NoMethodError: undefined method `enabled' for Notable:Module

Does Notable only work with ActiveRecord?

Valid column type for notable_requests.ip column

PostgreSQL is the most popular database choice in the Rails world, as far as I can see you guys also use it extensively.

There are multiple advantages of using the inet column type for IP addresses, those help in the further problems investigations that seem very useful considering what you track with Notable (which is awesome by the way).

You can read some simple examples of those in Using PostgreSQL cidr and inet types, operators, and functions article.

If you decide to go with this, take a look at how it's implemented in the Devise gem.

Cheers, and keep up the good work @ankane, you create one of my favorite open-sourced Ruby gems 👍

[safely] no implicit conversion of nil into String

When records are added to Notable, I get an error in GetSentry that says "[safely] no implicit conversion of nil into String".

Save errors in a background job

Would it be wise to save the errors in a background job?

undefined method `verified_request?'

When using Rails’s config.api_only = true feature, you get the error:

undefined method `verified_request?'

app/controllers/application_controller.rb

class ApplicationController < ActionController::API
    protect_from_forgery with: :null_session
end

Default logging not working

Rails 4.2.4, Ruby 2.2.3, running on my production instance of Ubuntu 14.04 at Digital Ocean.

Latest gem version of notable, no settings specified, mounting notable-web at as shown with devise (checks user#admin? == true)

Nothing.. no logging..

Purposefully hitting 404s, etc.

Notable.enabled == true when asked in the console

Feature: Ability to disable features

In the Gem todo list, was it referring to the website's features if they're using Rollbar or the gem's feature? I'm thinking about how to disable a site's feature (ex: one of the payment options) if it's causing errors due to a third party api.

Switch to ActiveJob instead of individual gem integrations

Consider a request-local and/or model-local 'skip noting model validation errors'

We have some GET requests which return meta data which check if certain actions on the returned resources are valid. Since these produce active model validation errors, we are getting false positives on these requests.

Rather than dig into the notable internals to disable these, I took a heavier hammer and added to our base controller.

  after_action :clear_non_notable_notes, on: [:index, :show]

  def clear_non_notable_notes
    Notable.clear_notes
  end

would you accept a PR to conditionally disable the model validations by either/and model-specific flag, request-specific flag?