Giter Site home page Giter Site logo

basecamp / trashed Goto Github PK

View Code? Open in Web Editor NEW
190.0 20.0 26.0 70 KB

Tell StatsD about request time, GC, objects and more. Latest Rails 4 and Ruby 2.1 support, and ancient Rails 2 and Ruby 1.8 support.

License: MIT License

Ruby 100.00%
ruby statsd rack instrument

trashed's Introduction

Trashed

Keep an eye on resource usage.

  • Sends per-request object counts, heap growth, GC time, and more to StatsD.
  • Sends snapshots of resource usage, e.g. live String objects, to StatsD.
  • Supports new stuff: Rails 5.1 and latest Ruby 2.x features.
  • Supports old stuff: Rails 2/3/4, Ruby 1.9+, REE, Ruby 1.8 with RubyBench patches.

Setup

Rails 5

On Rails 5 (and Rails 3 and 4), add this to the top of config/application.rb:

require 'trashed/railtie'

And in the body of your app config:

module YourApp
  class Application < Rails::Application
    config.trashed.statsd = YourApp.statsd

Rails 2

On Rails 2, add the middleware to config/environment.rb:

Rails::Initializer.run do |config|
  reporter = Trashed::Reporter.new
  reporter.logger = Rails.logger
  reporter.statsd = YourApp.statsd

  config.middleware.use Trashed::Rack, reporter
end

Custom dimensions

You probably want stats per controller, action, right?

Set a #timing_dimensions lambda to return a list of dimensions to qualify per-request measurements like time elapsed, GC time, objects allocated, etc.

For example:

config.trashed.timing_dimensions = ->(env) do
  # Rails 3, 4, and 5, set this. Other Rack endpoints won't have it.
  if controller = env['action_controller.instance']
    name    = controller.controller_name
    action  = controller.action_name
    format  = controller.rendered_format || :none
    variant = controller.request.variant || :none  # Rails 4.1+ only!

    [ :All,
      :"Controllers.#{name}",
      :"Actions.#{name}.#{action}.#{format}+#{variant}" ]
  end
end

Results in metrics like:

YourNamespace.All.Time.wall
YourNamespace.Controllers.SessionsController.Time.wall
YourNamespace.Actions.SessionsController.index.json+phone.Time.wall

Similarly, set a #gauge_dimensions lambda to return a list of dimensions to qualify measurements which gauge current state, like heap slots used or total number of live String objects.

For example:

config.trashed.gauge_dimensions = ->(env) {
  [ :All,
    :"Stage.#{Rails.env}",
    :"Hosts.#{`hostname -s`.chomp}" ]
}

Results in metrics like:

YourNamespace.All.Objects.T_STRING
YourNamespace.Stage.production.Objects.T_STRING
YourNamespace.Hosts.host-001.Objects.T_STRING

Version history

3.2.8 (January 31, 2022)

  • REE: Fix that GC.time is reported in microseconds instead of milliseconds

3.2.7 (November 8, 2017)

  • Ruby 1.8.7 compatibility

3.2.6 (June 21, 2017)

  • Mention Rails 5 support

3.2.5 (Feb 26, 2015)

  • Support Ruby 2.2 GC.stat naming, avoiding 2.1 warnings

3.2.4 (July 25, 2014)

  • Fix compatibility with Rails 3.x tagged logging - @calavera

3.2.3 (June 23, 2014)

  • Report CPU/Idle time in tenths of a percent

3.2.2 (March 31, 2014)

  • Reduce default sampling rates.
  • Stop gauging all GC::Profiler data. Too noisy.
  • Report gauge readings as StatsD timings.
  • Support providing a Statsd::Batch since using Statsd#batch results in underfilled packets at low sample rates.
  • Fix bug with sending arrays of timings to StatsD.
  • Record GC timings in milliseconds.

3.1.0 (March 30, 2014)

  • Report percent CPU/idle time: Time.pct.cpu and Time.pct.idle.
  • Measure out-of-band GC count, time, and stats. Only meaningful for single-threaded servers like Unicorn. But then again so is per-request GC monitoring.
  • Support @tmm1's GC::OOB (https://github.com/tmm1/gctools).
  • Measure time between GCs.
  • Spiff up logger reports with more timings.
  • Support Rails log tags on logged reports.
  • Allow instruments' #start to set timings/gauges.

3.0.1 (March 30, 2014)

  • Sample requests to instrument based on StatsD sample rate.

3.0.0 (March 29, 2014)

  • Support new Ruby 2.0 and 2.1 GC stats.
  • Gauge GC details with GC::Profiler.
  • Performance rework. Faster, fewer allocations.
  • Rework counters and gauges as instruments.
  • Batch StatsD messages to decrease overhead on the server.
  • Drop NewRelic samplers.

2.0.5 (December 15, 2012)

  • Relax outdated statsd-ruby dependency.

2.0.0 (December 1, 2011)

  • Rails 3 support.
  • NewRelic samplers.

1.0.0 (August 24, 2009)

  • Initial release.

trashed's People

Contributors

calavera avatar dependabot[bot] avatar dhh avatar eileencodes avatar jeremy avatar jphenow avatar noahhl avatar tsevan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

trashed's Issues

ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users

Stacktrace (most recent call first):

  org/jruby/RubyArray.java:1613:in `each'
  org/jruby/RubyEnumerable.java:977:in `each_with_index'
  trashed/instruments/ruby_gc_profiler.rb:12:in `measure'
    timings[:"#{captured}.time"] ||= 1000 * GC::Profiler.total_time
  trashed/instruments/ruby_gc_profiler.rb:7:in `start'
    measure state, timings, gauges, :OOBGC
  trashed/meter.rb:31:in `instrument!'
    @timers.each { |i| i.start state, timings, gauges }

Why is everything submitted as a timing?

Hi,

I'm interested in collecting the following stats, but why they are submitted as timings... with a sample rate of 0.05(?) bewilder me greatly:

Rack.Server.All.GC.count:59|ms|@0.05
Rack.Server.All.GC.heap_allocated_pages:1093|ms|@0.05
Rack.Server.All.GC.heap_available_slots:447947|ms|@0.05
Rack.Server.All.GC.heap_live_slots:426958|ms|@0.05
Rack.Server.All.GC.heap_marked_slots:356005|ms|@0.05
Rack.Server.All.GC.heap_sorted_length:1069|ms|@0.05
Rack.Server.All.GC.heap_tomb_pages:0|ms|@0.05
Rack.Server.All.GC.major_gc_count:9|ms|@0.05
Rack.Server.All.GC.malloc_increase_bytes:313296|ms|@0.05
Rack.Server.All.GC.malloc_increase_bytes_limit:16777216|ms|@0.05
Rack.Server.All.GC.minor_gc_count:51|ms|@0.05
Rack.Server.All.GC.old_objects:301503|ms|@0.05
Rack.Server.All.GC.total_allocated_pages:1119|ms|@0.05
Rack.Server.All.GC.total_freed_objects:1429038|ms|@0.05

Most of these things look like counters, with things like heap_marked_slots being a gauge. Is reporting at timings an oversight? bug? just the way you do things at basecamp? Any guidance here would be welcome.

Please check configuration and setup for Rails 2

Used the following setup in my environment.rb per instructions:

Rails::Initializer.run do |config|
  ...
  require 'trashed/reporter'
  require 'statsd'

  statsd_reporter = Trashed::Reporter.new
  statsd_reporter.logger = Rails.logger
  statsd_reporter.statsd = Statsd.new('127.0.0.1')

  config.middleware.use Trashed::Rack, statsd_reporter
end

I can see the rack middleware is loaded using bundle exec rake middleware:

use Trashed::Rack, #<Trashed::Reporter:0x1102cc660 @timing_dimensions=#<Proc:0x0000000110511268@/Users/marcusyoung/Develop/registria/vendor/cache/trashed-3133b6603148/lib/trashed/reporter.rb:16>, @gauge_sample_rate=0.05, @timing_sample_rate=0.1, @gauge_dimensions=#<Proc:0x00000001105110b0@/Users/marcusyoung/Develop/registria/vendor/cache/trashed-3133b6603148/lib/trashed/reporter.rb:17>, @statsd=#<Statsd:0x1102baf00 @batch_size=10, @prefix=nil, @postfix=nil, @port=8125, @host="127.0.0.1">, @logger=nil>

But I'm not getting any stats generated and verified statsd_reporter.statsd above can send information to my local Statsd setup using Pry. Is there something missing in my environment?

Dependency on statsd-ruby breaks installation

Thank you for open sourcing trashed. Although I'm not using it anymore I thought I would point out the following in case I'm not the only to experience this issue.

It looks like the new dependency on a 37signals specific version of statsd-ruby has broken installation of trashed as version 0.3.1.signals.1 of statsd-ruby has not been pushed to rubygems.

The dependency was added in bb28604.

Ruby 1.9 support

I happened to try to use trashed with Ruby 1.9.3.

However when I try to start console with my Rails application I see the following error:

gems/trashed-3.2.7/lib/trashed/instruments/ruby_gc.rb:15:in `stat': non-hash given (TypeError)
gems/trashed-3.2.7/lib/trashed/instruments/ruby_gc.rb:15:in `<class:RubyGC>'
gems/trashed-3.2.7/lib/trashed/instruments/ruby_gc.rb:3:in `<module:Instruments>'
gems/trashed-3.2.7/lib/trashed/instruments/ruby_gc.rb:2:in `<module:Trashed>'
gems/trashed-3.2.7/lib/trashed/instruments/ruby_gc.rb:1:in `<top (required)>'

The reason is that GC.stat output was very basic in Ruby 1.9: https://ruby-doc.org/core-1.9.3/GC.html#method-c-stat.

Example:

irb(main):001:0> GC.stat
=> {:count=>19, :heap_used=>138, :heap_length=>138, :heap_increment=>0, :heap_live_num=>25491, :heap_free_num=>30907, :heap_final_num=>18}
irb(main):002:0> GC.stat :total_allocated_objects
TypeError: non-hash given
	from (irb):2:in `stat'
	from (irb):2

However it was extended to the modern state in Ruby 2.x: https://ruby-doc.org/core-2.1.0/GC.html#method-c-stat.

Could it be fixed?

Rounding errors or smth

Is that okay or not?

9 Jun 19:38:53 - DEBUG: Bad line: -0.046373953111469746,ms,@0.1 in msg "Rack.Request.Controllers.countries.Time.idle:-0.046373953111469746|ms|@0.1"
9 Jun 19:38:56 - DEBUG: Bad line: -0.09394823436741717,ms,@0.1 in msg "Rack.Request.Actions.countries.index.text/html+[].Time.idle:-0.09394823436741717|ms|@0.1"
9 Jun 19:40:35 - DEBUG: Bad line: -0.0016081093635875732,ms,@0.1 in msg "Rack.Request.Actions.random.index.none+[].Time.idle:-0.0016081093635875732|ms|@0.1"
9 Jun 19:40:35 - DEBUG: Bad line: -0.051120046875439584,ms,@0.1 in msg "Rack.Request.All.Time.idle:-0.051120046875439584|ms|@0.1"
9 Jun 19:41:17 - DEBUG: Bad line: -0.055048562498996034,ms,@0.1 in msg "Rack.Request.Controllers.frags.Time.idle:-0.055048562498996034|ms|@0.1"
9 Jun 19:41:17 - DEBUG: Bad line: -0.055048562498996034,ms,@0.1 in msg "Rack.Request.Actions.frags.index.text/html+[].Time.idle:-0.055048562498996034|ms|@0.1"
9 Jun 19:41:23 - DEBUG: Bad line: -0.8728879189081986,ms,@0.1 in msg "Rack.Request.Actions.countries.index.text/html+[].Time.pct.idle:-0.8728879189081986|ms|@0.1"
9 Jun 19:41:24 - DEBUG: Bad line: -0.06603962500000193,ms,@0.1 in msg "Rack.Request.Controllers.frags.Time.idle:-0.06603962500000193|ms|@0.1"
9 Jun 19:42:34 - DEBUG: Bad line: -0.39619290358255427,ms,@0.1 in msg "Rack.Request.All.Time.pct.idle:-0.39619290358255427|ms|@0.1"
9 Jun 19:43:00 - DEBUG: Bad line: -0.014542890625307336,ms,@0.1 in msg "Rack.Request.Controllers.random.Time.idle:-0.014542890625307336|ms|@0.1"
9 Jun 19:43:00 - DEBUG: Bad line: -0.7258155233490782,ms,@0.1 in msg "Rack.Request.All.Time.pct.idle:-0.7258155233490782|ms|@0.1"
9 Jun 19:43:39 - DEBUG: Bad line: -2.0303885993533575,ms,@0.1 in msg "Rack.Request.All.Time.pct.idle:-2.0303885993533575|ms|@0.1"
9 Jun 19:44:14 - DEBUG: Bad line: -0.014303578122053295,ms,@0.1 in msg "Rack.Request.Controllers.random.Time.idle:-0.014303578122053295|ms|@0.1"
9 Jun 19:45:51 - DEBUG: Bad line: -0.9865908093262699,ms,@0.1 in msg "Rack.Request.All.Time.pct.idle:-0.9865908093262699|ms|@0.1"
9 Jun 19:45:51 - DEBUG: Bad line: -0.0034391406225040555,ms,@0.1 in msg "Rack.Request.Controllers.random.Time.idle:-0.0034391406225040555|ms|@0.1"
9 Jun 19:45:51 - DEBUG: Bad line: -0.0034391406225040555,ms,@0.1 in msg "Rack.Request.Actions.random.index.none+[].Time.idle:-0.0034391406225040555|ms|@0.1"

p.s. Where frags and random are controller names

Error using with dogstatsd-ruby

I had this naive idea that DataDog dogstatsd-ruby will be compatible with this gem, but I'm suppose not. Looking into this myself, but in case somebody else will find answer before me

app error: undefined method `[]' for 0.1:float (nomethoderror)
/dogstatsd-ruby-3.0.0/lib/datadog/statsd.rb:390:in `send_stats'
/dogstatsd-ruby-3.0.0/lib/datadog/statsd.rb:198:in `timing'
/trashed-3.2.6/lib/trashed/reporter.rb:91:in `block (2 levels) in send_to_statsd'
/trashed-3.2.6/lib/trashed/reporter.rb:90:in `each'
/trashed-3.2.6/lib/trashed/reporter.rb:90:in `block in send_to_statsd'
/trashed-3.2.6/lib/trashed/reporter.rb:83:in `each'
/trashed-3.2.6/lib/trashed/reporter.rb:83:in `send_to_statsd'
/trashed-3.2.6/lib/trashed/reporter.rb:77:in `block in report_statsd'
/dogstatsd-ruby-3.0.0/lib/datadog/statsd.rb:307:in `batch'
/trashed-3.2.6/lib/trashed/reporter.rb:76:in `report_statsd'
/trashed-3.2.6/lib/trashed/reporter.rb:22:in `report'
/trashed-3.2.6/lib/trashed/rack.rb:18:in `block in call'
/trashed-3.2.6/lib/trashed/rack.rb:18:in `tap'
/trashed-3.2.6/lib/trashed/rack.rb:18:in `call'
/rack-contrib-1.5.0/lib/rack/contrib/runtime.rb:18:in `call'
/rack-contrib-1.5.0/lib/rack/contrib/sendfile.rb:99:in `call'
/railties-4.2.8/lib/rails/engine.rb:518:in `call'
/railties-4.2.8/lib/rails/application.rb:165:in `call'
/railties-4.2.8/lib/rails/railtie.rb:194:in `public_send'
/railties-4.2.8/lib/rails/railtie.rb:194:in `method_missing'
/unicorn-5.2.0/lib/unicorn/http_server.rb:562:in `process_client'
/unicorn-5.2.0/lib/unicorn/http_server.rb:658:in `worker_loop'
/unicorn-5.2.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
/unicorn-5.2.0/lib/unicorn/http_server.rb:132:in `start'
/unicorn-5.2.0/bin/unicorn:126:in `<top (required)>'
/bin/unicorn:22:in `load'
/bin/unicorn:22:in `<main>'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.