test-prof/test-prof

Ruby Tests Profiling Toolbox

Ruby Tests Profiling Toolbox

TestProf is a collection of different tools to analyze your test suite performance.

Why does test suite performance matter? First of all, testing is a part of a developer's feedback loop (see @searls talk) and, secondly, it is a part of a deployment cycle.

Simply speaking, slow tests waste your time making you less productive.

TestProf toolbox aims to help you identify bottlenecks in your test suite. It contains:

• Plug'n'Play integrations for general Ruby profilers (ruby-prof, stackprof)

• Factories usage analyzers and profilers

• ActiveSupport-backed profilers

• RSpec and minitest helpers to write faster tests

• RuboCop cops

• etc.

📑 Documentation

Who uses TestProf

• Discourse reduced ~27% of their test suite time
• Gitlab reduced 39% of their API tests time and improved factories usage
• CodeTriage
• Dev.to
• Open Project
• ...and others

Resources

• TestProf: a good doctor for slow Ruby tests

• TestProf II: Factory therapy for your Ruby tests

• Paris.rb, 2018, "99 Problems of Slow Tests" talk [video, slides]

• BalkanRuby, 2018, "Take your slow tests to the doctor" talk [video], slides]

• RailsClub, Moscow, 2017, "Faster Tests" talk [video (RU), slides]

• RubyConfBy, 2017, "Run Test Run" talk [video, slides]

• Tips to improve speed of your test suite by Benoit Tigeot

Installation

Add test-prof gem to your application:

group :test do
  gem "test-prof", "~> 1.0"
end

And that's it)

Supported Ruby versions:

• Ruby (MRI) >= 2.5.0 (NOTE: for Ruby 2.2 use TestProf < 0.7.0, Ruby 2.3 use TestProf ~> 0.7.0, Ruby 2.4 use TestProf <0.12.0)

• JRuby >= 9.1.0.0 (NOTE: refinements-dependent features might require 9.2.7+)

Supported RSpec version (for RSpec features only): >= 3.5.0 (for older RSpec versions use TestProf < 0.8.0).

Usage

Check out our docs.

What's next

Have an idea? Propose a feature request!

Already using TestProf? Share your story!

License

The gem is available as open source under the terms of the MIT License.

test-prof's Issues

Ruby Interpreter Error with TEST_RUBY_PROF=call_stack

Environment:

Ruby 2.3.4

Rails 4.2.0, RSpec 3.1.0

TestProf 0.4.0

Executed:

bundle exec spec TEST_RUBY_PROF=call_stack

Expected:

To generate a call stack report on test suite.

Actual:

Catastrophic failure...

7fc1ad0cb000-7fc1ad0cd000 r-xp 00000000 08:01 392504                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fc1ad0cd000-7fc1ad2cd000 ---p 00002000 08:01 392504                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fc1ad2cd000-7fc1ad2ce000 r--p 00002000 08:01 392504                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fc1ad2ce000-7fc1ad2cf000 rw-p 00003000 08:01 392504                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fc1ad2cf000-7fc1ad2d6000 r-xp 00000000 08:01 392517                     /lib/x86_64-linux-gnu/librt-2.13.so
7fc1ad2d6000-7fc1ad4d5000 ---p 00007000 08:01 392517                     /lib/x86_64-linux-gnu/librt-2.13.so
7fc1ad4d5000-7fc1ad4d6000 r--p 00006000 08:01 392517                     /lib/x86_64-linux-gnu/librt-2.13.so
7fc1ad4d6000-7fc1ad4d7000 rw-p 00007000 08:01 392517                     /lib/x86_64-linux-gnu/librt-2.13.so
7fc1ad4d7000-7fc1ad4ee000 r-xp 00000000 08:01 392494                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fc1ad4ee000-7fc1ad6ed000 ---p 00017000 08:01 392494                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fc1ad6ed000-7fc1ad6ee000 r--p 00016000 08:01 392494                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fc1ad6ee000-7fc1ad6ef000 rw-p 00017000 08:01 392494                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fc1ad6ef000-7fc1ad6f3000 rw-p 00000000 00:00 0
7fc1ad6f3000-7fc1ad9af000 r-xp 00000000 08:01 23032                      /usr/local/lib/libruby.so.2.3.0
7fc1ad9af000-7fc1adbae000 ---p 002bc000 08:01 23032                      /usr/local/lib/libruby.so.2.3.0
7fc1adbae000-7fc1adbb7000 rw-p 002bb000 08:01 23032                      /usr/local/lib/libruby.so.2.3.0
7fc1adbb7000-7fc1adbc8000 rw-p 00000000 00:00 0
7fc1adbc8000-7fc1adbe8000 r-xp 00000000 08:01 392497                     /lib/x86_64-linux-gnu/ld-2.13.so
7fc1adbfe000-7fc1adc21000 r--s 00000000 08:01 23031                      /usr/local/bin/ruby
7fc1adc21000-7fc1adc63000 rw-p 00000000 00:00 0
7fc1adc63000-7fc1addda000 r--p 00000000 08:01 1262                       /usr/lib/locale/locale-archive
7fc1addda000-7fc1adddf000 rw-p 00000000 00:00 0
7fc1adde1000-7fc1adde5000 r-xp 00000000 00:00 0
7fc1adde5000-7fc1adde7000 rw-p 00000000 00:00 0
7fc1adde7000-7fc1adde8000 r--p 0001f000 08:01 392497                     /lib/x86_64-linux-gnu/ld-2.13.so
7fc1adde8000-7fc1adde9000 rw-p 00020000 08:01 392497                     /lib/x86_64-linux-gnu/ld-2.13.so
7fc1adde9000-7fc1addea000 rw-p 00000000 00:00 0
7fff3cf2f000-7fff3d72e000 rw-p 00000000 00:00 0                          [stack]
7fff3d7b6000-7fff3d7b7000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]


[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

/usr/local/bin/ruby -I/home/vagrant/.gem/gems/rspec-core-3.1.5/lib:/home/vagrant/.gem/gems/rspec-support-3.1.1/lib /home/vagrant/.gem/gems/rspec-core-3.1.5/exe/rspec spec/concepts/acquisition/ failed
rake aborted!
TypeError: no implicit conversion from nil to integer
/home/vagrant/.gem/gems/rspec-core-3.1.5/lib/rspec/core/rake_task.rb:97:in `exit'
/home/vagrant/.gem/gems/rspec-core-3.1.5/lib/rspec/core/rake_task.rb:97:in `run_task'
/home/vagrant/.gem/gems/rspec-core-3.1.5/lib/rspec/core/rake_task.rb:109:in `block (2 levels) in define'
/home/vagrant/.gem/gems/rspec-core-3.1.5/lib/rspec/core/rake_task.rb:107:in `block in define'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli/exec.rb:74:in `load'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli/exec.rb:74:in `kernel_load'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli/exec.rb:27:in `run'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli.rb:362:in `exec'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli.rb:22:in `dispatch'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/cli.rb:13:in `start'
/home/vagrant/.gem/gems/bundler-1.15.4/exe/bundle:30:in `block in <top (required)>'
/home/vagrant/.gem/gems/bundler-1.15.4/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
/home/vagrant/.gem/gems/bundler-1.15.4/exe/bundle:22:in `<top (required)>'
/usr/local/bin/bundle:22:in `load'
/usr/local/bin/bundle:22:in `<main>'

ruby 2.2.0 TypeError: nil can't be coerced into Float

Tell us about your environment

Ruby Version: 2.2.0

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever): RSpec Core 3.4.1

TestProf Version: 0.4.8

What did you do?

Ran this command:

RD_PROF=1 bundle exec rspec /path/to/spec_spec.rb

What did you expect to happen?

Tests would run, slowest let/before blocks would be displayed.

What actually happened?

Error message:

bundler: failed to load command: rspec (/Users/rolah/Code/project/vendor/bundle/ruby/2.2.0/bin/rspec)
TypeError: nil can't be coerced into Float
  /Users/rolah/Code/project/vendor/bundle/ruby/2.2.0/gems/test-prof-0.4.8/lib/test_prof/rspec_dissect/rspec.rb:39:in `+'
  /Users/rolah/Code/project/vendor/bundle/ruby/2.2.0/gems/test-prof-0.4.8/lib/test_prof/rspec_dissect/rspec.rb:39:in `example_finished'

Report is called `.html` but is really a text file

Tell us about your environment

Ruby Version: 2.3.4

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):

• FactoryBot 4.8.2
• RSpec 3.6
• Rails 5.0.x

TestProf Version: 0.4.3

What did you do?

TEST_RUBY_PROF=1 bundle exec rspec ./spec/models/my_model_spec.rb

What did you expect to happen?

TestProf reports:
[TEST PROF INFO] RubyProf report generated: tmp/test_prof/ruby-prof-report-flat-wall-total.html
So i opened the file in the browser but there is no html. As it turns out, it's a flat text file.
Your docs report "we're using FlatPrinter" which is fine. Seems like this file should be called .txt instead of .html if that's the default report format.

$ cat tmp/test_prof/ruby-prof-report-flat-wall-total.html
Measure Mode: wall_time
Thread ID: 70135209785700
Fiber ID: 0
Total: 11.020172
Sort by: self_time

 %self      total      self      wait     child     calls  name
  5.51      0.607     0.607     0.000     0.000       10   <Class::BCrypt::Engine>#__bc_crypt
  3.46      0.382     0.382     0.000     0.000     4304   <Class::Dir>#[]
  3.38      0.373     0.373     0.000     0.000    90880   Kernel#class
  ...

ruby_prof/rspec.rb:14:in `example_started': undefined method `example` for #<RSpec::Core::Example:0x00005639a203b8d0

Firstly, thank you for the wonderful project and numerous thoughful write-ups. I was really excited to give it a go.

Tell us about your environment

ruby 2.3.7p456 (2018-03-28 revision 63024) [x86_64-linux]
rspec-core-2.14.8
rails-4.1.16
factory_girl-4.5.0
test-prof-0.7.1

Also using spring.

What did you do?

Included test-prof in my test group, used the TAG_PROF=type TAG_PROF_EVENT=sql.active_record bin/rspec ENV var to run a specific spec. Note, I'm using binstubs and Spring.

What did you expect to happen?

Complete without breaking.

What actually happened?

TestProf seems to be calling the #example method on the notification object within the notification callback, whhich ends up resolving to the spec context where this method is undefined, if I'm reading the rror correctly.

Ruby Prof extremely slow after running a large amount of tests

The TEST_RUBY_PROF=1 is extremely slow when generating the report after running all rspec tests in the app. I killed it after the process was using 100% CPU and didn't stop for an hour. Is there a way to see what the problem is or to make it verbose in some way to see what is going on?

Can't enable profilers with ENV vars if spring is in Gemfile

Hi!
I ran TEST_STACK_PROF=1 bundle exec rspec but profiler was not activated because I had spring required in my Gemfile (without require: false). The problem seems to be here
If Spring is defined? it doesn't actually mean that it's running.

I think it's going to be a common problem because spring tells you (in README) to add it in Gemfile without require: false.

Fabrication support for FactoryProf

Add Fabrication gem support to FactoryProf:

• Track factories usage (top-level invocations and total)

• Collect factory stacks and generate Factory Flame reports.

before_all hooks

I'm working on speeding up the discourse tests using let_it_be. We need a way to run code that sets up global variables before each before_all block is invoked. I can do this by providing a custom BeforeAll adapter within discourse, but if you would like, I could make it more general and submit a PR here. What would you prefer?

Spec with `let_it_be` fails after the first example when run after capybara JS spec

Tell us about your environment

Ruby Version: 2.6.2

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever): RSpec 3.8, FactoryBot 4.8.2, Rails 5.1.7

TestProf Version: 0.8.0

What did you do?

I have a spec file that uses let_it_be. If I run this test on its own, it passes.

When I run a JS feature spec (That uses capybara with chrome driver) and then the spec, the spec fails after the first example.

What did you expect to happen?

The spec passes even if run together with a JS spec before it.

What actually happened?

I get errors about missing data and a !!! before_all transaction has been already rollbacked and could work incorrectly at the end so that means the before_all transaction was rolled back prematurely after the first example.

I used pry to step through the code and find out where it's happening. In ActiveRecord::TestFixtures#teardown_fixtures which is run after each example, ActiveRecord::Base.clear_active_connections! is called.

If the test is run individually, ActiveRecord::Base.connection.open_transactions remains at 1 even after that is called.

When run after the JS test, ActiveRecord::Base.connection.open_transactions becomes 0 in the teardown for the non-JS spec's first example. Meaning the transaction for before_all was rolled back. I'm not really sure why.

I tried enabling AR logging and I don't see any unexpected explicit rollback.

UPDATE:

I've narrowed it down to running a JS spec that uses DatabaseCleaner's delete strategy. When running a non-JS spec with deletion it works fine.

I know you don't really support the use of let_it_be with DatabaseCleaner, but that seems to work fine except for this case with JS tests.

Any idea what might be happening here?

Can `let_it_be` and `before_all` be used in the same example group?

RSpec.describe Object do
  let_it_be(:foo) { 'this is foo' }
  
  before_all { @bar = 'this is bar' }

  it 'does something' do
    @foo #=> nil
    @bar #=> nil
    foo #=> 'this is foo'
    bar #=> NameError: undefined local variable or method 'bar'
  end
end

If I remove the let_it_be then the before_all runs. In the test @bar #=> this is bar and of course foo #=> NameError for 'foo'.

EventProf Minitest support

Add Minitest integration support to EventProf.

Implementation ideas

Tracking

I propose to use minitest-reporters for tests tracking purposes (at least for now). It works pretty similar to RSpec reporter listeners, so could be easily integrated with EventProf core functionality.

Configuration

We should support both env variables and CLI options (thus, we have to provide a Minitest plugin (see Rails plugin, for example)):

# that should work
EVENT_PROF=sql.active_record rake test

# and that too
ruby my_test.rb --event_prof=sql.active_record

Broken specs after installing

Ruby 2.2.5
Rspec, factory girl, capybara

After installing test-prof 37 broken specs appeared:
412 examples, 37 failures, 7 pending
After uninstalling test-prof:
412 examples, 0 failures, 7 pending
The most common error - deadlock

ActiveRecord::StatementInvalid:
        PG::TRDeadlockDetected: ERROR:  deadlock detected at character 13
        DETAIL:  Process 11977 waits for RowExclusiveLock on relation 5270960 of database 5270706; blocked by process 30236.
        Process 30236 waits for ShareRowExclusiveLock on relation 5270996 of database 5270706; blocked by process 11977.
        HINT:  See server log for query details.

0.4.9 breaks standard Rails setup

The update from test-prof v0.4.8 to v0.4.9 broke my test setup.

The guides describe to require the RSpec recipe "before_all" in the spec_helper.rb. Commit 4ffac87#diff-dee29c4f171f219aa13eb1dc872f1775R3 breaks the setup because ActiveRecord is not loaded in the spec_helper.rb but in the rails_helper.rb.

I'd suggest to either adjust the guides or to move the AR check to somewhere where the class is available.

Support merging results?

How complex would it be to support merging test results? We're running tests in parallel, its easy to configure each one to spit out separate files but it would be great to be able to merge results across the entire test suite into one report of flamechart.

[TEST PROF ERROR] appears when running FactoryProf, but results are correct

Ruby 2.3.4

Rails 4.2.0, RSpec 3.1.0

TestProf 0.4.0

After FPROF=1 rspec run I've got the following output:

[TEST PROF INFO] FactoryProf enabled (simple mode)
[TEST PROF ERROR]

However, when tests are finished, I have proper output about factories usage.

It also appears with FPROF=flamegraph rspec

How to use test-prof along with spring?

I'm new (and happy) to test-prof, step by step adding more recipes to my suite. I'm trying to run the rspec build using spring (previously started with a bin/rake db:setup command, for instance, to speed up our CI. But spring seems to ignore the env variables that start the test-prof tools.

So this works with test-prof, but not with spring:

TEST_STACK_PROF=1 FPROF=1 bundle exec rspec spec/tasks

And this works with spring, but not with test-prof:

bundle exec spring rspec spec/tasks
# or 
bin/rspec spec/tasks

Question is: how to use spring and activate FPROF on a particular run?

NoMethodError: undefined method `version' for nil:NilClass

Tell us about your environment

Ruby Version:

ruby 2.2.6p396

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):

RSpec (via rspec-rails v 3.6.0)

TestProf Version:

test-prof (0.4.4)

What did you do?

ran specs when using let_it_be

What did you expect to happen?

not to fail :)

What actually happened?

NoMethodError:
  undefined method `version' for nil:NilClass

This is due to the fact that it is looking for 'rspec' gem specifically instead of 'rspec' or 'rspec-rails'

Issue with create_default and traits

I have this factory, with a trait that allows changing the association general definition like this:

FactoryBot.define do
  factory :idea do
    association :status, :default

    trait :bla do
      association :status, :bla
    end
  end
end 

When I use FactoryDefault to define a default status like this in a shared context:

shared_context 'My context' do  
  let_it_be(:status) { create :status }

  before do
    set_factory_default :status, status
  end
end

And write my spec:

include_context 'My context'

it do
  create :idea #=> is created with the default :status from shared context
  create :idea, :bla #=> is ALSO created with the default :status from shared context
end

So in practice trait :bla is not used at all, set_factory_default is overriding that. This was, at least, surprising and unexpected. I expected the trait behaviour to preveal, since it is explicitly declaring the association.

Is this intented behaviour or is this a 🐛?

Ruby Version: 2.3.7
Framework Version: Rspec 3.6
TestProf Version: 0.5.0

NoMethodError: undefined method `dump' for nil:NilClass in ./lib/test_prof/ruby_prof/rspec.rb:21:in `example_finished'

..snip..
# bin/rspec:9:in `load'
# bin/rspec:9:in `<main>'
#
#   Showing full backtrace because every line was filtered out.
#   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
#   RSpec::Configuration#backtrace_inclusion_patterns for more information.
# ------------------
# --- Caused by: ---
# NoMethodError:
#   undefined method `dump' for nil:NilClass
#   /Users/dev/foo/vendor/bundle/ruby/2.4.0/gems/test-prof-0.7.5/lib/test_prof/ruby_prof/rspec.rb:21:in `example_finished'

https://github.com/palkan/test-prof/blob/master/lib/test_prof/ruby_prof/rspec.rb#L21

Tell us about your environment

Ruby Version: ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin18]

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):

⇒  rspec --version
RSpec 3.7
  - rspec-core 3.7.1
  - rspec-expectations 3.7.0
  - rspec-mocks 3.7.0
  - rspec-rails 3.7.2
  - rspec-support 3.7.1

TestProf Version: 0.7.5

RubyProf Version:

  gem 'ruby-prof',
    :git => 'https://github.com/ruby-prof/ruby-prof',
    :ref => 'a813bdd7891806cd26ec8c71ea6297827e07fa54',
    require: false

What did you do?

Tried to run rspec on one of my spec files. I had:rprof set on my spec test case.

TEST_RUBY_PROF=multi bin/rspec ./spec/features/foo/bar_spec.rb:120

What did you expect to happen?

What actually happened?

Got the above error

[5] pry(#<TestProf::RubyProf::Listener>)> whereami

From: /Users/dev/foo/vendor/bundle/ruby/2.4.0/gems/test-prof-0.7.5/lib/test_prof/ruby_prof/rspec.rb @ line 21 TestProf::RubyProf::Listener#example_finished:

    19: def example_finished(notification)
    20:   binding.pry
 => 21:   return unless profile?(notification.example)
    22:   notification.example.metadata[:rprof_report].dump(
    23:     notification.example.full_description.parameterize
    24:   )
    25: end

[6] pry(#<TestProf::RubyProf::Listener>)> notification.example.metadata[:rprof_report]
=> nil

Wrong time spent values due to patched Time class

TestProf uses Time.now internally to measure the time, but it can be patched/freezed by tools like timecop.

We shouldn't rely on Time.now and use our own copy of the method (like RSpec does).

Improve specs sampling

TestProf allows to run a random subset of your test suite but this feature has some limitations:

• it only supports running example groups (or test files) and not individual examples
• it doesn't work with RSpec filters.

We have to improve sampling and add the following features:

• Make it possible to sample examples or groups:

# sample individual examples
SAMPLE=10 rspec
# or
SAMPLE=10 rake test

# sample example groups
SAMPLE_GROUPS=10 rspec

# or files for Minitest
SAMPLE_GROUPS=10 rake test

• Make it work with RSpec filters.

# with tags
SAMPLE=10 rspec --tag type:controller

# or filters
SAMPLE=10 rspec -e "feature"

Should run ten random controller specs.

Issue with stubbing methods using before_all/let_it_be

Hi,
In the last couple of months, we've introduced test-prof into our test environment and I have to say it works really nice. Great job guys 👏

What I was wondering is if we can consider this use case:
spec_helper.rb

config.before(:each) do 
  $redis = MockRedis.new
  allow($redis).to receive(:publish).and_return(true)
end

in consultation.rb we have callback where we do
$redis.publish "notification", params

in specs/models/consultation.rb:

let_it_be(:consultation) { create(:consultation) } 
....

This unit specs are going to fail with following error

NoMethodError:
        undefined method `publish' for #<MockRedis::Database:0x005649957bbdf0>

The reason is - consultation is created before we do stubbing $redis. When we've used let/let! in our unit specs it was good.

@palkan What do you think?
It might be good to consider stubbing way inside ruby-prof hooks even Mr. Chelimsky says differently
https://github.com/rspec/rspec-mocks/pull/115/files

Detect database_cleaner usage along with before_all

Using database_cleaner along with before_all (and other group-wide helpers, e.g. let_it_be) could break things (see #107, #106).

We need to find a way to warn users about possible issues by printing a warning (or raising?):

Warning! Usage of database_cleaner could break before_all/let_it_be functionality.
Consider turning off or removing database_cleaner at all.

See https://test-prof.evilmartians.io/#/active_record_shared_connection.

When non-transactional strategy is used (i.e. truncation/deletion), it's better to raise an exception ('cause it would definitely break everything).

From #106 we can see that :transaction strategy could also be breaking; probably, the reason is that is used along with transactional_fixtures = true. Need to investigate these scenarios.

before_all transaction has been already rollbacked and could work incorrectly

I'm getting a few of these error messages:

!!! before_all transaction has been already rollbacked and could work incorrectly

Any idea how I should go about debugging them, or more info as to the meaning?

Hangs indefinitely after tests

Is there something I need to do to cause TestProf to stop profiling? I'm finding that as soon as I enable this, with only instructions from the Readme. After all tests are done, the process just hangs indefinitely (on our CI server, it was 5 hours before the task was killed).

Is this a known issue?

I'm just using the following in spec_helper.rb

TestProf.configure do |config|
  config.color = true
end

TestProf::RubyProf.configure do |config|
  config.printer = :call_stack
end

and

TEST_RUBY_PROF=1 bundle exec rspec

Ruby version: 2.4.3
Latest version of TestProf and RubyProf

Support for Ruby 2.2.x

Hi, would it be possible to provide support for Ruby 2.2.x? At the moment I'm using Ruby 2.2.5p319 and when I run bundle I get:

Bundler could not find compatible versions for gem "ruby":
  In Gemfile:
    ruby

    test-prof was resolved to 0.1.0, which depends on
      ruby (>= 2.3.0)

AnyFixture vs before_all

It seems both accomplish quite a similar goal - create shared setup only once instead of per example. However, I'm not sure which one is better to use. What are the considerations, that one has to take into account when choosing? Or are they complementary?

Add totals to FactoryProf simple report

Add a header with the general information on factories usage:

[TEST PROF INFO] Factories usage

# new block to be added
Total: 12321
Total top-level: 3234
Total uniq factories: 42

# then goes the existing table
total       top-level            name
# ...

Unexpected behavior using rubocop autocorrect

Tell us about your environment

Ruby Version:
2.4.4

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):
RSpec 3.7.0
FactoryBot 4.10.0
Rails 5.2.0
Rubocop 0.56.0

TestProf Version:
0.5.0

What did you do?

Installed and turned on the Rubocop RSpec/AggregateFailures cop, with AutoCorrect: true

What did you expect to happen?

My it { is_expected.to eq(xyz) } blocks would be aggregated, but not pending or its(:property) blocks (from the rspec-its gem)

What actually happened?

pending { is_expected.to blahblahblah } blocks were aggregated and activated (no longer pending) and its(:property_a) { is_expected.to eq(property_a) } blocks were changed to regular it comparisons (no longer checking against the property, but against the object.) Both changes cause the test suite to fail.

No output when running FactoryDoctor

I get output when I run:

$ EVENT_PROF=sql.active_record rspec
Running via Spring preloader in process 12713
[TEST PROF INFO] EventProf enabled (sql.active_record)

Randomized with seed 59903
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................[TEST PROF INFO] EventProf results for sql.active_record

Total time: 00:32.365
Total events: 137490

Top 5 slowest suites (by time):

... – 00:02.208 (10042 / 33)
... – 00:02.021 (6947 / 32)
... – 00:01.979 (7116 / 30)
... – 00:01.480 (5592 / 15)
... – 00:01.265 (4761 / 9)



Finished in 5 minutes 13 seconds (files took 1.78 seconds to load)
723 examples, 0 failures

However, when I run with FDOC=1 I don't get any output.

$ FDOC=1 rspec
Running via Spring preloader in process 13001
[TEST PROF INFO] FactoryDoctor enabled

Randomized with seed 59903
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 5 minutes 28 seconds (files took 1.75 seconds to load)
723 examples, 0 failures

In my rails_helper.rb I have

require "test-prof"

TestProf.configure do |config|
  # the directory to put artifacts (reports) in ("tmp" by default)
  config.output_dir = "tmp/test_prof"

  # use unique filenames for reports (by simply appending current timestamp)
  config.timestamps = true

  # color output
  config.color = true
end

What is my malfunction here?

Using threads appears to confuse `before_all`.

Tell us about your environment

Ruby Version:
2.4.2

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):
rspec: 3.7
rails: 5.0.7

TestProf Version:
0.4.9

What did you do?

I added a test to make sure concurrent updates were not triggering a race condition, which involved multiple simultaneous threads.

What did you expect to happen?

The test would pass or fail on its own merits.

What actually happened?

The tested passed, but I also got this error at the end of an unrelated test in a different file:

An error occurred in an `after(:context)` hook.
Failure/Error: transaction.rollback

NoMethodError:
  undefined method `rollback' for nil:NilClass
# /Users/robert/.rvm/gems/ruby-2.4.2@backoffice/gems/test-prof-0.4.9/lib/test_prof/recipes/rspec/before_all.rb:24:in `block in before_all'

Adding if ActiveRecord::Base.connection.open_transactions > 0 to the end of the offending line makes the error go away. I don't understand enough of what's going on to say if that's the correct fix or if that's just masking a different issue.

bug in before_all with MySql2 adapter

this is the error message I got when trying to use it with MySQL:

Failure/Error: ActiveRecord::Base.connection.begin_transaction(joinable: false)

NoMethodError:
   undefined method `begin_transaction' for #<ActiveRecord::ConnectionAdapters::Mysql2Adapter:0x007fc049cc3060>

include context with `let_it_be` in RSpec's config

Tell us about your environment

Ruby Version:2.5.3

Framework Version: RSpec 3.8

TestProf Version: 0.7.3

What did you do?

I want to create a context that'll be included in all of the specs that requires rails_helper

I've added following content to rails_helper.rb:

require 'test_prof/recipes/rspec/let_it_be'
require 'test_prof/recipes/rspec/before_all'

RSpec.shared_context :regions do
  let_it_be(:global_region) { FactoryBot.create(:region, :global) }
  let_it_be(:poland_region) { FactoryBot.create(:region, :poland) }
end

RSpec.configure do |config|
  config.include_context :regions
end

What did you expect to happen?

Have access to global_region and poland_region from each of the spec's that require rails_helper

What actually happened?

Failure/Error: let_it_be(:global_region) { FactoryBot.create(:region, :global) }

NoMethodError:
  undefined method `let_it_be' for RSpec::ExampleGroups::SerpCheckChannel:Class

I'm able to use created context within a single spec with include_context :regions_spec inside of the spec, but I'd rather include this context once for all of the specs.

Order of truncating tables in after(:suite) for AnyFixture

Say you have verious levels of nesting of models:

class Foo < ApplicationRecord
  has_many :bars, dependent: :destroy
end

class Bar < ApplicationRecord
  belongs_to :foo
  has_many :bazs, dependent: :destroy
end

class Baz < ApplicationRecord
  belongs_to :bar
end

The logical thing to do is to first delete all the bazs, then the bars and finally the foos. If this order is not respected, one might potentially get an error like this one:

ActiveRecord::InvalidForeignKey:
  PG::ForeignKeyViolation: ERROR:  update or delete on table "products" violates foreign key constraint "fk_from_subscriptions_to_products" on table "subscriptions"
  DETAIL:  Key (id, facility_id)=(1, 1) is still referenced from table "subscriptions".
  :
              DELETE FROM "products"

What do you think? Is it feasible to build some logic for the order of the deletion of tables?

README configuration causes error on first run of stackprof

When running TEST_STACK_PROF=1 rspec with the config mentioned in the README

TestProf.configure do |config|
  # the directory to put artifacts (reports) in ("tmp" by default)
  config.output_dir = "tmp/test_prof"

  # use unique filenames for reports (by simply appending current timestamp)
  config.timestamps = true

  # color output
  config.color = true
end

Produces an error if "tmp/test_prof" doesn't exist. Ideally, TestProf would create this directory if it doesn't exist.

any_fixture DSL is not usable in Ruby 2.3.4

Tell us about your environment

Ruby Version: 2.3.4

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever): rspec 3.7

TestProf Version: 0.5.0

What did you do?

Tried to require 'test_prof/any_fixture/dsl'

What did you expect to happen?

The DSL to do whatever it needs to so I can using TestProf::AnyFixture::DSL

What actually happened?

TypeError: wrong argument type Module (expected Class)
from {my gem path}/lib/test_prof/any_fixture/dsl.rb:13:in `refine'

Looks like the test suite works around this:

https://github.com/palkan/test-prof/blob/02b8bc141ece9d3c021df5c6ffa7eb17e3ff10ea/spec/integrations/fixtures/rspec/any_fixture_fixture.rb#L8-L18

Am I expected to do the same in rails_helper or wherever?

🤖FactoryBot killed 🙅FactoryGirl

Since FactoryGirl has been renamed to FactoryBot we have to support both names now (for users still using good old FactoryGirl).

I propose to handle this duality by using our own TestProf::FactoryBot (just an alias for FactoryGirl or FactoryBot) everywhere in codebase.

We should also use factory_bot by default in our tests and add a separate Travis gemfile for factory_girl.

TagProf empty output

Ruby 2.3.4

Rails 4.2.0, RSpec 3.1.0

TestProf 0.4.0

I run TAG_PROF=type rspec, and its output contains just table headers:

type time total %total %time avg

And nothing after that. I've tried with configuration from "Pro-Tip: More Types" and without it, results are the same.

Add percentage and total test time to EventProf

Add a percentage of time spent in the event (from the total test suite/group/example time)

Before:

[TEST PROF INFO] EventProf results for factory.create

Total time: 03:07.353
Total events: 7459

Something (./my_spec.rb:21) – 09:16.100 (7 / 3)
...

After:

[TEST PROF INFO] EventProf results for factory.create

Total time: 03:07.353 of 07:25:232 (42.02%)
Total events: 7459

Something (./my_spec.rb:21) – 09:16.100 (7 / 3) of 12:29:233 (74.13%)
...

let_it_be reload alias not functioning

Ruby Version: 2.5.1

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever): RSpec 3.7.0

TestProf Version: 0.9.0

What did you do?

TestProf::LetItBe.configure do |config|
  # define an alias with `reload: true` by default
  config.alias_to :let_it_be_with_reload, reload: true
end

What did you expect to happen?

I expected that let_it_be_with_reload(:x) would function identically to let_it_be(:x, reload: true)

What actually happened?

The created object is not removed at when leaving the the RSpec context.

The following RSpec test illustrates the problem:

RSpec.describe 'let_it_be functions correctly', order: :defined do
  let_it_be(:email) { FactoryGirl.build(:email) }

  context 'without reload' do
    context 'first context' do
      let_it_be(:user_one) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end

    context 'second context' do
      let_it_be(:user_one) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end
  end

  context 'with explicit reload' do
    context 'first context' do
      let_it_be(:user_one, reload: true) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end

    context 'second context' do
      let_it_be(:user_one, reload: true) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end
  end

  context 'with reload alias' do
    context 'first context' do
      let_it_be_with_reload(:user_one) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end

    context 'second context' do
      let_it_be_with_reload(:user_one) { FactoryGirl.create :c_user, email: email }

      it 'has only one user' do
        expect(User.count).to eq 1
      end
    end
  end
end

let_it_be works correctly, it also works correctly with the explicit reload, but it does not work correctly with the alias:

$ rspec -fdoc spec/let_it_be_spec.rb

Randomized with seed 55677

let_it_be functions correctly
  without reload
    first context
      has only one user
    second context
      has only one user
  with explicit reload
    first context
      has only one user
    second context
      has only one user
  with reload alias
    first context
      has only one user
    second context
      has only one user (FAILED - 1)

Fixture_file_upload not working in combination with let_it_be helper

Tell us about your environment

Ruby Version:
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):
RSpec
TestProf Version:
0.7.2

What did you do?

When I do a controller specific test I wanted to attach a fixture to the model by basically calling fixture_file_upload during the controller tests. This is fine when using a let block for every test. However when I use this in combination with a let_it_be block it throws sometimes this strange error:

 An error occurred in an `after(:context)` hook.
Failure/Error: transaction.rollback

NoMethodError:
  undefined method `rollback' for nil:NilClass
# -e:1:in `<main>'

ActiveRecord::RecordNotFound:
     Couldn't find Educator with 'id'=1

This is how my let blocks look like:
let_it_be(:educator, refind: true) { create :educator } -> Runs fine within let block
/// Some other stuff
let(:somemodel) { create :somemodel, avatar: fixture_file_upload(Rails.root.join('spec', 'fixtures', 'images', 'somelogo.png')) }

What did you expect to happen?

I expected that all the tests will pass regardless of use of fixture_file_upload or not.

What actually happened?

The Educator is dropped before I can actually call it in the next method.

Ruby prof "inconsistent call tree" error

Tell us about your environment

Ruby Version: 2.5.1
Framework Version: RSpec 3.6.0
TestProf Version: 0.6.0
RubyProf Version: 0.17.0

Error is independent from using spring or not.

What did you do?

TEST_RUBY_PROF=call_stack bin/rspec spec/models

What did you expect to happen?

A ruby-prof output :P

What actually happened?

Specs run correctly, but after the build there was an error with the following stack:

Running via Spring preloader in process 75237
[TEST PROF INFO] Activating TEST_RUBY_PROF with `Spring.after_fork`
[TEST PROF INFO] RubyProf enabled

... build running ...

Finished in X minute Y seconds (files took 4.16 seconds to load)
Z examples, 0 failures, 1 pending

Randomized with seed 31213
DEPRECATION WARNING: Method 'eliminate_methods!' is dprecated
Please call 'exclude_methods!' before starting the profile run instead.
Traceback (most recent call last):
	80: from -e:1:in `<main>'
	79: from /Users/dgilperez/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	78: from /Users/dgilperez/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	77: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/test-prof-0.6.0/lib/test_prof/ruby_prof.rb:171:in `block in run'
	76: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/test-prof-0.6.0/lib/test_prof/ruby_prof.rb:108:in `dump'
	75: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:13:in `eliminate_methods!'
	74: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:13:in `each'
	73: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:14:in `block in eliminate_methods!'
	72: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:14:in `each'
	71: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:14:in `block (2 levels) in eliminate_methods!'
	70: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/profile/legacy_method_elimination.rb:40:in `eliminate_methods'
	69: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/method_info.rb:116:in `eliminate!'
	68: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/method_info.rb:116:in `each'
	67: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/method_info.rb:116:in `block in eliminate!'
	66: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:74:in `eliminate!'
	65: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:74:in `each'
	64: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:76:in `block in eliminate!'
	63: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	62: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	61: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	60: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	59: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	58: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	57: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	56: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	55: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	54: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	53: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	52: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	51: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	50: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	49: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	48: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	47: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	46: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	45: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'
	44: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `each'
	43: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:105:in `block in merge_call_tree'
	42: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:102:in `merge_call_tree'

        .... same pattern for 40 lines ... 

	 1: from /Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:103:in `block in merge_call_tree'
/Users/dgilperez/.rvm/gems/ruby-2.5.1/gems/ruby-prof-0.17.0/lib/ruby-prof/call_info.rb:91:in `find_call': inconsistent call tree (RuntimeError)

So this looks like a ruby-prof error? Maybe related to Ruby 2.5.1? Can anyone reproduce?

before_all remove resource from DB after first `each` hook

Hello,

We started using test-prof library to speed up our current specs, but we struggled into very weird issue.
So, the current specs are looking as bellow:

  before_all do
    @calendar_access = create(:calendar_access)
    @agenda = @calendar_access.agenda
  end

  let(:agenda) { @agenda }
  let(:calendar_access) { @calendar_access }
  let(:start) { 1.day.ago }

  before(:each) do
    binding.pry
    @diff = ConsultationsDiff.new(agenda, old_consultations, old_consultations, new_consultations)
  end

..........

So first example passes well, but on second one in debugger we're over:

agenda.reload #=> Couldn't find DoctorAgenda with 'id'=145 [WHERE "agendas"."type" IN ('DoctorAgenda')]
@agenda.reload #=> Couldn't find DoctorAgenda with 'id'=145 [WHERE "agendas"."type" IN ('DoctorAgenda')]
Agenda.all #=> []
....
same for `callendar_access`. 

I think it's because of DatabaseCleaner but when I comment it the issue remains. 

Do you have any thought on this ?

Any help is appreciated.

Regards

TestProf::BeforeAll::AdapterMissing ??

Environment

Ruby Version:
2.4.0

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever):
rails-5.1.3
rspec-3.6.0
factory_girl_rails-4.8.0

TestProf Version:
test-prof-0.7.4

What did you do?

# rails_helper.rb
require 'test_prof/recipes/rspec/let_it_be'

Global find-and-replace let(: with let_it_be(:.

What did you expect to happen?

It would "just work"

What actually happened?

Lots of this:

TestProf::BeforeAll::AdapterMissing:
  Please, provide an adapter for `before_all` through `TestProf::BeforeAll.adapter = MyAdapter`
# /Users/johncox/.rvm/gems/ruby-2.4.0@dont_fear/gems/test-prof-0.7.4/lib/test_prof/before_all.rb:24:in `rollback_transaction'
# /Users/johncox/.rvm/gems/ruby-2.4.0@dont_fear/gems/test-prof-0.7.4/lib/test_prof/recipes/rspec/before_all.rb:22:in `block in before_all'

Any thoughts? I feel like I'm just missing something simple in the documentation, but I've been through it several times.

Unitialized constant `TestProf::FactoryBot`

Tell us about your environment

Ruby Version: 2.4.2

Framework Version (RSpec, Minitest, FactoryGirl, Rails, whatever): Rspec 3.7.0, FactoryBot 4.8.2, Rails 5.2.pre

TestProf Version: 0.4.8

What did you do?

I am following the instructions to add the Factory Default recipe.

What did you expect to happen?

Code would load and work.

What actually happened?

Get an error...

/Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/factory_default/factory_bot_patch.rb:6:in `<module:RunnerExt>': uninitialized constant TestProf::FactoryBot (NameError)
Did you mean?  TestProf::FactoryDefault
	from /Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/factory_default/factory_bot_patch.rb:5:in `<module:FactoryDefault>'
	from /Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/factory_default/factory_bot_patch.rb:4:in `<module:TestProf>'
	from /Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/factory_default/factory_bot_patch.rb:3:in `<top (required)>'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	from /Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/factory_default.rb:3:in `<top (required)>'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:59:in `require'
	from /Users/mss67/.rvm/gems/ruby-2.4.2@memberhub-commerce-api/gems/test-prof-0.4.8/lib/test_prof/recipes/rspec/factory_default.rb:3:in `<top (required)>'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:135:in `require'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
	from /Users/mss67/.rvm/rubies/ruby-2.4.2/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:40:in `require'
	from /Users/mss67/projects/5p/memberhub-commerce-api/spec/spec_helper.rb:3:in `<top (required)>'

Tag Profiler not running

I'm running my rspec suite with TAG_PROF=type bundle exec rspec and there is no test-prof output. I've followed the installation instructions and added test-prof to the Gemfile, bundled, and added this configuration to by rails helper:

require 'test-prof'

TestProf.configure do |config|
  # the directory to put artifacts (reports) in ("tmp" by default)
  config.output_dir = "tmp/test_prof"

  # use unique filenames for reports (by simply appending current timestamp)
  config.timestamps = true

  # color output
  config.color = true
end

I'm using Rails 5.0.4, rspec-rails 3.5.0, and test-prof 0.1.0.

Saving reports to file for later analysis

I'm using your gem to analyze a huge project and I'm wondering if there's an option to collect all Profiler's results into plain text files for later analysis.

Right now all the results are outputted to stdout and are not handy to be managed when are executed in CI. I wish to have it in plain files and get it from CI as artifacts.

Any idea? Thank you :)

Add `let_it_be` aliases support

Ref discourse/discourse#7414 (comment)

Add ability to register let_it_be aliases with the predefined options:

TestProf::LetItBe.configure do |config|
  config.alias_to :fab!, refind: true
end

Unable to load gem

Hi. Trying to use your gem in my Rails 5 application, and I get the following error:

Gem Load Error is: undefined method `shared_context' for RSpec:Module

The error occurs at

/ruby-2.3.1/gems/test-prof-0.2.0/lib/test_prof/ruby_prof/rspec.rb:4

According to the documentation I just need to add the Gem to the project, which I have done. Bundler ran fine, no errors.

Am I required to put some configuration into rails_helper?.

Any help with this would be greatly appreciated.

Use both let_it_be and create_default

Hello,

I'm trying to use both let_it_be and create_default in a test file and I have difficulties to do so because create_default is reset after each test.

My spec files looks like this :

context "with a valid Foo" do
  it "does A" do ... end
  it "does B" do ... end
  it "does C" do ... end
end

context "with a valid Bar" do
  it "does A" do ... end
  it "does B" do ... end
  it "does C" do ... end
end


private
# In the end I'd like all my created records to belong to the same tenant
let_it_be(:tenant){ create_default(:tenant) }

My tenant is created before all test and is used for the first test but then FactoryDefault is cleared and my tenant is not used anymore as default for the other 5 tests.

Am I missing something ? How should I benefit from those two mechanisms ?