Brittle specs

[ruby-concurrency]

Brittle specs

[pitr-ch]

It looks like sleep 0.1 was not enough in one of the agent tests. I am thinking that we should probably remove all the sleep based tests in favor of explicit waiting on events and such. In Agent case most of the sleep calls could be replaces by agent.await.

[pitr-ch]

another

1) Concurrent::Runnable instance #run! creates a new thread
     Failure/Error: Thread.should_receive(:new).with(no_args()).and_return(t)
       (<Thread (class)>).new(no args)
           expected: 1 time with no arguments
           received: 5 times with no arguments
     # ./spec/concurrent/runnable_spec.rb:159:in `Concurrent'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/error_generator.rb:144:in `__raise'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/error_generator.rb:48:in `raise_expectation_error'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/message_expectation.rb:259:in `generate_error'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/message_expectation.rb:215:in `verify_messages_received'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/method_double.rb:205:in `verify'
     # org/jruby/RubyArray.java:1613:in `each'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/method_double.rb:205:in `verify'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/proxy.rb:112:in `verify'
     # org/jruby/RubyArray.java:1613:in `each'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/proxy.rb:112:in `verify'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/space.rb:14:in `verify_all'
     # org/jruby/RubyHash.java:1382:in `each_value'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks/space.rb:13:in `verify_all'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-mocks-2.14.6/lib/rspec/mocks.rb:18:in `verify'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/mocking/with_rspec.rb:18:in `verify_mocks_for_rspec'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:307:in `verify_mocks'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:299:in `run_after_each'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:120:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:112:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:254:in `with_around_each_hooks'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:111:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:109:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:390:in `run_examples'
     # org/jruby/RubyArray.java:2409:in `map'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:386:in `run_examples'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:371:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:369:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:372:in `run'
     # org/jruby/RubyArray.java:2409:in `map'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:372:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:369:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:28:in `run'
     # org/jruby/RubyArray.java:2409:in `map'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:28:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/reporter.rb:58:in `report'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:25:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/runner.rb:80:in `run'
     # /usr/local/var/lib/rbenv/versions/jruby-1.7.12/lib/ruby/gems/shared/gems/rspec-core-2.14.8/lib/rspec/core/runner.rb:17:in `autorun'

[pitr-ch]

another can be found at https://travis-ci.org/jdantonio/concurrent-ruby/builds/25938067, I may have caused some of them. I was trying to fix it (at least some), but I am out of time. I would suggest to postpone the 0.6 release until we hunt them down.

[jdantonio]

We'll work on the brittle specs for 0.7.0. This is a holiday weekend in the US (Memorial Day is Monday) so I have many family activities planned. We've been fixing brittle specs as we find them. We have a high degree of confidence that it's the tests and not the code so I don't think we need to push too hard to fix them all this weekend or to continue holding off 0.6.0.

[mighe]

Generally speaking, sleep based test are not a great idea, even if sometimes are the only way to check if a thread is sleeping or dead.
Our suite is becoming also really slow, mostly due to sleeps. In #16 we were discussing about stress tests and we implemented ImmediateExecutor to run some kinds of test without concurrency, this way it is possible to highlight easily issues with locks and deadlocks.
For the next release we could start thinking a more coherent way to write tests

[pitr-ch]

A better approach could be to terminate/kill everything naturally after each test and test that there are no threads hanging. It could be useful to object-ify any global states we have in the gem into a single object holding it, like ActorWorld in Akka, allowing creation of more concurrent-ruby worlds. We could have a file e.g. concurrent/global which would crete single global world for convenience, but we would not be limited to that. This would also allow us to run tests concurrently which should be possible with gem calling itself concurrent-ruby ;)

[jdantonio]

• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/27467109

[jdantonio]

• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/27997662
• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/27997663
• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/28070742
• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/28074869
• https://travis-ci.org/ruby-concurrency/concurrent-ruby/jobs/28074871