jasmine:ci rake task doesn't return non zero exit code if exception occurs outside of tests

[lyahdav]

jasmine:ci rake task doesn't return non zero exit code if exception occurs outside of tests

[ptolemarch]

The same thing happens right now if you're running Firefox Beta (Firefox 19), which doesn't seem to be supported by Selenium just yet:

/Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/firefox/launcher.rb:79:in `connect_until_stable': unable to obtain stable firefox connection in 60 seconds (127.0.0.1:7055) (Selenium::WebDriver::Error::WebDriverError)
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/firefox/launcher.rb:37:in `block in launch'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/firefox/socket_lock.rb:20:in `locked'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/firefox/launcher.rb:32:in `launch'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/firefox/bridge.rb:20:in `initialize'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/common/driver.rb:31:in `new'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver/common/driver.rb:31:in `for'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/selenium-webdriver-2.29.0/lib/selenium/webdriver.rb:67:in `for'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/bundler/gems/jasmine-gem-479dccb62f9e/lib/jasmine/selenium_driver.rb:20:in `initialize'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/bundler/gems/jasmine-gem-479dccb62f9e/lib/jasmine/run_specs.rb:20:in `new'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/bundler/gems/jasmine-gem-479dccb62f9e/lib/jasmine/run_specs.rb:20:in `<top (required)>'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/activesupport-3.2.8/lib/active_support/dependencies.rb:245:in `load'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/activesupport-3.2.8/lib/active_support/dependencies.rb:245:in `block in load'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/activesupport-3.2.8/lib/active_support/dependencies.rb:236:in `load_dependency'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/activesupport-3.2.8/lib/active_support/dependencies.rb:245:in `load'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/configuration.rb:746:in `block in load_spec_files'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/configuration.rb:746:in `map'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/configuration.rb:746:in `load_spec_files'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/command_line.rb:22:in `run'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/runner.rb:69:in `run'
    from /Users/cogent/.rvm/gems/ruby-1.9.3-p194@slashmob/gems/rspec-core-2.10.1/lib/rspec/core/runner.rb:10:in `block in autorun'

Since this still returns as a success, rake ship proceeds unimpeded.

[cantino]

Does anyone know why this is not returning a non-zero exit status?

[mkornblum]

We just ran into this as well, Jasmine CI task fails but still returns 0, so the build proceeds as if all is well.

[cantino]

We never figured out why this was happening. It'd be great to understand the root cause.

[jboyens]

This is likely due to the fact that we don't wrap the entire run script in a begin...rescue. I've created a feature in Tracker for this: https://www.pivotaltracker.com/story/show/54066951