heroku/kensa

A tool to help Heroku add-on providers integrate their services with Heroku

DEPRECATION

Kensa is now deprecated in favor of the addons-admin Heroku CLI plugin.

Kensa

Kensa is a command-line utility to help Heroku add-on providers integrating their services to Heroku. It offers commands to create and validate manifests, and to run the same API calls Heroku runs on your service to provision and deprovision resources.

Setup

Install it like any Ruby Gem:

$ gem install kensa

Usage

Refer to the Heroku Add-ons Resource Center for more information on usage, and how to build your Heroku add-on:

https://devcenter.heroku.com/articles/building-a-heroku-add-on

Meta

Maintained by the Heroku Ecosystem Team (and you!).

Released under the MIT license. https://github.com/heroku/kensa

kensa's Issues

No feedback for CommandInvalid exception

Due to the ordering of these two lines, no error message is ever printed when a Client::CommandInvalid is raised; instead, the usage is just spit out with no helpful information:

https://github.com/heroku/kensa/blob/master/bin/kensa#L19-20

key not found: :ciphers (KeyError)

Testing POST /heroku/resources
Check response/Users/appleapple/.rvm/gems/ruby-2.4.0/gems/rest-client-1.8.0/lib/restclient/request.rb:163:in `fetch': key not found: :ciphers (KeyError)

this type of error is shown.

Error: undefined method 'yellow'

I ran a command similar to this with Kensa 2.4.3:

kensa run -f addon-manifest.json -p myplan sh

And got this error:

Testing config data
  Check is a hash [PASS]
  Check all config keys were previously defined in the manifest [PASS]
  Check all keys in the manifest are present/home/andrew/.gem/ruby/2.0.0/gems/kensa-2.4.3/lib/heroku/kensa/check.rb:206:in `block in call!': undefined method `yellow' for #<Heroku::Kensa::ProvisionResponseCheck:0x007f76a5f5d3b8> (NoMethodError)

When I run the same command in Kensa 2.4.2, it works fine. I am completely new to Kensa so I'm far from sure that this is a regression, but it seemed worth flagging up here.

In particular, this change looks suspicious:

87905ca#diff-f66b8de549216f35e05b5c866024b854L171

Required config var

kensa test provision fails if my addon does not define a config var. kensa push allows me to get away with this just fine and it seems to work in production.

use .netrc instead of interactive password prompting

invalid option: --filename

$ be kensa --filename addons/contrib/heroku/addon-manifest.json 
invalid option: --filename

Usage: kensa [OPTIONS] command
       kensa init
       kensa create <app_name> --template
       kensa test   <type> [arg1 arg2 ...]
       kensa run    <command> [arg1 arg1 ...]

OPTIONS

  -f, --filename path-to-file
    Sets the manifest file to operate on, default is addon-manifest.json.

Works with -f.

template location

https://github.com/heroku/kensa/blob/master/bin/kensa#L71
kensa-create-sinatra
not kensa-template-sinatra

Errors: "FAILED: Slug is invalid"

If your manifest ID has underscores, you get "FAILED: Slug is invalid" when you try to kensa push. The manifest test should probably flag if something isn't allowed.

Pushing invalid manifest provides unhelpful feedback

pvh was tried to push a manifest, which turned out to be invalid. Rather than providing a useful explanation of what went wrong it simply provided an output of the usage/help.

kensa push always asks for credentials

It looks like it just prompts every time
https://github.com/heroku/kensa/blob/master/lib/heroku/kensa/client.rb#L71
It would be nice if it could store them like the heroku client

Segmentation fault on kensa push

Using rbenv, 1.9.3-p484, on 10.8.5 osx

/usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/mechanize-1.0.0/lib/mechanize.rb:11:in `<top (required)>': iconv will be deprecated in the future, use String#encode instead.

Enter your Heroku Provider credentials.
Email: <email>
Password: 

dyld: lazy symbol binding failed: Symbol not found: _yajl_set_static_value
  Referenced from: /usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/yajl-ruby-0.8.3/ext/yajl/yajl.bundle
  Expected in: flat namespace

dyld: Symbol not found: _yajl_set_static_value
  Referenced from: /usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/yajl-ruby-0.8.3/ext/yajl/yajl.bundle
  Expected in: flat namespace

/usr/local/var/rbenv/versions/1.9.3-p484/lib/ruby/gems/1.9.1/gems/yajl-ruby-0.8.3/lib/yajl.rb:37: [BUG] Segmentation faultruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin12.5.0]

please update yajl-ruby version

Could you please update yajl-ruby's version?

• https://rubygems.org/gems/yajl-ruby

Many other gems start using v1.x.x, so I had a conflict between them and kensa.

Thanks in advance - K

Duplicate Provision Issue

The Duplicate Provision check is only run if !api_requires?("many_per_app") (if the manifest does not specify many_per_app; however, inside the actual check class, the opposite condition is found (if api_requires?("many_per_app")), which can never be true when reached. Thus, the error message "multiple provisions cannot return the same id" will never occur.

In any case, the Heroku apps are unclear on how an add-on should respond to a duplicate provision request. Wouldn't Heroku prevent provisioning the same add-on multiple times if the add-on does not allow many_per_app? I was planning to directly use the UUID for the returned Id since we do not allow many_per_app, yet kensa tests the same provision data.

tests not matching docs: syslog_drain_url

Tests look for: response["syslog_drain_url"]

in kensa test: Check syslog_drain_url is returned if required

but docs say expected response is 'log_drain_url' on:

https://devcenter.heroku.com/articles/accessing-app-logs

SSO does not send required 'app' parameter

The API specification lists app as a passed parameter to the SSO request: https://addons.heroku.com/provider/resources/technical/reference/api-spec

Kensa SSO does not send the app parameter.

kensa test fails due to rest-client-1.8.0???

I'm not a ruby expert at all. When I run kensa test I get this:

Tried to install a newer version of rest-client, now when I gem list I see:

So I try to get rid of the old version using gem uninstall rest-client --version 1.8.0, but then I get this scary message:

Please help me and tell me what to do in order to get kensa to test and push properly. I really don't care which gem version it's using, just need it to work.

Can't upgrade rest-client with security issues because Kensa is locked into 1.6.x

Issues:

• http://www.osvdb.org/show/osvdb/117461
• rest-client/rest-client#369

Learned about from bundler-audit

Name: rest-client
Version: 1.6.9
Advisory: CVE-2015-1820
Criticality: Unknown
URL: https://github.com/rest-client/rest-client/issues/369
Title: rubygem-rest-client: session fixation vulnerability via Set-Cookie headers in 30x redirection responses
Solution: upgrade to >= 1.8.0

Name: rest-client
Version: 1.6.9
Advisory: OSVDB-117461
Criticality: Unknown
URL: http://www.osvdb.org/show/osvdb/117461
Title: Rest-Client Gem for Ruby logs password information in plaintext
Solution: upgrade to >= 1.7.3

Not clear on how to update an existing addon manifest

I am trying to use kensa push to update manifest for my addon and always got error:

% kensa push
FAILED: Can't revise stale manifest; please base your changes on the latest manifest

It's pretty clueless and cost me long time found for update it need a "$base" field to identify what's the update based on. I hope there are at least some document there guiding me.

-- wpc

kensa run all fails with return code 32512

I'm using kensa for my integration tests for the Heroku Add-on Partner API, and it's failing with a non-zero status code when running kensa run all. The output from the command is:

Starting all...

 [FAIL]
    run exited abnormally, expected 0, got 32512

I am provisioning the resource synchronously and using the documentation described here.

The error appears to be happening here.

Any ideas?

Kensa doesn't send several documented fields during provision check

Kensa doesn't send the "log_input_url" or the "uuid" during the provision check which is documented here: https://devcenter.heroku.com/articles/add-on-provider-api#provision

The documentation lists several deprecated fields however it lists the uuid as the only reliably unique field.

Kensa sso HTML validation fails

A check in Kensa fails with:

Check displays the heroku layout [FAIL] ! could not find Heroku layout

When returned document follows this spec: https://addons.heroku.com/provider/resources/technical/how/sso/4

It seems that a lookup for 'div#heroku-header' should be updated to search for a JS inclusion.

`kensa test all` does not run all tests

We recently had a brief outage with our add-on. Our engineers ran "kensa test all", which they just discovered doesn't include "kensa test sso". It would be great if "all" included single sign on.

provision test lacks OAuth grant

Referencing https://devcenter.heroku.com/articles/unified-api#receiving-the-grant-code

But my local addon only receives the following parameters

%{
  "callback_url" => "http://localhost:7779/callback/999",
  "heroku_id" => "[email protected]",
  "logplex_token" => nil,
  "options" => %{},
  "plan" => "test",
  "region" => "amazon-web-services::us-east-1",
  "uuid" => "057ef4ee-967e-46c8-a3c0-aaf3c9a37c82"
}

Especially testing OAuth with kensa would be great.

Thanks ✨

Check displays the heroku layout

Fails when the new layout system is implemented:
https://github.com/heroku/boomerang

kensa push is not working

kensa push
Found credentials [email protected], proceed? (y/N) y
Not authorized to push this manifest. Please make sure you have permissions to push it

how to solve it this problem pls help for me!!

Kensa dependency on rest-client ~>1.2.0 is causing conflicts in my Gemfile

Hey all,

First off, Kensa rocks. And I like it so much that I want it to be part of my CI. However, to run it in my rake tasks, I need kensa to be listed in my Gemfile.

Kensa depends on rest-client ~>1.2.0 — However, I have another gem which depends on rest-client ~> 1.3 (and the current version is 1.6.1 or somesuch). This is causing conflicts in my Gemfile.

Bundler could not find compatible versions for gem "rest-client":
  In Gemfile:
    kensa depends on
      rest-client (~> 1.2.0)

    rest-client (1.6.1)

Is the dependency on 1.2 intentional, or can that be loosened up a bit?

—Nick

kensa test provision doesn't send json params

So if I try and parse the body, it blows up. kensa test provision should send a representative body

Authentication failure when Two-factor Authentication is on

kensa push failed with Authentication failure until I turned off Two-factor Authentication in my app.

(It did ask for username and password before failing)

kensa sso should allow extra GET parameters to be set

https://devcenter.heroku.com/articles/how-addons-work#logging-users-in-via-sso

We pass a redirect parameter through the Heroku SSO when emailing links to our customers - I'd like to be able to test this functionality with Kensa.

Install fails on 1.8.7

glenngillen@Nibbler ~/Development/heroku/glenn-provider (master)$ ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-darwin11.2.0]
glenngillen@Nibbler ~/Development/heroku/glenn-provider (master)$ bundle install kensa
Fetching source index for http://rubygems.org/
Using archive-tar-minitar (0.5.2) 
Using columnize (0.3.4) 
Using haml (3.1.2) 
Using json (1.5.3) 
Using mime-types (1.16) 
Using rest-client (1.6.3) 
Using heroku-nav (0.1.22) 
Using ruby_core_source (0.1.5) 
Installing linecache19 (0.5.12) 
Gem::InstallError: linecache19 requires Ruby version >= 1.9.2.
An error occured while installing linecache19 (0.5.12), and Bundler cannot continue.
Make sure that `gem install linecache19 -v '0.5.12'` succeeds before bundling.

Kensa test updates heroku_id, tries to delete the old heroku_id

I'm running kensa test

1. It creates a resource with heroku_id app1@kensa..
2. It updates the resource app1@kensa to app2@kensa
3. It tires deleting app1@kensa to which it wants a 200 OK response however I cannot find app1@kensa anymore since it got updated

Am I doing something wrong, should I not update the heroku_id?

Add ability to pass in a region to Provision test

I'm testing my add-on accepts provision requests correctly for multiple regions but I can't find a way to pass the different region codes in.

something like

kensa test provision --region eu

Manifest test does not run with 'test all'

From documentation:

  manifest
    Confirm that the manifest is valid.  Automatically runs before all tests.

It does not appear to run before test all however.

Kensa doesn't work in Ruby 2.0.0

iconv was deprecated and now causing error if we try to use kensa in Ruby >2.0

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can image, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

Did the callback_url format change?

Sometime in the last three weeks, it looks like the callback_url format for the provisioning process changed from:

https://api.heroku.com/vendor/apps/572458

To the following format:

https://api.heroku.com/vendor/apps/app1308714%40heroku.com

Was this change on purpose? It definitely broke part of my implementation that relies on the callback URL, but luckily I saw the exception. :-)

PlanChangeCheck doesn't match add-on API spec for plan change

This is albeit a tiny detail, but my implementation was relying on it and was broken as a result:

In the Plan Change section of the add-on API Specification, the listed Input JSON is the following:

Action :  PUT /heroku/resources/:id
Input  :  json { "heroku_id": "[email protected]", "plan": "premium" }

However, when running kensa test planchange 123, I noticed that the "heroku_id" parameter isn't actually passed:
https://github.com/heroku/kensa/blob/1.1.4/lib/heroku/kensa/check.rb#L356

Should the add-on API spec be updated to match the kensa implementation, or vice-versa?

kensa test sso X fails if JSON is returned

If you accidentally return json you will receive the error, even if the status is good/accepted

Check displays the heroku layout/Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/check.rb:457:in `block in call!': undefined method `search' for #<Mechanize::File:0x007fa208b999c8> (NoMethodError)
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/check.rb:28:in `check'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/check.rb:418:in `check'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/check.rb:456:in `call!'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/check.rb:45:in `call'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/client.rb:168:in `block in run_check'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/client.rb:165:in `each'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/client.rb:165:in `run_check'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/client.rb:64:in `test'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/lib/heroku/kensa/client.rb:21:in `run!'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/gems/kensa-1.4.1/bin/kensa:17:in `<top (required)>'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/bin/kensa:23:in `load'
    from /Users/Gingitsune/.rvm/gems/ruby-1.9.3-rc1/bin/kensa:23:in `<main>'

This can be reproduced like this:

  #GET /heroku/resources/:id
  #This is a Heroku SSO, check creds then redirect
  def sso_login
    pass, message, status = sso_auth? #check credentials and return: bool, string, [200,401]

    # render :text => message, :status => status, :layout => true
    render :json => message, :status => status #blows up `kensa test sso 1`
  end

kensa test provision - Question about "Check all keys in the manifest are present" test

What is the rational behind the "Check all keys in the manifest are present" test when running kensa test provision? Defined here

e.g.

Check all keys in the manifest are present [FAIL]
    BLAH_URL is missing from the manifest

If I understand this correctly, this checks that any config vars being set by the add-on are returned in the initial response from the add-on provision call.

My problem is that when our add-on is provisioned, we need to set our config var to the app's heroku domain. (e.g. 123xyz.herokuapp.com). Since the add-on provider api does not enable this call until after the add-on is provisioned (makes sense), we really don't want to return any value for our config var as it might break something in our user's app, should they have already set up the config for our add-on. The way we are currently doing this is to set the config var on a heroku/resources update call. But, this causes our kensa provision tests to fail.

So, my question is: why does kensa seem to require that our add-on return values for the config vars we need to set up on the provision call?

Any feedback on this situation?

`kensa run` fails for addons without plan changes

Per https://addons.heroku.com/provider/resources/technical/build/provisioning:

If your add-on offers multiple plans, you should also implement support for instant plan changes.

But kensa run fails if plan change requests are not supported.

kensa push gives 500 Internal Server Error (RestClient::InternalServerError)

$ ruby -v
ruby 2.0.0p195 (2013-05-14 revision 40734) [x86_64-darwin11.4.2]

$ kensa push
Enter your Heroku Provider credentials.
Email: xxxxx
Password: 
/path/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!': 500 Internal Server Error (RestClient::InternalServerError)
  from /path/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result'
  from /path/gems/rest-client-1.6.7/lib/restclient/request.rb:178:in `block in transmit'
  from /path/lib/ruby/2.0.0/net/http.rb:852:in `start'
  from /path/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit'
  from /path/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
  from /path/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
  from /path/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post'
  from /path/gems/kensa-1.4.6/lib/heroku/kensa/client.rb:93:in `push'
  from /path/gems/kensa-1.4.6/lib/heroku/kensa/client.rb:21:in `run!'
  from /path/gems/kensa-1.4.6/bin/kensa:17:in `<top (required)>'
  from /path/bin/kensa:23:in `load'
  from /path/bin/kensa:23:in `<main>'
  from /path/bin/ruby_noexec_wrapper:14:in `eval'
  from /path/bin/ruby_noexec_wrapper:14:in `<main>'

Wrong password on `kensa push` shows usage

Should show "bad credentials" or similar.

Doesn't enforce SSL for production URLs

Both the base and SSO URLs should mandate https before being able to be pushed.

Missing Sydney region

I'm looking at https://github.com/heroku/kensa/blob/master/lib/heroku/kensa/manifest.rb#L6 and it doesn't show sydney as a valid region. But according to https://devcenter.heroku.com/articles/add-on-manifest#api-regions it should be a valid region.

Am I missing something?

Thank you,

Test URLs in addon manifest failing because of self-signed certificate

kensa test should not fail, when the test urls are ssl and use self-signed certificates. This wasn't the case with an older version of Kensa, but the latest one starts failing:

Testing POST /api/heroku/resources
  Check response/var/lib/gems/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:445:in `rescue in transmit': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (RestClient::SSLCertificateNotVerified)
    from /var/lib/gems/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:350:in `transmit'
    from /var/lib/gems/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in `execute'
    from /var/lib/gems/2.1.0/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in `execute'
    from /var/lib/gems/2.1.0/gems/rest-client-1.8.0/lib/restclient/resource.rb:67:in `post'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/http.rb:39:in `request'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/http.rb:13:in `post'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:369:in `block in call!'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:29:in `check'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:355:in `call!'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:61:in `block in to_proc'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:38:in `instance_eval'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:38:in `run'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:573:in `call!'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/check.rb:50:in `call'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/client.rb:159:in `block in run_check'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/client.rb:156:in `each'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/client.rb:156:in `run_check'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/client.rb:54:in `test'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/lib/heroku/kensa/client.rb:22:in `run!'
    from /var/lib/gems/2.1.0/gems/kensa-3.0.0/bin/kensa:17:in `<top (required)>'
    from /usr/local/bin/kensa:23:in `load'
    from /usr/local/bin/kensa:23:in `<main>'

Kensa doesn't send syslog_token

According to the documentation here: https://devcenter.heroku.com/articles/accessing-app-logs

..adding a require on 'syslog_drain' to your manifest should send a syslog_token in the provisioning request.

As of kensa version 1.4.6, I don't see this in the payload. Am I missing something?

'kensa test sso X' blows up on 'Check displays the heroku layout'

tested in 1.8.7, 1.9.2 and 1.9.3 and always get this stack trace

Check displays the heroku layout/Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:455:in `block in call!': undefined method `search' for #<Mechanize::File:0x007ff80c013ef0> (NoMethodError)
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:28:in `check'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:416:in `check'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:454:in `call!'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:45:in `call'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:168:in `block in run_check'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:165:in `each'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:165:in `run_check'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:64:in `test'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:21:in `run!'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/gems/kensa-1.4.0/bin/kensa:17:in `<top (required)>'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/bin/kensa:19:in `load'
    from /Users/scott/.rvm/gems/ruby-1.9.2-p290/bin/kensa:19:in `<main>'

kensa test provision explodes if JSON is not returned

I returned "foo" from the provision call:

$ kensa test provision
/Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/1.9.1/rubygems/custom_require.rb:55:in `require': iconv will be deprecated in the future, use String#encode instead.


Testing manifest id key
  Check if exists [PASS]
  Check is a string [PASS]
  Check is not blank [PASS]

Testing manifest api key
  Check if exists [PASS]
  Check is a hash [PASS]
  Check contains password [PASS]
  Check contains test url [PASS]
  Check contains production url [PASS]
  Check production url uses SSL [PASS]
  Check sso url uses SSL [PASS]
  Check contains config_vars array [PASS]
  Check containst at least one config var [PASS]
  Check all config vars are uppercase strings [PASS]
  Check all config vars are prefixed with the addon id [PASS]
  Check deprecated fields [PASS]

done.


Testing POST /heroku/resources
  Check response [PASS]
  Check valid JSON/Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/okjson.rb:180:in `lex': undefined method `length' for nil:NilClass (NoMethodError)
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/okjson.rb:41:in `decode'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:301:in `block in call!'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:28:in `check'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:299:in `call!'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/check.rb:45:in `call'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:168:in `block in run_check'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:165:in `each'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:165:in `run_check'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:54:in `test'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/lib/heroku/kensa/client.rb:21:in `run!'
  from /Users/david/.rbenv/versions/1.9.3-p125/lib/ruby/gems/1.9.1/gems/kensa-1.4.0/bin/kensa:17:in `<top (required)>'
  from /Users/david/.rbenv/versions/1.9.3-p125/bin/kensa:19:in `load'
  from /Users/david/.rbenv/versions/1.9.3-p125/bin/kensa:19:in `<main>'