capistrano-plugins/capistrano-ssh-doctor

Quickly troubleshoot capistrano issues related to SSH

capistrano-ssh-doctor's Issues

Refactor report class: extract messages out

As can be seen in messages module that is included in report class: textual messages are tightly coupled to the report itself.

A step forward would be to more decouple report and textual messages.

Please change `scm` setting to `:git`.

Running ssh:doctor, I get:

$ bundle exec cap staging ssh:doctor
It seems you are NOT using git as a Capistrano strategy. At the moment capistrano-ssh-doctor supports only git.
Please change `scm` setting to `:git`.

If I add to my deploy.rb file:

set :scm, :git

I get

lusk 16:46:32 {12408_deployLearning} ~/code/sg6$ bundle exec cap staging ssh:doctor
[Deprecation Notice] `set :scm, :git` is deprecated.
To ensure your project is compatible with future versions of Capistrano,
remove the :scm setting and instead add these lines to your Capfile after
`require "capistrano/deploy"`:

    require "capistrano/scm/git"
    install_plugin Capistrano::SCM::Git

It seems you are NOT using git as a Capistrano strategy. At the moment capistrano-ssh-doctor supports only git.
Please change `scm` setting to `:git`.

Using cap 3.9.0:

$ cap -v
Capistrano Version: 3.9.0 (Rake Version: 12.0.0)

Please change `scm` setting to `:git`.

Hi guys,

This seems to be the same as issue #9

Maybe I am missing something but I just cant not get passed this.

Setup is:
Rails Version: 5.1.6
Cap Version: 3.11.0 (Rake Version: 12.3.1)
capistrano-ssh-doctor version: 1

Capfile:

require "capistrano/setup"
require "capistrano/deploy"

require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git

require 'capistrano/rbenv'
require 'capistrano/rbenv_install'

require 'capistrano/bundler'
require 'capistrano/rails'

require 'capistrano/unicorn_nginx'
require 'capistrano/postgresql'

require 'capistrano/safe_deploy_to'
require 'capistrano/ssh_doctor'

Dir.glob("lib/capistrano/tasks/*.rake").each { |r| import r }

deploy.rb

lock "~> 3.11.0"
set :rbenv_ruby, "2.5.1"

set :application, "some_app"
set :repo_url, "[email protected]:wonderphil/some_app.git"

Gemfile

....
group :development do
  gem "capistrano", "~> 3.10"                
  gem 'capistrano-rbenv'                     
  gem 'capistrano-rbenv-install'             
  gem 'capistrano-bundler'                  
  gem 'capistrano-rails'                     
  gem 'capistrano-unicorn-nginx'             
  gem 'capistrano-postgresql'                
  gem 'capistrano-safe-deploy-to'            
  gem 'capistrano-ssh-doctor'
end
....

when I run:

$ cap production ssh:doctor
It seems you are NOT using git as a Capistrano strategy. At the moment capistrano-ssh-doctor supports only git.
Please change `scm` setting to `:git`.

If i add the scm setting i get

$ cap production ssh:doctor
[Deprecation Notice] `set :scm, :git` is deprecated.
To ensure your project is compatible with future versions of Capistrano,
remove the :scm setting and instead add these lines to your Capfile after
    `require "capistrano/deploy"`:

    require "capistrano/scm/git"
    install_plugin Capistrano::SCM::Git
It seems you are NOT using git as a Capistrano strategy. At the moment capistrano-ssh-doctor 
supports only git.
Please change `scm` setting to `:git`.

Any help?

Tool throws an error.

While trying to run the following command I get an error. I have no idea if this is something wrong my code or a bug in the function. I am running a homestead box in vagrant where I have installed ruby through apt-get. ssh [email protected] 'hostname; uptime' does work.

bundle exec cap staging ssh:doctor --trace

bundle exec cap staging ssh:doctor --trace
** Invoke staging (first_time)
** Execute staging
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke ssh:doctor (first_time)
** Execute ssh:doctor
** Invoke ssh:config:git (first_time)
** Execute ssh:config:git
** Invoke ssh:config:repo_url (first_time)
** Execute ssh:config:repo_url
** Invoke ssh:config:password (first_time)
** Execute ssh:config:password
** Invoke ssh:config:agent_forwarding (first_time)
** Execute ssh:config:agent_forwarding
** Invoke ssh:local:private_key_exists (first_time)
** Execute ssh:local:private_key_exists
** Invoke ssh:local:agent_running_env_var (first_time)
** Execute ssh:local:agent_running_env_var
** Invoke ssh:local:agent_running_ssh_add (first_time)
** Execute ssh:local:agent_running_ssh_add
4096 SHA256:RfPTmZLA/sLeUxe5Rgn9VgQoTa4fMWJyMRBWUZXFC2o /home/vagrant/.ssh/id_rsa (RSA)
** Invoke ssh:local:repo_access (first_time)
** Execute ssh:local:repo_access
31c638fe5a33d9e40a5c9eee07a94deb9cb76adc        HEAD
... Truncated all of the branches here
ef090b63ca0b772390a1db03cc7c118af20e8733        refs/tags/1.8.9
** Invoke ssh:remote:agent_running (first_time)
** Execute ssh:remote:agent_running
#<Thread:0x000055c0e6d56e50@/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/runners/parallel.rb:11 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
        8: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/runners/parallel.rb:13:in `block (2 levels) in execute'
        7: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `run'
        6: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `instance_exec'
        5: from /var/lib/gems/2.5.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:87:in `block (4 levels) in <top (required)>'
        4: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:77:in `capture'
        3: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `_execute'
        2: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `tap'
        1: from /var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:129:in `block in _execute'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/formatters/pretty.rb:8:in `write': Exception while executing on host 167.99.230.85: undefined method `<' for nil:NilClass (SSHKit::Runner::ExecuteError)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host 167.99.230.85: undefined method `<' for nil:NilClass
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/formatters/pretty.rb:8:in `write'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:129:in `block in _execute'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `tap'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `_execute'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:77:in `capture'
/var/lib/gems/2.5.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:87:in `block (4 levels) in <top (required)>'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `instance_exec'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `run'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/runners/parallel.rb:13:in `block (2 levels) in execute'
NoMethodError: undefined method `<' for nil:NilClass
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/formatters/pretty.rb:8:in `write'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:129:in `block in _execute'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `tap'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:128:in `_execute'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:77:in `capture'
/var/lib/gems/2.5.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:87:in `block (4 levels) in <top (required)>'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `instance_exec'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/backends/netssh.rb:54:in `run'
/var/lib/gems/2.5.0/gems/sshkit-1.5.1/lib/sshkit/runners/parallel.rb:13:in `block (2 levels) in execute'
Tasks: TOP => ssh:remote:agent_running

Here is my staging file

# Simple Role Syntax
# ==================
# Supports bulk-adding hosts to roles, the primary server in each group
# is considered to be the first unless any hosts have the primary
# property set.  Don't declare `role :all`, it's a meta role.

role :app, %w{[email protected]}
role :web, %w{[email protected]}
role :db,  %w{[email protected]}


# Extended Server Syntax
# ======================
# This can be used to drop a more detailed server definition into the
# server list. The second argument is a, or duck-types, Hash and is
# used to set extended properties on the server.

server '167.99.230.85', user: 'runcloud', roles: %w{web app}, my_property: :my_value


# Custom SSH Options
# ==================
# You may pass any option but keep in mind that net/ssh understands a
# limited set of options, consult[net/ssh documentation](http://net-ssh.github.io/net-ssh/classes/Net/SSH.html#method-c-start).
#
# Global options
# --------------
set :ssh_options, {
	keys: %w(~/.ssh/id_rsa),
    forward_agent: true,
    auth_methods: %w(password)
}
#
# And/or per server (overrides global)
# ------------------------------------
# server 'example.com',
#   user: 'user_name',
#   roles: %w{web app},
#   ssh_options: {
#     user: 'user_name', # overrides user setting above
#     keys: %w(/home/user_name/.ssh/id_rsa),
#     forward_agent: false,
#     auth_methods: %w(publickey password)
#     # password: 'please use keys'
#   }

And my deploy file

# config valid only for Capistrano 3.1
lock '3.2.1'

set :application, 'my_app_name'
set :repo_url, '[email protected]:roots/bedrock.git'

# Default branch is :master
# ask :branch, proc { `git rev-parse --abbrev-ref HEAD`.chomp }.call

# Default deploy_to directory is /var/www/my_app
# set :deploy_to, '/var/www/my_app'

# Default value for :scm is :git
# set :scm, :git

# Default value for :format is :pretty
# set :format, :pretty

# Default value for :log_level is :debug
# set :log_level, :debug

# Default value for :pty is false
# set :pty, true

# Default value for :linked_files is []
# set :linked_files, %w{config/database.yml}

# Default value for linked_dirs is []
# set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system}

# Default value for default_env is {}
# set :default_env, { path: "/opt/ruby/bin:$PATH" }

# Default value for keep_releases is 5
# set :keep_releases, 5

namespace :deploy do

  desc 'Restart application'
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      # Your restart mechanism here, for example:
      # execute :touch, release_path.join('tmp/restart.txt')
    end
  end

  after :publishing, :restart

  after :restart, :clear_cache do
    on roles(:web), in: :groups, limit: 3, wait: 10 do
      # Here we can do anything such as:
      # within release_path do
      #   execute :rake, 'cache:clear'
      # end
    end
  end

end

connecting to github repo from server: success for 1-9, error for 10 only

First of all, this is an awesome sanity checker - thank you tons for providing it!

I keep running into the following error when running bundle exec cap staging deploy:

Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I have a feeling this is going to end up being an issue on my end, not yours, but: have you ever seen numbers 1-9 succeed but 10 still fail, when trying to connect to a Github repo from a staging server? Here's the output when running bundle exec cap staging ssh:doctor:

SSH agent forwarding report
---------------------------
1. [success] `repo_url` setting ok
2. [success] ssh private key file exists
3. [success] `ssh-agent` process seems to be running locally
4. [success] `ssh-agent` process recognized by `ssh-add` command
5. [success] ssh private keys added to `ssh-agent`
6. [success] application repository accessible from local machine
7. [success] all hosts using passwordless login
8. [success] `forward_agent` ok for all hosts
9. [success] ssh agent successfully forwarded to remote hosts
10. [error] It seems Capistrano cannot access application git repository from these hosts:
ssh.phx.nearlyfreespeech.net
Actions:
- make sure all the previous checks pass. That should make this one work too.

This happens when using the format

set :repo_url, '[email protected]:{github-organization}/{private-repo}.git'

in my deploy.rb. When using either

set :repo_url, 'https://{github-username}:{github-password}@github.com/{github-organization}/{private-repo}.git'

or

set :repo_url, 'https://{github-oauth-token}:@github.com/{github-organization}/{private-repo}.git'

, I am able to successfully connect to the Github repo from the staging server - but those are less than ideal as (I think) they'd require putting Github credentials of some sort in the repo.

I'm admittedly not super strong re: ssh-agent, forwarding, etc. so I'm sure there are a few things I could be missing. I am encouraged to know that having checks 1-9 pass should make 10 work too, but perplexed as to what else might be causing check 10 to fail. Thanks for any insight/assistance you may be able to provide!

TypeError: no implicit conversion of Array into String

I'm getting this as soon as I run the command:

$ cap production ssh:doctor
(Backtrace restricted to imported tasks)
cap aborted!
TypeError: no implicit conversion of Array into String

Tasks: TOP => ssh:local:private_key_exists
(See full trace by running task with --trace)

Trace output:

$ cap production ssh:doctor --trace
** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke ssh:doctor (first_time)
** Execute ssh:doctor
** Invoke ssh:config:git (first_time)
** Execute ssh:config:git
** Invoke ssh:config:repo_url (first_time)
** Execute ssh:config:repo_url
** Invoke ssh:config:password (first_time)
** Execute ssh:config:password
** Invoke ssh:config:agent_forwarding (first_time)
** Execute ssh:config:agent_forwarding
** Invoke ssh:local:private_key_exists (first_time)
** Execute ssh:local:private_key_exists
cap aborted!
TypeError: no implicit conversion of Array into String
/var/lib/gems/2.3.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:53:in `exists?'
/var/lib/gems/2.3.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:53:in `block (3 levels) in <top (required)>'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/var/lib/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:16:in `invoke'
/var/lib/gems/2.3.0/gems/capistrano-ssh-doctor-1.0.0/lib/capistrano/tasks/ssh_doctor.rake:112:in `block (2 levels) in <top (required)>'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/var/lib/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/var/lib/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/application.rb:15:in `run'
/var/lib/gems/2.3.0/gems/capistrano-3.4.0/bin/cap:3:in `<top (required)>'
/usr/local/bin/cap:23:in `load'
/usr/local/bin/cap:23:in `<main>'
Tasks: TOP => ssh:local:private_key_exists

Make sure that the `keys` file path existence is tested for each server

For example, if the user specify private key like this:

server "111.112.113.114", user: "deploy", roles: %w{web app db}, keys: "/foo/bar.pem"

Although we have the private_key_exists task, with the key specified like that:

• the key path will not be tested for existence
• we're also not testing if that specific key is loaded to local ssh_agent, and also forwarded to the remote server

This was brought up by the user in #3

If the wrong ssh private key is used, and login to the server fails, running the task will also error out

When a user uses wrong private ssh key to login to the server, that fails.
It's bad that the ssh:doctor also fails on task remote:agent_running.

Any private ssh key can be specified like this:

server "111.112.113.114", user: "deploy", roles: %w{web app db}, keys: "/foo/bar.pem"

I'm not sure if the key that's specified in this way can have agent_forwarding ssh feature..

I am not receiving more information and still dont know the problem

Hi Bruno, nice job in this gem.
This what I get running this tool. I am using a .pem autentication file. But I don't even know if it is finding the file. Could you help me please?

Report should have colored output

Right now, report output is handled with plain puts.

It would be better if the output was green for success messages, and red for errors.