Giter Site home page Giter Site logo

infrataster's Introduction

Infrataster

![Gitter](https://badges.gitter.im/Join Chat.svg)

Gem Version Code Climate

Infrastructure Behavior Testing Framework.

Basic Usage with Vagrant

First, create Gemfile:

source 'https://rubygems.org'

gem 'infrataster'

Install gems:

$ bundle install

Install Vagrant: Official Docs

Create Vagrantfile:

# Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.box = "hashicorp/precise64"

  config.vm.define :proxy do |c|
    c.vm.network "private_network", ip: "192.168.33.10"
    c.vm.network "private_network", ip: "172.16.33.10", virtualbox__intnet: "infrataster-example"
  end

  config.vm.define :app do |c|
    c.vm.network "private_network", ip: "172.16.33.11", virtualbox__intnet: "infrataster-example"
  end
end

Start VMs:

$ vagrant up

Initialize rspec directory:

$ rspec --init
  create   spec/spec_helper.rb
  create   .rspec

require 'infrataster/rspec' and define target servers for testing in spec/spec_helper.rb:

# spec/spec_helper.rb
require 'infrataster/rspec'

Infrataster::Server.define(
  :proxy,           # name
  '192.168.0.0/16', # ip address
  vagrant: true     # for vagrant VM
)
Infrataster::Server.define(
  :app,             # name
  '172.16.0.0/16',  # ip address
  vagrant: true,    # for vagrant VM
  from: :proxy      # access to this machine via SSH port forwarding from proxy
)

# Code generated by `rspec --init` is following...

Or

# spec/spec_helper.rb
require 'infrataster/rspec'

Infrataster::Server.define(:proxy) do |server|
    server.address = '192.168.0.0/16'
    server.vagrant = true
end
Infrataster::Server.define(:app) do |server|
    server.address = '172.16.0.0/16'
    server.vagrant = true
    server.from = :proxy
end

# Code generated by `rspec --init` is following...

Then, you can write spec files:

# spec/example_spec.rb
require 'spec_helper'

describe server(:app) do
  describe http('http://app') do
    it "responds content including 'Hello Sinatra'" do
      expect(response.body).to include('Hello Sinatra')
    end
    it "responds as 'text/html'" do
      expect(response.headers['content-type']).to eq("text/html")
    end
  end
end

Run tests:

$ bundle exec rspec
2 examples, 2 failures

Currently, the tests failed because the VM doesn't respond to HTTP request.

It's time to write provisioning instruction like Chef's cookbooks or Puppet's manifests!

Server

"Server" is a server you tests. This supports Vagrant, which is very useful to run servers for testing. Of course, you can test real servers.

You should define servers in spec_helper.rb like the following:

Infrataster::Server.define(
  # Name of the server, this will be used in the spec files.
  :proxy,
  # IP address of the server
  '192.168.0.0/16',
  # If the server is provided by vagrant and this option is true,
  # SSH configuration to connect to this server is got from `vagrant ssh-config` command automatically.
  vagrant: true,
)

Infrataster::Server.define(
  # Name of the server, this will be used in the spec files.
  :app,
  # IP address of the server
  '172.16.0.0/16',
  # If the server is provided by vagrant and this option is true,
  # SSH configuration to connect to this server is got from `vagrant ssh-config` command automatically.
  vagrant: true,
  # Which gateway is used to connect to this server by SSH port forwarding?
  from: :proxy,
  # options for resources
  mysql: {user: 'app', password: 'app'},
)

You can specify SSH configuration manually too:

Infrataster::Server.define(
  # ...
  ssh: {host_name: 'hostname', user: 'testuser', keys: ['/path/to/id_rsa']}
)

fuzzy IP address

Infrataster has "fuzzy IP address" feature. You can pass IP address which has netmask (= CIDR) to Infrataster::Server#define. This needs vagrant option or ssh option which has host_name because this fetches all IP address via SSH and find the matching one.

Infrataster::Server.define(
  :app,
  # find IP address matching 172.16.0.0 ~ 172.16.255.255
  '172.16.0.0/16',
)

Of course, you can set fully-specified IP address too.

Infrataster::Server.define(
  :app,
  '172.16.11.22',
  # or
  '172.16.11.22/32',
)

#ssh_exec

You can execute a command on the server like the following:

describe server(:proxy) do
  let(:time) { Time.now }
  before do
    current_server.ssh_exec "echo 'Hello' > /tmp/test-#{time.to_i}"
  end
  it "executes a command on the current server" do
    result = current_server.ssh_exec("cat /tmp/test-#{time.to_i}")
    expect(result.chomp).to eq('Hello')
  end
end

This is useful to test cases which depends on the status of the server.

Resources

"Resource" is what you test by Infrataster. For instance, the following code describes http resource.

describe server(:app) do
  describe http('http://example.com') do
    it "responds content including 'Hello Sinatra'" do
      expect(response.body).to include('Hello Sinatra')
    end
  end
end

http resource

http resource tests HTTP response when sending HTTP request. It accepts method, params and header as options.

describe server(:app) do
  describe http(
    'http://app.example.com',
    method: :post,
    params: {'foo' => 'bar'},
    headers: {'USER' => 'VALUE'}
  ) do
    it "responds with content including 'app'" do
      expect(response.body).to include('app')

      # `response` is a instance of `Faraday::Response`
      # See: https://github.com/lostisland/faraday/blob/master/lib/faraday/response.rb
    end
  end

  # Gzip support
  describe http('http://app.example.com/gzipped') do
    it "responds with content deflated by gzip" do
      expect(response.headers['content-encoding']).to eq('gzip')
    end
  end

  describe http('http://app.example.com/gzipped', inflate_gzip: true) do
    it "responds with content inflated automatically" do
      expect(response.headers['content-encoding']).to be_nil
      expect(response.body).to eq('plain text')
    end
  end

  # Redirects
  describe http('http://app.example.com/redirect', follow_redirects: true) do
    it "follows redirects" do
      expect(response.status).to eq(200)
    end
  end

  # Custom Faraday middleware
  describe http('http://app.example.com', faraday_middlewares: [
    YourMiddleware,
    [YourMiddleware, options]
  ]) do
    it "uses the middlewares" do
      expect(response.status).to eq(200)
    end
  end
end

capybara resource

capybara resource tests your web application by simulating real user's interaction.

describe server(:app) do
  describe capybara('http://app.example.com') do
    it 'shows food list' do
      visit '/'
      click_link 'Foods'
      expect(page).to have_content 'Yummy Soup'
    end
  end
end

mysql_query resource

mysql_query resource is now in infrataster-plugin-mysql.

pgsql_query resource

pgsql_query resource sends a query to PostgreSQL server.

pgsql_query is provided by infrataster-plugin-pgsql by @SnehaM.

dns resource

dns resource sends a query to DNS server.

dns is provided by infrataster-plugin-dns by @otahi.

memcached resource

memcached resource sends a query to memcached server.

memcached is provided by infrataster-plugin-memecached by @rahulkhengare.

redis resource

redis resource sends a query to redis server.

redis is provided by infrataster-plugin-redis by @rahulkhengare.

firewall resource

firewall resource tests your firewalls.

firewall is provided by infrataster-plugin-firewall by @otahi.

Example

Tests

Unit Tests

Unit tests are under spec/unit directory.

$ bundle exec rake spec:unit

Integration Tests

Integration tests are under spec/integration directory.

$ bundle exec rake spec:integration:prepare
$ bundle exec rake spec:integration

Presentations and Articles

Introducing Infrataster

Changelog

Changelog

Contributing

  1. Fork it ( http://github.com/ryotarai/infrataster/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

infrataster's People

Contributors

aibou avatar ainoya avatar englishm avatar gitter-badger avatar gongo avatar kitaitimakoto avatar otahi avatar rrreeeyyy avatar ryotarai avatar tylerjl avatar yui-knk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

infrataster's Issues

Fix warning: The `:example_group` key in an example group's metadata hash is deprecated

With RSpec 3, deprecation warning is shown.

The `:example_group` key in an example group's metadata hash is deprecated. Use the example group's hash directly for the computed keys and `:parent_example_group` to access the parent example group metadata instead. Called from /Users/ryota/src/github.com/ryotarai/infrataster/lib/infrataster/contexts.rb:11:in `yield'.
The `:example_group` key in an example group's metadata hash is deprecated. Use the example group's hash directly for the computed keys and `:parent_example_group` to access the parent example group metadata instead. Called from /Users/ryota/src/github.com/ryotarai/infrataster/lib/infrataster/contexts.rb:35:in `yield'.

tcp_port resource

describe server(:app) do
  describe tcp_port(80) do
    it { should be_opened }
  end
end

Postgresql plugin for infrataster

Currently I am working on testing database servers with infrataster.
So I am planning to develop postgresql plugin for infrataster as current version of infrataster does not include it.
I am thinking of using pg rubygem for developing this plugin.
Will it be a good approach?
I think this plugin will also be useful for other users of infrataster.
What is your opinion about it?

Define servers with fuzzy addresses.

Server.define(
  :app,
  '192.168.0.0/16',
  vagrant: true
)

This collects IP addresses of the server via ip addr and determine the ip address.
This can be used for vagrant server or the server which has host option of ssh option.

Support mcollective communications

Any objections to adding mcollective support so that things can be ran over mcollective instead of ssh?

I may have time to work on a pull request for it soon, but figured I would ask.

Support for Test Kitchen

It would be really great if we could integrate infrataster with our existing test-kitchen-based tests.
We often use Test Kitchen with the kitchen-digitalocean driver so that we can launch tests in the cloud from a VM. Being able to run infrataster against that same instance would allow us to use infrataster as part of our existing CI system. Currently, I am only able to run infrataster tests locally where I have control over the IP address assigned to a VM (e.g. w/ Vagrant).

Thoughts?

How do you specify the port for ssh servers

I have a server which runs sshd which is not running on the standard port. My question is, how do I specify this in my spec_helper.rb ?

I tried:

require 'rubygems'
require 'infrataster/rspec'
require 'rspec'

Infrataster::Server.define(
     :hosted,
     '192.168.1.8',
     vagrant: false,
     ssh: {user: 'lca', keys: [File.expand_path('~/.id_rsa.pub'), port: 55001]},
     https: {host: 'https://myhosed.com'})

RSpec.configure do |c|
  #c.formatter = :documentation 
  c.deprecation_stream = '/dev/null'
end

but got this error:

  1) server 'hosted' http 'http://myhosed.com' with {:params=>{}, :method=>:get, :headers=>{}} Check ssh exec
     Failure/Error: current_server.ssh_exec "echo 'Hello once' > ~/testing.txt"
     TypeError:
       no implicit conversion of Hash into String
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh/authentication/key_manager.rb:65:in `expand_path'
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh/authentication/key_manager.rb:65:in `add'
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh/authentication/session.rb:61:in `block in authenticate'
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh/authentication/session.rb:61:in `each'
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh/authentication/session.rb:61:in `authenticate'
     # /usr/local/share/gems/gems/net-ssh-2.9.1/lib/net/ssh.rb:206:in `start'
     # /home/lowks/.gem/ruby/gems/infrataster-0.2.0/lib/infrataster/server.rb:81:in `ssh'
     # /home/lowks/.gem/ruby/gems/infrataster-0.2.0/lib/infrataster/server.rb:88:in `ssh_exec'
     # ./spec/hosted_spec.rb:9:in `block (3 levels) in <top (required)>'

2 deprecations logged to /dev/null

I am guessing this has something to do with my port definition ?

http resource - no domain

What if I don't have an associated domain? I just want to hit the IP address of the vagrant VM? What should the parameter be for http?

memcached plugin for infrataster

Currently I am working on testing database servers in combination with memcached servers using infrataster.

Memcached:
Memcached is an in-memory key-value store for small chunks of arbitrary data.
For more details about memcached refer,
http://memcached.org/

I am planning to develop memcached plugin for infrataster. This plugin used to query memcached server to check the statistics of server, set and get the values from server.

This plugin will useful for other users also.
Please share your thoughts.

Execution with rake spec : rake aborted!

While executing test cases with rake spec, it shows following error:

[root@localhost infrataster]# rake spec
rake aborted!
Don't know how to build task 'spec:integration'

Tasks: TOP => spec
(See full trace by running task with --trace)

Make plugable by rubygems.

  • By plugin system, users can create and use resources.
  • If all resources are included in infrataster, infrataster's rubygems dependency will be big. (== It takes long time to install)

infrataster-init command

For easy setup.
infrataster-init creates Vagrantfile, Rakefile, Gemfile, spec_helper.rb and sample spec file.

Define servers with block

Server.define(:app) do |server|
  server.address '192.168.10.10'
  server.vagrant
  server.from :proxy
  server.mysql({user: 'user', password: 'password'})
end

Testing a HTTPS resource results in NoMethodError exception

(English speakers scroll down for solution)

Infrataster version: 0.32
Ruby version: 2.4.1p111

specファイルでhttpsリソースを指定してinfratasterを実行するとNoMethodErrorが発生します。

spec_helper.rbのinfrataster向けの内容:

require 'infrataster/rspec'

Infrataster::Server.define(
    :reddit,
    'www.reddit.com'
)

specダイレクトリーでの対応するspecファイル内容:

require 'spec_helper'

describe server(:reddit) do
    describe http('https://www.reddit.com') do
        it "responds content including 'Want to join'" do
            expect(response.body).to include('Want to join')
        end
    end
end

他のサイトにもやってみたんですけど、どうやらhttpsだとこのexceptionは発生します。httpだけだと無事で実行します。

これを再現できますか?または、上記の内容は何かが間違ってるでしょうか?

よろしくお願いします。

https resource for infrataster

Currently I am testing HTTPS response when sending HTTPS request.
So I am planning to add https resource in infrataster.
I can use rubygem faraday (same as http resource) for developing this resource.
And the authentication options like client certification, client key etc can be passed to farday ssl options.
I think this resource will be useful for other users of infrataster.
What is your opinion about it?

RSpec deprecation warning

$ rspec
...

Deprecation Warnings:

--------------------------------------------------------------------------------
RSpec::Core::ExampleGroup#example is deprecated and will be removed
in RSpec 3. There are a few options for what you can use instead:

  - rspec-core's DSL methods (`it`, `before`, `after`, `let`, `subject`, etc)
    now yield the example as a block argument, and that is the recommended
    way to access the current example from those contexts.
  - The current example is now exposed via `RSpec.current_example`,
    which is accessible from any context.
  - If you can't update the code at this call site (e.g. because it is in
    an extension gem), you can use this snippet to continue making this
    method available in RSpec 2.99 and RSpec 3:

      RSpec.configure do |c|
        c.expose_current_running_example_as :example
      end

(Called from /Users/english/git/projectname/vendor/bundle/ruby/2.0.0/gems/infrataster-0.1.13/lib/infrataster/rspec.rb:13:in `block (2 levels) in <top (required)>')
--------------------------------------------------------------------------------


If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.

1 deprecation warning total

Finished in 7.37 seconds
3 examples, 0 failures

Ruby version:

ruby --version
ruby 2.0.0p481 (2014-05-08 revision 45883) [x86_64-darwin13.2.0]

Gemfile.lock:

GEM
  remote: https://rubygems.org/
  specs:
    capybara (2.4.1)
      mime-types (>= 1.16)
      nokogiri (>= 1.3.3)
      rack (>= 1.0.0)
      rack-test (>= 0.5.4)
      xpath (~> 2.0)
    cliver (0.3.2)
    diff-lcs (1.2.5)
    faraday (0.9.0)
      multipart-post (>= 1.2, < 3)
    infrataster (0.1.13)
      capybara
      faraday
      net-ssh
      net-ssh-gateway
      poltergeist
      rspec (~> 2.0)
      thor
    mime-types (2.3)
    mini_portile (0.6.0)
    multi_json (1.10.1)
    multipart-post (2.0.0)
    net-ssh (2.9.1)
    net-ssh-gateway (1.2.0)
      net-ssh (>= 2.6.5)
    nokogiri (1.6.3.1)
      mini_portile (= 0.6.0)
    poltergeist (1.5.1)
      capybara (~> 2.1)
      cliver (~> 0.3.1)
      multi_json (~> 1.0)
      websocket-driver (>= 0.2.0)
    rack (1.5.2)
    rack-test (0.6.2)
      rack (>= 1.0)
    rspec (2.99.0)
      rspec-core (~> 2.99.0)
      rspec-expectations (~> 2.99.0)
      rspec-mocks (~> 2.99.0)
    rspec-core (2.99.1)
    rspec-expectations (2.99.2)
      diff-lcs (>= 1.1.3, < 2.0)
    rspec-mocks (2.99.2)
    thor (0.19.1)
    websocket-driver (0.3.4)
    xpath (2.0.0)
      nokogiri (~> 1.3)

PLATFORMS
  ruby

DEPENDENCIES
  infrataster

Integration test.

  • Add integration test like example directory.
  • Run integration test on CI server.
  • Jenkins on DigitalOcean

Working with SSL and Virtualhosts

I'm trying to test my apaches' behavior.
It has few virtualhosts: localhost (for mod-status), site1 (http, https), site2 (http, https).

If had defined to servers:

# Ugly way to get non-loopback ip
ip = Socket.ip_address_list.find { |x| x.ip_address.include? '172.17' }

Infrataster::Server.define(:local, '127.0.0.1')
Infrataster::Server.define(:public, ip.ip_address)

Then I used the following specs:

# frozen_string_literal: true
require_relative 'spec_helper'

describe server(:public) do
  describe http('http://site1.com/any-url/') do
    it 'responses with redirect to another site' do
      expect(response.headers['Location']).to eq('https://another-site.com/any-url/')
    end

    it 'response code is 301' do
      expect(response.status).to eq(301)
    end
  end

  describe http('https://site1.com/any-url/') do
    it 'responses with redirect to another site' do
      expect(response.headers['Location']).to eq('https://another-site.com/any-url/')
    end

    it 'response code is 301' do
      expect(response.status).to eq(301)
    end
  end
end

But I got:

Failure/Error: expect(response.status).to eq(301)
Faraday::SSLError:
hostname "172.17.0.2" does not match the server certificate

Obviously, I can disable ssl validation, but may be there is another way I can access site correctly?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.