Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Infrataster - Infra Behavior Testing Framework #oedo04

Infrataster - Infra Behavior Testing Framework #oedo04

at 大江戸 Ruby 会議 04
https://github.com/ryotarai/infrataster

Ryota Arai

April 19, 2014
Tweet

More Decks by Ryota Arai

Other Decks in Programming

Transcript

  1. Infrataster
    ~ Infra Behavior Testing Framework ~
    Ryota Arai
    !
    େߐށ Ruby ձٞ 04 / #oedo04

    View Slide

  2. #oedo04 is a ੜ׆ൃදձ

    View Slide

  3. 1. Self Intro
    2. What I created recently

    View Slide

  4. Who I am

    View Slide

  5. Ryota Arai
    ߥҪ ྑଠ
    a.k.a. େ৔೿ͷएऀ
    !
    COOKPAD, Inc
    Infra Engineer
    Taken by @hsbt

    View Slide

  6. My First Asakusa.rb
    2012/4 I started using Ruby
    2012/11 େ৔೿ʹೖ৴
    2012/12/4 My First Asakusa.rb

    2014/4/19 Oedo RubyKaigi 04!!

    View Slide

  7. What I created

    View Slide

  8. Trending Repositories
    //github.com/trending

    View Slide

  9. I'd like to watch GH trends
    on RSS reader!

    View Slide

  10. //github-trends.ryotarai.info

    View Slide

  11. git-commit-autouser
    $ gem install git-commit-autouser
    $ cat ~/.gitconfig
    [autouser-github]
    url-regexp = github.com
    name = "Ryota Arai"
    email = [email protected]
    [autouser-company]
    url-regexp = git.example.com
    name = "Ohba-ha-no-wakamono"
    email = [email protected]
    [alias]
    ci = commit-autouser
    $ git remote -v
    origin [email protected]:…
    $ git ci -m 'Hello'
    $ git log -1
    Author: Ryota Arai

    $ git remote -v
    origin [email protected]:…
    $ git ci -m 'Hello'
    $ git log -1
    Author: Ohba-ha-no-wakamono

    http://ryotarai.github.io/git-commit-autouser/

    View Slide

  12. What did I contribute to?
    • serverspec/serverspec
    • opscode/chef
    • test-kitchen
    • kitchen-vagrant
    • etc…

    View Slide

  13. Infrataster
    tast‧er [countable]
    1. someone whose job is to test the quality of foods,
    teas, wines etc by tasting them
    (Longman Dictionary)

    View Slide

  14. What is Infrataster?
    It tests infrastructure's behavior
    from outside of servers.
    !
    What's that?

    View Slide

  15. Infrastructure as Code
    Automate building infra
    by Code
    !
    Chef, Puppet, Ansible …
    # Example: Chef
    package 'apache2' do
    action :install
    end
    !
    service 'apache2' do
    action :start
    end

    View Slide

  16. When I write how to set web-
    servers up…
    1. Write Serverspec tests.
    2. Write files like chef-cookbook.
    3. After tests passed, go back to 1.
    4. Finally, The cookbook is
    completed!
    VM
    Serverspec
    Test
    Provision

    View Slide

  17. Serverspec tests the status
    of the server.
    describe package('apache2') do
    it { should be_installed }
    end
    !
    describe service('apache2') do
    it { should be_enabled }
    it { should be_running }
    end
    !
    describe port(80) do
    it { should be_listening }
    end
    package 'apache2' do
    action :install
    end
    !
    service 'apache2' do
    action :start
    end
    Serverspec Chef

    View Slide

  18. I completed! But …
    $ sudo sh -c 'echo "192.168.33.10 web" >> /etc/hosts'
    $ curl -v http://web
    * About to connect() to 192.168.33.10 port 80 (#0)
    * Trying 192.168.33.10...
    (It takes long long time…)
    ^C
    $ telnet web 80
    Trying 192.168.33.10...
    (It takes long long time…)
    ^C

    View Slide

  19. Because of iptables
    $ sudo iptables -L
    Chain INPUT (policy DROP)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
    ACCEPT all -- anywhere anywhere
    ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
    !
    Chain FORWARD (policy DROP)
    target prot opt source destination
    !
    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination

    View Slide

  20. It works!
    $ sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport
    80 -j ACCEPT
    !
    $ curl http://web
    It works!
    This is the default web page for this server.
    The web server software is running but no content has been
    added, yet.

    View Slide

  21. How to test such case?
    Check whether the server returns 200 OK,
    when I send a HTTP request.

    View Slide

  22. How to test such case?
    Check whether the server returns 200 OK,
    when I send a HTTP request.
    describe http('http://app') do
    it "responds OK 200" do
    expect(response.code).to eq('200')
    end
    end
    This is what Infrataster is!

    View Slide

  23. Getting Started
    $ bundle install
    $ rspec —init
    # spec/spec_helper.rb
    require 'infrataster/rspec'
    # Gemfile
    gem 'infrataster'

    View Slide

  24. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    VM
    Host
    Proxy
    192.168.33.10
    *OGSB
    UBTUFS

    View Slide

  25. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    Infrataster::Server.define(
    :app, '172.16.33.11',
    vagrant: true,
    from: :proxy # access from proxy
    )
    VM
    Host
    Proxy
    192.168.33.10
    (172.16.33.10)
    App
    172.16.33.11
    *OGSB
    UBTUFS
    SSH Port Forwarding

    View Slide

  26. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    Infrataster::Server.define(
    :app, '172.16.33.11',
    vagrant: true,
    from: :proxy
    )
    Infrataster::Server.define(
    :db, '172.16.33.12',
    vagrant: true,
    from: :app, # access from app server
    mysql: {user: 'app', password: 'app'}
    )
    VM
    Host
    Proxy
    App
    172.16.33.11
    DB
    172.16.33.12
    *OGSB
    UBTUFS
    SSH Port Forwarding

    View Slide

  27. 'http' type spec
    describe server(:proxy) do # 192.168.33.10
    describe http('http://app') do
    it "responds content including 'Hello Sinatra'" do
    expect(response.body).to include('Hello Sinatra')
    end
    end
    describe http('http://static') do
    it "responds content including 'Welcome to nginx!'" do
    expect(response.body).to include('Welcome to nginx!')
    end
    end
    end

    View Slide

  28. 'capybara' type spec
    describe server(:app) do # Test from proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello Sinatra')
    end
    end
    end

    View Slide

  29. 'capybara' type spec
    Firefox
    Visit
    http://127.0.0.1:XXXXX
    App Server Listen 80
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end

    View Slide

  30. 'capybara' type spec
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end
    Firefox
    SSH
    SSH
    App Server
    Visit
    http://127.0.0.1:XXXXX
    Forward
    from 127.0.0.1:XXXX
    to app:80
    Listen 80

    View Slide

  31. 'capybara' type spec
    Firefox
    BrowserMob
    Proxy
    SSH
    SSH
    App Server
    Rewrite HTTP header
    => Host: app
    Visit
    http://127.0.0.1:XXXXX
    Forward
    from 127.0.0.1:XXXX
    to app:80
    Listen 80
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end

    View Slide

  32. 'mysql_query' type spec
    describe server(:db) do
    describe mysql_query('SHOW STATUS') do
    it 'responds uptime' do
    row = results.find {|r| r['Variable_name'] == 'Uptime' }
    expect(row['Value'].to_i).to be > 0
    end
    end
    end

    View Slide

  33. Run Specs
    $ bundle exec rspec
    Run options: include {:focus=>true}
    !
    All examples were filtered out;
    ignoring {:focus=>true}
    !
    server 'app' from 'proxy'
    capybara 'http://app'
    responds content including 'Hello
    Sinatra'
    http 'http://app'
    responds content including 'Hello
    Sinatra'
    responds OK 200
    responds as 'text/html'
    server 'proxy'
    http 'http://static'
    responds as 'text/html'
    responds content including
    'Welcome to nginx!'

    !
    Finished in 17.92 seconds
    11 examples, 0 failures
    !
    Randomized with seed 12750

    View Slide

  34. After Infrataster Test Passed
    There is no longer "iptables" problems!
    $ curl http://server_tested_with_infrataster
    It always works!

    View Slide

  35. You can see example:
    !
    //github.com/ryotarai/infrataster/
    tree/master/example

    View Slide

  36. Further development
    • Dog-fooding
    • Write unit-test specs & Refactor
    • Add types to support other protocols and
    middlewares
    • Support multi-hop SSH tunnel
    • Implement features that users need!
    Feedbacks are welcome!

    View Slide

  37. //github.com/ryotarai/
    infrataster

    View Slide