Infrataster - Infra Behavior Testing Framework #oedo04

at 大江戸 Ruby 会議 04
https://github.com/ryotarai/infrataster

Transcript

1. Infrataster
   ~ Infra Behavior Testing Framework ~
   Ryota Arai
   !
   େߐށ Ruby ձٞ 04 / #oedo04
   

   View Slide

2. #oedo04 is a ੜ׆ൃදձ
   

   View Slide

3. 1. Self Intro
   2. What I created recently
   

   View Slide

4. Who I am
   

   View Slide

5. Ryota Arai
   ߥҪ ྑଠ
   a.k.a. େ৔೿ͷएऀ
   !
   COOKPAD, Inc
   Infra Engineer
   Taken by @hsbt
   

   View Slide

6. My First Asakusa.rb
   2012/4 I started using Ruby
   2012/11 େ৔೿ʹೖ৴
   2012/12/4 My First Asakusa.rb
   …
   2014/4/19 Oedo RubyKaigi 04!!
   

   View Slide

7. What I created
   

   View Slide

8. Trending Repositories
   //github.com/trending
   

   View Slide

9. I'd like to watch GH trends
   on RSS reader!
   

   View Slide

10. //github-trends.ryotarai.info
    

    View Slide

11. git-commit-autouser
    $ gem install git-commit-autouser
    $ cat ~/.gitconfig
    [autouser-github]
    url-regexp = github.com
    name = "Ryota Arai"
    email = ryota.arai@gmail.com
    [autouser-company]
    url-regexp = git.example.com
    name = "Ohba-ha-no-wakamono"
    email = ryota.arai@example.com
    [alias]
    ci = commit-autouser
    $ git remote -v
    origin git@github.com:…
    $ git ci -m 'Hello'
    $ git log -1
    Author: Ryota Arai
    
    $ git remote -v
    origin git@git.example.com:…
    $ git ci -m 'Hello'
    $ git log -1
    Author: Ohba-ha-no-wakamono
    
    http://ryotarai.github.io/git-commit-autouser/
    

    View Slide

12. What did I contribute to?
    • serverspec/serverspec
    • opscode/chef
    • test-kitchen
    • kitchen-vagrant
    • etc…
    

    View Slide

13. Infrataster
    tast‧er [countable]
    1. someone whose job is to test the quality of foods,
    teas, wines etc by tasting them
    (Longman Dictionary)
    

    View Slide

14. What is Infrataster?
    It tests infrastructure's behavior
    from outside of servers.
    !
    What's that?
    

    View Slide

15. Infrastructure as Code
    Automate building infra
    by Code
    !
    Chef, Puppet, Ansible …
    # Example: Chef
    package 'apache2' do
    action :install
    end
    !
    service 'apache2' do
    action :start
    end
    

    View Slide

16. When I write how to set web-
    servers up…
    1. Write Serverspec tests.
    2. Write files like chef-cookbook.
    3. After tests passed, go back to 1.
    4. Finally, The cookbook is
    completed!
    VM
    Serverspec
    Test
    Provision
    

    View Slide

17. Serverspec tests the status
    of the server.
    describe package('apache2') do
    it { should be_installed }
    end
    !
    describe service('apache2') do
    it { should be_enabled }
    it { should be_running }
    end
    !
    describe port(80) do
    it { should be_listening }
    end
    package 'apache2' do
    action :install
    end
    !
    service 'apache2' do
    action :start
    end
    Serverspec Chef
    

    View Slide

18. I completed! But …
    $ sudo sh -c 'echo "192.168.33.10 web" >> /etc/hosts'
    $ curl -v http://web
    * About to connect() to 192.168.33.10 port 80 (#0)
    * Trying 192.168.33.10...
    (It takes long long time…)
    ^C
    $ telnet web 80
    Trying 192.168.33.10...
    (It takes long long time…)
    ^C
    

    View Slide

19. Because of iptables
    $ sudo iptables -L
    Chain INPUT (policy DROP)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
    ACCEPT all -- anywhere anywhere
    ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
    !
    Chain FORWARD (policy DROP)
    target prot opt source destination
    !
    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination
    

    View Slide

20. It works!
    $ sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport
    80 -j ACCEPT
    !
    $ curl http://web
    It works!
    This is the default web page for this server.
    The web server software is running but no content has been
    added, yet.
    
    

    View Slide

21. How to test such case?
    Check whether the server returns 200 OK,
    when I send a HTTP request.
    

    View Slide

22. How to test such case?
    Check whether the server returns 200 OK,
    when I send a HTTP request.
    describe http('http://app') do
    it "responds OK 200" do
    expect(response.code).to eq('200')
    end
    end
    This is what Infrataster is!
    

    View Slide

23. Getting Started
    $ bundle install
    $ rspec —init
    # spec/spec_helper.rb
    require 'infrataster/rspec'
    # Gemfile
    gem 'infrataster'
    

    View Slide

24. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    VM
    Host
    Proxy
    192.168.33.10
    *OGSB
    UBTUFS
    

    View Slide

25. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    Infrataster::Server.define(
    :app, '172.16.33.11',
    vagrant: true,
    from: :proxy # access from proxy
    )
    VM
    Host
    Proxy
    192.168.33.10
    (172.16.33.10)
    App
    172.16.33.11
    *OGSB
    UBTUFS
    SSH Port Forwarding
    

    View Slide

26. Define Servers
    # spec/spec_helper.rb
    Infrataster::Server.define(
    :proxy, '192.168.33.10',
    vagrant: true,
    )
    Infrataster::Server.define(
    :app, '172.16.33.11',
    vagrant: true,
    from: :proxy
    )
    Infrataster::Server.define(
    :db, '172.16.33.12',
    vagrant: true,
    from: :app, # access from app server
    mysql: {user: 'app', password: 'app'}
    )
    VM
    Host
    Proxy
    App
    172.16.33.11
    DB
    172.16.33.12
    *OGSB
    UBTUFS
    SSH Port Forwarding
    

    View Slide

27. 'http' type spec
    describe server(:proxy) do # 192.168.33.10
    describe http('http://app') do
    it "responds content including 'Hello Sinatra'" do
    expect(response.body).to include('Hello Sinatra')
    end
    end
    describe http('http://static') do
    it "responds content including 'Welcome to nginx!'" do
    expect(response.body).to include('Welcome to nginx!')
    end
    end
    end
    

    View Slide

28. 'capybara' type spec
    describe server(:app) do # Test from proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello Sinatra')
    end
    end
    end
    

    View Slide

29. 'capybara' type spec
    Firefox
    Visit
    http://127.0.0.1:XXXXX
    App Server Listen 80
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end
    

    View Slide

30. 'capybara' type spec
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end
    Firefox
    SSH
    SSH
    App Server
    Visit
    http://127.0.0.1:XXXXX
    Forward
    from 127.0.0.1:XXXX
    to app:80
    Listen 80
    

    View Slide

31. 'capybara' type spec
    Firefox
    BrowserMob
    Proxy
    SSH
    SSH
    App Server
    Rewrite HTTP header
    => Host: app
    Visit
    http://127.0.0.1:XXXXX
    Forward
    from 127.0.0.1:XXXX
    to app:80
    Listen 80
    describe server(:app) do # Test from
    proxy server
    describe capybara('http://app') do
    it "responds content including 'Hello
    Sinatra'" do
    visit '/'
    expect(page).to have_content('Hello
    Sinatra')
    end
    end
    end
    

    View Slide

32. 'mysql_query' type spec
    describe server(:db) do
    describe mysql_query('SHOW STATUS') do
    it 'responds uptime' do
    row = results.find {|r| r['Variable_name'] == 'Uptime' }
    expect(row['Value'].to_i).to be > 0
    end
    end
    end
    

    View Slide

33. Run Specs
    $ bundle exec rspec
    Run options: include {:focus=>true}
    !
    All examples were filtered out;
    ignoring {:focus=>true}
    !
    server 'app' from 'proxy'
    capybara 'http://app'
    responds content including 'Hello
    Sinatra'
    http 'http://app'
    responds content including 'Hello
    Sinatra'
    responds OK 200
    responds as 'text/html'
    server 'proxy'
    http 'http://static'
    responds as 'text/html'
    responds content including
    'Welcome to nginx!'
    …
    !
    Finished in 17.92 seconds
    11 examples, 0 failures
    !
    Randomized with seed 12750
    

    View Slide

34. After Infrataster Test Passed
    There is no longer "iptables" problems!
    $ curl http://server_tested_with_infrataster
    It always works!
    

    View Slide

35. You can see example:
    !
    //github.com/ryotarai/infrataster/
    tree/master/example
    

    View Slide

36. Further development
    • Dog-fooding
    • Write unit-test specs & Refactor
    • Add types to support other protocols and
    middlewares
    • Support multi-hop SSH tunnel
    • Implement features that users need!
    Feedbacks are welcome!
    

    View Slide

37. //github.com/ryotarai/
    infrataster
    

    View Slide