Infrataster ~ Infra Behavior Testing Framework ~ Ryota Arai ! େߐށ Ruby ձٞ 04 / #oedo04

#oedo04 is a ੜ׆ൃදձ

1. Self Intro 2. What I created recently

Who I am

Ryota Arai ߥҪ ྑଠ a.k.a. େ৔೿ͷएऀ ! COOKPAD, Inc Infra Engineer Taken by @hsbt

My First Asakusa.rb 2012/4 I started using Ruby 2012/11 େ৔೿ʹೖ৴ 2012/12/4 My First Asakusa.rb … 2014/4/19 Oedo RubyKaigi 04!!

What I created

Trending Repositories //github.com/trending

I'd like to watch GH trends on RSS reader!

//github-trends.ryotarai.info

git-commit-autouser $ gem install git-commit-autouser $ cat ~/.gitconfig [autouser-github] url-regexp = github.com name = "Ryota Arai" email = ryota.arai@gmail.com [autouser-company] url-regexp = git.example.com name = "Ohba-ha-no-wakamono" email = ryota.arai@example.com [alias] ci = commit-autouser $ git remote -v origin git@github.com:… $ git ci -m 'Hello' $ git log -1 Author: Ryota Arai $ git remote -v origin git@git.example.com:… $ git ci -m 'Hello' $ git log -1 Author: Ohba-ha-no-wakamono http://ryotarai.github.io/git-commit-autouser/

What did I contribute to? • serverspec/serverspec • opscode/chef • test-kitchen • kitchen-vagrant • etc…

Infrataster tast‧er [countable] 1. someone whose job is to test the quality of foods, teas, wines etc by tasting them (Longman Dictionary)

What is Infrataster? It tests infrastructure's behavior from outside of servers. ! What's that?

Infrastructure as Code Automate building infra by Code ! Chef, Puppet, Ansible … # Example: Chef package 'apache2' do action :install end ! service 'apache2' do action :start end

When I write how to set web- servers up… 1. Write Serverspec tests. 2. Write files like chef-cookbook. 3. After tests passed, go back to 1. 4. Finally, The cookbook is completed! VM Serverspec Test Provision

Serverspec tests the status of the server. describe package('apache2') do it { should be_installed } end ! describe service('apache2') do it { should be_enabled } it { should be_running } end ! describe port(80) do it { should be_listening } end package 'apache2' do action :install end ! service 'apache2' do action :start end Serverspec Chef

I completed! But … $ sudo sh -c 'echo "192.168.33.10 web" >> /etc/hosts' $ curl -v http://web * About to connect() to 192.168.33.10 port 80 (#0) * Trying 192.168.33.10... (It takes long long time…) ^C $ telnet web 80 Trying 192.168.33.10... (It takes long long time…) ^C

Because of iptables $ sudo iptables -L Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh ! Chain FORWARD (policy DROP) target prot opt source destination ! Chain OUTPUT (policy ACCEPT) target prot opt source destination

It works! $ sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT ! $ curl http://web

It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.

How to test such case? Check whether the server returns 200 OK, when I send a HTTP request.

How to test such case? Check whether the server returns 200 OK, when I send a HTTP request. describe http('http://app') do it "responds OK 200" do expect(response.code).to eq('200') end end This is what Infrataster is!

Getting Started $ bundle install $ rspec —init # spec/spec_helper.rb require 'infrataster/rspec' # Gemfile gem 'infrataster'

Define Servers # spec/spec_helper.rb Infrataster::Server.define( :proxy, '192.168.33.10', vagrant: true, ) VM Host Proxy 192.168.33.10 *OGSB UBTUFS

Define Servers # spec/spec_helper.rb Infrataster::Server.define( :proxy, '192.168.33.10', vagrant: true, ) Infrataster::Server.define( :app, '172.16.33.11', vagrant: true, from: :proxy # access from proxy ) VM Host Proxy 192.168.33.10 (172.16.33.10) App 172.16.33.11 *OGSB UBTUFS SSH Port Forwarding

Define Servers # spec/spec_helper.rb Infrataster::Server.define( :proxy, '192.168.33.10', vagrant: true, ) Infrataster::Server.define( :app, '172.16.33.11', vagrant: true, from: :proxy ) Infrataster::Server.define( :db, '172.16.33.12', vagrant: true, from: :app, # access from app server mysql: {user: 'app', password: 'app'} ) VM Host Proxy App 172.16.33.11 DB 172.16.33.12 *OGSB UBTUFS SSH Port Forwarding

'http' type spec describe server(:proxy) do # 192.168.33.10 describe http('http://app') do it "responds content including 'Hello Sinatra'" do expect(response.body).to include('Hello Sinatra') end end describe http('http://static') do it "responds content including 'Welcome to nginx!'" do expect(response.body).to include('Welcome to nginx!') end end end

'capybara' type spec describe server(:app) do # Test from proxy server describe capybara('http://app') do it "responds content including 'Hello Sinatra'" do visit '/' expect(page).to have_content('Hello Sinatra') end end end

'capybara' type spec Firefox Visit http://127.0.0.1:XXXXX App Server Listen 80 describe server(:app) do # Test from proxy server describe capybara('http://app') do it "responds content including 'Hello Sinatra'" do visit '/' expect(page).to have_content('Hello Sinatra') end end end

'capybara' type spec describe server(:app) do # Test from proxy server describe capybara('http://app') do it "responds content including 'Hello Sinatra'" do visit '/' expect(page).to have_content('Hello Sinatra') end end end Firefox SSH SSH App Server Visit http://127.0.0.1:XXXXX Forward from 127.0.0.1:XXXX to app:80 Listen 80

'capybara' type spec Firefox BrowserMob Proxy SSH SSH App Server Rewrite HTTP header => Host: app Visit http://127.0.0.1:XXXXX Forward from 127.0.0.1:XXXX to app:80 Listen 80 describe server(:app) do # Test from proxy server describe capybara('http://app') do it "responds content including 'Hello Sinatra'" do visit '/' expect(page).to have_content('Hello Sinatra') end end end

'mysql_query' type spec describe server(:db) do describe mysql_query('SHOW STATUS') do it 'responds uptime' do row = results.find {|r| r['Variable_name'] == 'Uptime' } expect(row['Value'].to_i).to be > 0 end end end

Run Specs $ bundle exec rspec Run options: include {:focus=>true} ! All examples were filtered out; ignoring {:focus=>true} ! server 'app' from 'proxy' capybara 'http://app' responds content including 'Hello Sinatra' http 'http://app' responds content including 'Hello Sinatra' responds OK 200 responds as 'text/html' server 'proxy' http 'http://static' responds as 'text/html' responds content including 'Welcome to nginx!' … ! Finished in 17.92 seconds 11 examples, 0 failures ! Randomized with seed 12750

After Infrataster Test Passed There is no longer "iptables" problems! $ curl http://server_tested_with_infrataster It always works!

You can see example: ! //github.com/ryotarai/infrataster/ tree/master/example

Further development • Dog-fooding • Write unit-test specs & Refactor • Add types to support other protocols and middlewares • Support multi-hop SSH tunnel • Implement features that users need! Feedbacks are welcome!

//github.com/ryotarai/ infrataster