Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Let's validate your releases with JRuby, RSpec and SSH

Let's validate your releases with JRuby, RSpec and SSH

A short talk on how we validate our released inside the logstash project with jruby, rspec and ssh.

Pere Urbón

June 02, 2016

More Decks by Pere Urbón

Other Decks in Technology


  1. 2 $>whoami • Pere Urbon-Bayes (software engineer). • Have always

    worked with databases, data and analytics. • Graphdevroom@FOSDEM • When not coding I enjoy my time with my kid and wife. I also enjoy movies, tv series and running too. Basically, I’m doing everything I can to enjoy life.
  2. 3 About testing? •How many of you test in regular

    basis? TDD? •How many do integration testing? •Acceptance testing? •User acceptance testing? •Other levels of validation? Gossiping around
  3. 10 Where everything started •Logstash 1.5 took long time to

    release (nearly a year). •https://www.elastic.co/blog/logstash-1-5-0-ga-released •Releases cadence need to be speed up. •Confidence with shipping new features and refactoring. •Manual testing never scales!. •A few times bugs discovered after release that would have been possible to caught with proper testing. •https://github.com/elastic/logstash/issues/4981 there is always a reason for it
  4. 11 What do we want to need? •Automatic test with

    many different operating system’s. •Test should be OS independent. we should run on windows •Possible to be run locally, but also in CI environment without much hassle. •Test should be as uniform as possible with other elastic products. for ops
  5. 12 What do we want to need? •Bootstrap machines to

    test specific situations on demand. •Have confidence with releases. •Test complex situations quick and easy. •Tests are easy to write and wire up. for developers
  6. 13

  7. 15 Vagrant OR Docker •Quick to setup for integration purposes,

    started with zabbix, end it up with elasticsearch. •Maintenance for multiple configurations make it hardest. (new img’s) •We need test repeatable, we found this difficult for different persons (environment) and different external services versions. •Networking was a bit of an issue… •(elasticsearch cluster, VM’s, using mac’s to develop) •Docker API changes very frequently, it’s not really trusty. •The toolset had hard times to keep up with api changes. Why not docker?
  8. 19 Test executors How to write your test code •RSpec

    http://rspec.info/ •Bacon https://github.com/chneukirchen/bacon •Test::Unit https://github.com/test-unit/test-unit •Minitest https://github.com/seattlerb/minitest Lots of options to write your test in Ruby, so we had to choose
  9. 21 Options to run our integration test with RSpec •ServerSpec/ServerInfra

    v2 To recicle is good, right? With Serverspec, you can write RSpec tests for checking your servers are configured correctly. Serverspec tests your servers' actual state by executing command locally, via SSH, via WinRM, via Docker API and so on. So you don't need to install any agent softwares on your servers and can use any configuration management tools, Puppet, Ansible, CFEngine, Itamae and so on. But the true aim of Serverspec is to help refactoring infrastructure code.
  10. 22 ServerSpec / ServerInfra v2 •Lots features and capabilities, from

    networking to X509 certificates. •Extensive list of OS supported. •Focused on checking infrastructure, not acceptance testing. •Overcomplicated syntax for a testing framework. The solution to test your infrastructure describe service('org.apache.httpd'), :if => os[:family] == 'darwin' do it { should be_enabled } it { should be_running } end
  11. 23 Options to run our integration test with RSpec •Test

    Kitchen or Kitchen-CI Not good, but … Test Kitchen is a test harness tool to execute your configured code on one or more platforms in isolation. A driver plugin architecture is used which lets you run your code on various cloud providers and virtualization technologies such as Amazon EC2, Blue Box, CloudStack, Digital Ocean, Rackspace, OpenStack, Vagrant, Docker, LXC containers, and more. Many testing frameworks are already supported out of the box including Bats, shUnit2, RSpec, Serverspec, with others being created weekly.
  12. 24 Kitchen CI •Lots of versatility, extensive list of supported

    providers and operating system’s. •Designed to test infra, but versatile enough to run QA testing. •Provision using chef. •RSpec test written using serverspec, but could also use bats (bash based) and other frameworks. •In our experience it was really slow due to all the necessary bootstrap involved. An intension to provide a complete automated solution for testing It will really be an awesome project in the feature
  13. 26 Our solution to the problem Test Code + Communications

    + Machine providers RSpec Layer VM 0 (ubuntu-1404) VM 1 (debian-8) VM n (windows?) ssh ssh ssh / win RPC command runner serial / parallel ssh, and other protocols local execution provisioning * scripts ansible chef …. Machines Tests Communications x
  14. 32

  15. 33 Certainly others are using other tools •Bats scripts run

    over vagrant ssh abstraction (elasticsearch) •Ansible as test executor over docker (beats) •…. There is a solution for everyones taste Are you doing anything similar?