Scalable QA With Docker

code https://github.com/xebia/scalable-qa-with-docker
Stop wasting time with manual configuration and maintenance

Are you tired of maintaining all your Jenkins slaves and the inconsistency of the setup? We need to become aware that it’s inefficient to keep all the environments, browsers and dependencies up to date manually.
During this session the attendees will be introduced to setting up scalable and solid test environments with practical examples using selenium, mesos and docker.

Key takeaways
How to build and scale the Selenium Grid(s) using Docker and Mesos?
How to setup a scalable cluster to run specific test environments in no-time?
How to leverage the mesos cluster to test with other frameworks than selenium?

Transcript

1. Software Development Done Right Scalable QA with Docker 1 Maarten van den Ende – @mjvdende Adé Mochtar – amochtar@xebia.com
2. Introduction Maarten van den Ende Test Automation Architect @mjvdende Adé Mochtar Data Center Automator amochtar@xebia.com 2
3. Slides http://bit.ly/tw-scale 3
4. Code Examples 4 $ git clone https://github.com/xebia/scalable-qa-with-docker.git $ cd scalable-qa-with-docker/ $ vagrant up Download data.zip from http://bit.ly/tw-scale-data and unzip in scalable-qa-with-docker/ dir 1 2 3
5. 5 Scaling tests is hard
6. 6 Test environment hard to maintain
7. 7 Adding resources takes time
8. Selenium Grid to the rescue Use Grid to run all your tests One interface for testing all the browsers Fixed set of browsers nodes are at your disposal, ready for more 8
9. 9 Selenium Grid Over Time
10. Selenium Grid Over Time 10 Not enough nodes configured - For example: growing need to regression test each branch after each commit So you really need to take care of your selenium grid... Selenium Grid hard to maintain - Browser updates - Selenium updates Selenium Grid stability - Selenium nodes go out of memory - Browser timeouts
12. 12 How To Scale Properly
13. How to scale properly Start test - Scale up - Start nodes when needed - Start different flavor End test - Scale down - Remove nodes Ready for More! 13 Server X Source Repository Continuous Integration Datacenter Y Testrunner Chrome node Testrunner Chrome node Testrunner System Under Test
14. Next 60 minutes 1. Setup environment 2. Selenium job 3. Setup selenium grid 4. Add node to grid 5. Scale the grid 14
15. 15 1. SET UP ENVIRONMENT
16. Set up Start environment 16 Host OS Virtualbox - Virtualisatie Infrastructure Docker Guest OS CoreOS App 1 App 2 Guest OS CoreOS Docker App 3 App 4 $ vagrant up Bringing machine 'core-01' up with 'virtualbox' provider... Bringing machine 'core-02' up with 'virtualbox' provider... ==> core-01: Importing base box 'coreos_766.3.0.box'... ==> core-01: Matching MAC address for NAT networking... ==> core-01: Setting the name of the VM: scalable-qa-with-docker_core- 01_1443034087957_90607 ==> core-01: Clearing any previously set network interfaces... ==> core-01: Preparing network interfaces based on configuration... 1. SET UP
17. What are Vagrant & Virtualbox Virtualbox - Runs virtual machines on your machine - Cross platform (OS X, Windows, Linux) - Open source Vagrant - Utility for building development environments - Utility for distributing these environments - Vagrant cloud - Open source Vagrant + Virtualbox provide reproducible environments, ideal for dev and test 17 1. SET UP
18. What is Docker Platform for building, shipping and running applications Standardized container format - Application packaging - Isolation - Distribution Runs any app, everywhere 18 1. SET UP
19. What is Docker Lightweight - Share system kernel and common files Open - Runs on Linux (and Windows) Secure - Isolate applications Share and collaborate - Store, distribute and manage Docker images in docker registry Scale - Containers spin up in seconds Accelerate developer - Copy production code and run locally Empower developer - Free to use the best language for the job without causing conflict issues Eliminate environment inconsistencies - Shipping the application with configs and dependencies as a container 19 1. SET UP
20. What is CoreOS Bare Linux OS, designed to run Linux Containers like Docker Designed for security, consistency, and reliability No package manager , have to run everything in a container Clustering is first class citizen Automatic updates 20 1. SET UP
21. CoreOS components Docker - Container runtime Etcd - Distributed K/V store, basis for clustering Fleet - Basic scheduler Systemd - Service manager Flannel - Networking between containers 21 1. SET UP
22. CoreOS Unit Files 22 [Unit] Description=petclinic After=docker.service Requires=docker.service [Service] Restart=always RestartSec=5 TimeoutStartSec=0 ExecStartPre=-/usr/bin/docker pull docker-registry:5000/petclinic ExecStartPre=-/usr/bin/docker kill petclinic ExecStartPre=-/usr/bin/docker rm petclinic ExecStart=/usr/bin/docker run \ -p 8282:8080 \ --name="petclinic" \ docker-registry:5000/petclinic ExecStop=/usr/bin/docker stop petclinic 1. SET UP
23. Up and Running System under test - Petclinic Source Repository - Gitbucket Continuous Integration - Jenkins 23 1. SET UP core-01 Source Repository Continuous Integration System Under Test core-02
24. 24 2. SELENIUM JOB
25. Run test job on Jenkins Create test Commit & Push Run test 25 Core 01 GitBucket Jenkins Testrunner 2. SELENIUM JOB
26. Does it scale? No, Why not? Test fixed to specific selenium node What if - Different browser tests - +100 tests? - Selenium upgrade 26 2. SELENIUM JOB
27. 27 3. SETUP SELENIUM GRID
28. Core 01 Node Setup Selenium Grid Start Selenium Grid - Add Hub - Add node: chrome - Add node: Firefox 28 3. SETUP SELENIUM GRID GitBucket Jenkins Node Hub
29. Does it Scale? Yes we can test multiple browsers! But…!? What if? - +100 tests? - Selenium upgrades? - Grid Stability? 29 3. SETUP SELENIUM GRID
30. 30 4. ADD NODE TO GRID
31. Add Core to Grid Start Selenium Grid Add Selenium node on new core Test with new node 31 4. ADD NODE TO GRID Core 01 GitBucket Jenkins Testrunner Testrunner Core 02 Testrunner
32. Does it scale? Yes we can 100+ tests!!! But what if - Selenium upgrade? - Grid Stability? 32 4. ADD NODE TO GRID
33. 33 5. SCALE THE GRID
34. Really scale up the grid! Manually scaling grids does not scale Add scheduling component: Mesos 34 5. SCALE THE GRID
35. Mesos to rule them all Mesos groups separate machines into one big resource pool 35 5. SCALE THE GRID
36. What is Mesos? Mesosphere provides features such as application scheduling, scaling, fault- tolerance, and self-healing. It also provides application service discovery, port unification, and end-point elasticity. 36 5. SCALE THE GRID
37. Mesos Architecture 37 5. SCALE THE GRID
38. Scale: Manage all the resources Mesos master Mesos slave Mesos plugin for Jenkins - Registers framework to start Jenkins slaves on Mesos slaves 38 5. SCALE THE GRID Core 01 GitBucket Jenkins Mesos Master Core 02 Mesos slave
39. Bonus: Other test tool 39 ADDED BONUS SCALE ALL THE THINGS
40. All the things? Mesos is a generic scheduler Can run any job, not just selenium Use your mesos cluster to run any test - Integration testing - Security testing - Performance testing - ... 40 ADDED BONUS: SCALE ALL THE THINGS
41. Questions? Scalable QA With Docker