Parallel Testing with Docker

Setting up parallel testing environment with Docker

Transcript

1. CONTINUOUS INTEGRATION && DOCKER GOING PARALLEL by Michael Bodnarchuk @davert
2. WHO AM I? Web developer with Ruby, JavaScript and PHP experience Lead developer of Space Traveler Codeception PHP Testing Framework
3. WHAT IS ALL ABOUT? Development vs Testing Environment Using Docker for Testing Overview of CI Servers Running Paralell Jobs
4. DEVELOPMENT ENVIRONMENT use same OS use same PHP/Java/Ruby/Python version use same services (Nginx, MySQL, Mongo) use same configurations setup fake smtp server
6. TESTING ENVIRONMENT same as Development Environment prepared to run on CI server uses database with data fixtures should be cleaned after tests execution may contain additional services like Selenium
7. TESTING+DEVELOPMENT SETUP Use provisioning: Chef, Ansible, Puppet, ... Create/delete databases for each test execution Solve isolation issues for parallel testing
8. OR USE
9. WHAT IS DOCKER Docker is an open platform distributed apps Runs and manages isolated Linux containers Provides GitHub-like infrastructure
10. Host Container 1 Container 2 Container 3 Container ... Docker Client docker pull docker run docker ... Docker Index Docker Daemon
11. DOCKER IMAGES Isolated Filesystem Built by Docker Uses Incremental FS Drivers Can be stored in Docker Registry
12. IMAGES BUILT FROM DOCKERFILE F R O M f e d o r a : 2 0 R U N y u m i n s t a l l - y m o n g o d b - s e r v e r & & y u m c l e a n a l l R U N m k d i r - p / v a r / l i b / m o n g o d b & & \ t o u c h / v a r / l i b / m o n g o d b / . k e e p & & \ c h o w n - R m o n g o d b : m o n g o d b / v a r / l i b / m o n g o d b A D D m o n g o d b . c o n f / e t c / m o n g o d b . c o n f V O L U M E [ " / d a t a / d b " ] E X P O S E 2 7 0 1 7 U S E R m o n g o d b W O R K D I R / v a r / l i b / m o n g o d b C M D m o n g o d
14. DOCKER CONTAINER Running container != booting OS No daemons will be executed on start Only one process can be run at once
15. RUN CONTAINER $ d o c k e r r u n - i t - p 2 7 0 1 7 : 2 7 0 1 7 - v / v a r / w w w / p r o j e c t / d a t a : / d a t a m o n g o 1
16. CONTINUOUS INTEGRATION (CI) IS A DEVELOPMENT PRACTICE THAT REQUIRES DEVELOPERS TO INTEGRATE CODE INTO A SHARED REPOSITORY SEVERAL TIMES A DAY. EACH CHECK-IN IS THEN VERIFIED BY AN AUTOMATED BUILD, ALLOWING TEAMS TO DETECT PROBLEMS EARLY. thoughtworks.com/continuous-integration
17. TRAVIS CI
18. TRAVIS CI FEATURES SaaS Free for open source projects Simple configuration Virtualization via OpenVPN/Docker Concurrent builds Plans start from 129$/month
19. JENKINS CI
20. JENKINS FEATURES Free Hosted Dozens of Plugins The Most Popular and Trusted Old-School Interface
22. RUNNING PARALLEL BUILDS TravisCI: out-of-box. Runs same tests suites for different language versions. I.e. PHP 5.3, 5.4, 5.5 JenkinsCI: using multi-configuration project (MatrixReloaded plugin). Builds can be executed on slave servers.
23. BUILD MATRIX 1. Configure axis 2. Combinate values 3. Use current values as environment vars
24. CONFIGURATION MATRIX EXAMPLE
26. RUN CONCURENT JOB d o c k e r r u n - i - t - v $ W O R K S P A C E : / p r o j e c t t e s t _ c o n t a i n e r . / r u n t e s t s . s h $ S U I T E
27. WHAT'S INSIDE TEST CONTAINER? Test container contain prepared database . / r u n t e s t s . s h starts all required services (nginx, mysql, selenium, redis, etc) Container stops when tests are finished No supervisors: we can execute only one process per run
28. RUNTESTS.SH # ! / b i n / s h e c h o " S t a r t i n g S e r v i c e s . . . . " s e r v i c e e l a s t i c s e a r c h s t a r t > / d e v / n u l l 2 > & 1 s e r v i c e n g i n x s t a r t > / d e v / n u l l 2 > & 1 s e r v i c e p h p 5 - f p m s t a r t > / d e v / n u l l 2 > & 1 s e r v i c e m y s q l s t a r t > / d e v / n u l l 2 > & 1 s e r v i c e m e m c a c h e d s t a r t > / d e v / n u l l 2 > & 1 p h a n t o m j s - - w e b d r i v e r = 4 4 4 4 > / d e v / n u l l 2 > & 1 & m a i l c a t c h e r - f > / d e v / n u l l 2 > & 1 & e c h o " R u n n i n g t e s t s " c d / p r o j e c t / $ 1 # s w i t c h t o a p p l i c a t i o n c o d e c e p t r u n $ 2 # r u n t e s t s f r o m s p e c i f i c s u i t e
29. LESSONS LEARNED stopped containers should be removed containers should be auto-rebuilt on test data updates
30. CONCLUSIONS Test execution time can be improved by running tests in parallel Docker makes easy to run isolated concurrent builds
31. QUESTIONS? Talk by GitHub: Projects: , , Michael Bodnarchuk @davert DavertMik Codeception Robo Task Runner JSter