Containerized End-2-End-Testing - ContainerDays 2016 in Hamburg

Example: Sakuli Example Bakery Testing
https://github.com/toschneck/sakuli-example-bakery-testing

Setting up unit- and integration tests are tasks which developers have learned to deal. But testing UIs from the perspective of an end user is mostly still a challenge. The key question usually to answer is: How it is possible to make these end-2-end tests stable, scalable and reproducible? The upcoming container technologies bring the hope of managing and automating your UI tests as easily as your container deployment. After a short introduction to the concepts, a live demo will show how it is possible to test a web- or a rich-client application in a containerized Linux UI. The example will be demonstrated by the open source tools "Docker" and "Sakuli". The execution and reporting of the tests will be managed by the continuous integration platform "Jenkins", which brings the ability to integrate everything into your build pipeline.

Transcript

1. Containerized End­2­End Testing + , Tobias Schneck ConSol Software GmbH
2. Test Pyramid Robust Independent Isolated Fast
3. Characteristics of End­2­End Testing Different types of testing: Regression tests Functional approval tests Parallel tests with GUIs are complex Stateful tests: user logins, sessions, history Setup and cleanup of test data
4. Manual effort > effort for test automation Advantages of Container Technology Isolation of environments Repository for versioning and distribution Reproducible application environment Dockerfile, docker­compose.yml Starting containers on­the­fly Optimized for parallel execution Less memory overhead Shared Linux kernel
5. Containerized GUIs # # # s t a r t t h e d o c k e r c o n t a i n e r v i a x ­ f o r w a r d i n g d o c k e r r u n ­ i t ­ e D I S P L A Y = $ D I S P L A Y ­ v / t m p / . X 1 1 ­ u n i x : / t m p / . X 1 1 ­ u n i x : r w r a s c h / i n k s c a p e # # # s t a r t t h e d o c k e r c o n t a i n e r w i t h V N C i n t e r f a c e # c o n n e c t v i a U R L : h t t p : / / l o c a l h o s t : 6 9 1 1 / v n c _ a u t o . h t m l ? p a s s w o r d = v n c p a s s w o r d d o c k e r r u n ­ i t ­ p 5 9 1 1 : 5 9 0 1 ­ p 6 9 1 1 : 6 9 0 1 c o n s o l / c e n t o s ­ x f c e ­ v n c d o c k e r r u n ­ i t ­ p 5 9 1 2 : 5 9 0 1 ­ p 6 9 1 2 : 6 9 0 1 c o n s o l / u b u n t u ­ x f c e ­ v n c
6. What's provided by ? Category Web tests through HTML selectors Restricted to browser content Open Source & Java API Headless execution Test writing assistance (recorder, screenshot finder) Automatable / Test result reporting (CI, DB, monitoring environment)
7. Nagios OMD Incinga Monitoring Integration
8. Sakuli End­2­End Testing Container
9. Demo ­ Sakuli Container # s t a r t t h e d o c k e r c o n t a i n e r d o c k e r r u n ­ i t ­ p 5 9 1 1 : 5 9 0 1 ­ p 6 9 1 1 : 6 9 0 1 c o n s o l / s a k u l i ­ c e n t o s ­ x f c e d o c k e r r u n ­ i t ­ p 5 9 1 2 : 5 9 0 1 ­ p 6 9 1 2 : 6 9 0 1 c o n s o l / s a k u l i ­ u b u n t u ­ x f c e # s t a r t i n p a r a l l e l v i a d o c k e r ­ c o m p o s e # u s e d o c k e r ­ c o m p o s . y m l f r o m h t t p s : / / g i t h u b . c o m / C o n S o l / s a k u l i / t r e e / m a s t e r / d o c k e r d o c k e r ­ c o m p o s e u p
10. Bakery Demo Setup
11. Bakery Demo Setup
12. Bakery Demo g i t c l o n e h t t p s : / / g i t h u b . c o m / t o s c h n e c k / s a k u l i ­ e x a m p l e ­ b a k e r y ­ t e s t i n g . g i t # s t a r t j e n k i n s j e n k i n s / d e p l o y _ j e n k i n s . s h # s t a r t O M D m o n t i o r i n g o m d ­ n a g i o s / d e p l o y _ o m d . s h # s t a r t t h e b u i l d o f t h e a p p l i c a t i o n i m a g e s b a k e r y ­ a p p / a p p ­ d e p l o y m e n t ­ d o c k e r ­ c o m p o s e / d e p l o y _ a p p . s h # s t a r t t e s t s s a k u l i ­ t e s t s / e x e c u t e _ a l l . s h # s t a r t t e s t s f o r m o n i t o r i n g s a k u l i ­ t e s t s / e x e c u t e _ a l l _ 4 _ m o n i t o r i n g . s h
13. What's next? Headless execution ­ Linux: VNC & Docker Windows: ? Video recording of the test execution (error documentation) Web UI to handle Sakuli test suites Connect 3rd­party test management tools (HP QC, TestRail, ...) Improve test result presentation in CI tools Selenium as an alternative to Sahi
14. Links https://github.com/ConSol/sakuli https://github.com/toschneck/sakuli­ example­bakery­testing https://twitter.com/sakuli_e2e https://twitter.com/consol_de
15. Thank you Hamburg! Tobias Schneck ConSol Software GmbH Franziskanerstraße 38 D­81669 München Tel: +49­89­45841­100 Fax: +49­89­45841­111 tobias.schneck@consol.de info@consol.de www.consol.de