Containerized End­2­End Testing + , Tobias Schneck ConSol Software GmbH 

Test Pyramid Robust Independent Isolated Fast 

Characteristics of End­2­End Testing Different types of testing: Regression tests Functional approval tests Parallel tests with GUIs are complex Stateful tests: user logins, sessions, history Setup and cleanup of test data Manual effort > effort for test automation 

Advantages of Container Technology Isolation of environments Repository for versioning and distribution Reproducible application environment Dockerfile, docker­compose.yml Starting containers on­the­fly Optimized for parallel execution Less memory overhead Shared Linux kernel 

Containerized GUIs # # # s t a r t t h e d o c k e r c o n t a i n e r v i a x ­ f o r w a r d i n g d o c k e r r u n ­ i t ­ e D I S P L A Y = $ D I S P L A Y ­ v / t m p / . X 1 1 ­ u n i x : / t m p / . X 1 1 ­ u n i x : r w r a s c h / i n k s c a p e # # # s t a r t t h e d o c k e r c o n t a i n e r w i t h V N C i n t e r f a c e # c o n n e c t v i a U R L : h t t p : / / l o c a l h o s t : 6 9 1 1 / v n c _ a u t o . h t m l ? p a s s w o r d = v n c p a s s w o r d d o c k e r r u n ­ i t ­ p 5 9 1 1 : 5 9 0 1 ­ p 6 9 1 1 : 6 9 0 1 c o n s o l / u b u n t u ­ x f c e ­ v n c d o c k e r r u n ­ i t ­ p 5 9 1 2 : 5 9 0 1 ­ p 6 9 1 2 : 6 9 0 1 c o n s o l / c e n t o s ­ x f c e ­ v n c 

What's provided by ? Category Web tests through HTML selectors Restricted to browser content Open Source & Java API Headless execution Test writing assistance (recorder, tag finder) Automatable / Test result reporting (CI, DB, monitoring environment) 

Nagios OMD Incinga Monitoring Integration 

Test Definition (Java Script) 

Test Case Structure / / t e s t c a s e . j s / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * I n i t i a l i z a t i o n * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * / _ d y n a m i c I n c l u d e ( $ i n c l u d e F o l d e r ) ; v a r t e s t C a s e = n e w T e s t C a s e ( 6 0 , 7 0 ) ; v a r e n v = n e w E n v i r o n m e n t ( ) ; v a r a p p N o t e p a d = n e w A p p l i c a t i o n ( " g e d i t " ) ; v a r r e g i o n = n e w R e g i o n ( ) ; / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * D e s c r i p t i o n o f t h e t e s t c a s e * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * / t r y { / / . . . / * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * E x c e p t i o n h a n d l i n g a n d s h u t d o w n o f t e s t c a s e * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * / } c a t c h ( e ) { t e s t C a s e . h a n d l e E x c e p t i o n ( e ) ; } f i n a l l y { t e s t C a s e . s a v e R e s u l t ( ) ; } 

Call all Sahi Functions for Web Testing / / t e s t a c a s e . j s / * * * * * * * * * * * * * * * * * * * * * * * * * C a l l S a h i F u n c t i o n s * * * * * * * * * * * * * * * * * * * * * * * / _ n a v i g a t e T o ( " h t t p : / / s a h i . e x a m p l e . c o m / _ s _ / d y n / D r i v e r _ i n i t i a l i z e d " ) ; _ h i g h l i g h t ( _ l i n k ( " S S L M a n a g e r " ) ) ; _ i s V i s i b l e ( _ l i n k ( " S S L M a n a g e r " ) ) ; _ h i g h l i g h t ( _ l i n k ( " L o g s " ) ) _ c l i c k ( _ l i n k ( " L o g s " ) ) t e s t C a s e . e n d O f S t e p ( " T e s t S a h i l a n d i n g p a g e " , 5 ) ; 

Fluent API for UI Testing / / t e s t a c a s e . j s / * * * o p e n c a l c u l a t o r a p p * * * / v a r c a l c u l a t o r A p p = n e w A p p l i c a t i o n ( " g a l c u l a t o r " ) . o p e n ( ) ; t e s t C a s e . e n d O f S t e p ( " O p e n C a l c u l a t o r " , 3 ) ; / * * * c a l c u l a t e 5 2 5 + 1 0 0 * * * / v a r c a l c u l a t o r R e g i o n = c a l c u l a t o r A p p . g e t R e g i o n ( ) ; c a l c u l a t o r R e g i o n . t y p e ( " 5 2 5 " ) . f i n d ( " p l u s . p n g " ) . c l i c k ( ) . t y p e ( " 1 0 0 " ) ; c a l c R e g i o n . f i n d ( " r e s u l t . p n g " ) . c l i c k ( ) ; c a l c R e g i o n . w a i t F o r I m a g e ( " 6 2 5 " , 5 ) ; t e s t C a s e . e n d O f S t e p ( " c a l c u l a t e 5 2 5 + 1 0 0 " , 2 0 ) ; 

Custom Functions / / e . g . e x c l u d e d i n t o a s e p a r a t e c o m m o n . j s / * * * * * * * * * * * C o m b i n e c l i c k a n d h i g h l i g h t * * * * * * * * * / f u n c t i o n c l i c k H i g h l i g h t ( $ s e l e c t o r ) { _ h i g h l i g h t ( $ s e l e c t o r ) ; _ c l i c k ( $ s e l e c t o r ) ; } / * * * * * * * * * * * * * * * * O p e n P D F i n n a t i v e v i e w e r * * * * * * * * * * * * * * / v a r P D F _ E D I T O R _ N A M E = " m a s t e r p d f e d i t o r 3 " ; f u n c t i o n o p e n P d f F i l e ( p d f F i l e L o c a t i o n ) { r e t u r n n e w A p p l i c a t i o n ( P D F _ E D I T O R _ N A M E + ' " ' + p d f F i l e L o c a t i o n + ' " ' ) . o p e n ( ) ; } 

Test Definition (Java) p u b l i c c l a s s F i r s t E x a m p l e T e s t e x t e n d s A b s t r a c t S a k u l i T e s t { p r i v a t e s t a t i c f i n a l S t r i n g C I T R U S _ U R L = " h t t p : / / w w w . c i t r u s f r a m e w o r k . o r g / " ; p r i v a t e E n v i r o n m e n t e n v ; p r i v a t e R e g i o n s c r e e n ; @ B e f o r e C l a s s @ O v e r r i d e p u b l i c v o i d i n i t T C ( ) t h r o w s E x c e p t i o n { s u p e r . i n i t T C ( ) ; e n v = n e w E n v i r o n m e n t ( ) ; s c r e e n = n e w R e g i o n ( ) ; b r o w s e r . o p e n ( ) ; } @ O v e r r i d e p r o t e c t e d T e s t C a s e I n i t P a r a m e t e r g e t T e s t C a s e I n i t P a r a m e t e r ( ) t h r o w s E x c e p t i o n { r e t u r n n e w T e s t C a s e I n i t P a r a m e t e r ( " t e s t _ c i t r u s " ) ; } @ T e s t p u b l i c v o i d t e s t M y A p p ( ) t h r o w s E x c e p t i o n { / / . . . t e s t c o d e } } For the Maven dependencies, take a look at the . Java DSL Documentaion 

Test Definition (Java) p u b l i c c l a s s F i r s t E x a m p l e T e s t e x t e n d s A b s t r a c t S a k u l i T e s t { / / . . . i n i t i a l i z i n g c o d e @ T e s t p u b l i c v o i d t e s t C i t r u s H t m l C o n t e n t ( ) t h r o w s E x c e p t i o n { b r o w s e r . n a v i g a t e T o ( C I T R U S _ U R L ) ; E l e m e n t S t u b h e a d i n g 1 = b r o w s e r . p a r a g r a p h ( " C i t r u s I n t e g r a t i o n T e s t i n g " ) ; h e a d i n g 1 . h i g h l i g h t ( ) ; a s s e r t T r u e ( h e a d i n g 1 . i s V i s i b l e ( ) ) ; E l e m e n t S t u b d o w n l o a d = b r o w s e r . l i n k ( " / D o w n l o a d v . * / " ) ; d o w n l o a d . h i g h l i g h t ( ) ; a s s e r t T r u e ( d o w n l o a d . i s V i s i b l e ( ) ) ; d o w n l o a d . c l i c k ( ) ; E l e m e n t S t u b d o w n l o a d L i n k = b r o w s e r . c e l l ( " 2 . 6 . 1 " ) ; d o w n l o a d L i n k . h i g h l i g h t ( ) ; a s s e r t T r u e ( d o w n l o a d L i n k . i s V i s i b l e ( ) ) ; } @ T e s t p u b l i c v o i d t e s t C i t r u s P i c t u r e s ( ) t h r o w s E x c e p t i o n { b r o w s e r . n a v i g a t e T o ( C I T R U S _ U R L ) ; e n v . s e t S i m i l a r i t y ( 0 . 8 ) ; s c r e e n . f i n d ( " c i t r u s _ l o g o . p n g " ) . h i g h l i g h t ( ) ; e n v . t y p e ( K e y . E N D ) ; s c r e e n . f i n d ( " c o n s o l _ l o g o . p n g " ) . h i g h l i g h t ( ) ; } } 

Sakuli End­2­End Testing Container 

Demo ­ Sakuli Container # s t a r t t h e d o c k e r c o n t a i n e r d o c k e r r u n ­ i t ­ p 5 9 1 1 : 5 9 0 1 ­ p 6 9 1 1 : 6 9 0 1 c o n s o l / s a k u l i ­ u b u n t u ­ x f c e d o c k e r r u n ­ i t ­ p 5 9 1 2 : 5 9 0 1 ­ p 6 9 1 2 : 6 9 0 1 c o n s o l / s a k u l i ­ c e n t o s ­ x f c e d o c k e r r u n ­ i t ­ p 5 9 1 3 : 5 9 0 1 ­ p 6 9 1 3 : 6 9 0 1 c o n s o l / s a k u l i ­ u b u n t u ­ x f c e ­ j a v a d o c k e r r u n ­ i t ­ p 5 9 1 4 : 5 9 0 1 ­ p 6 9 1 4 : 6 9 0 1 c o n s o l / s a k u l i ­ c e n t o s ­ x f c e ­ j a v a # s t a r t i n p a r a l l e l v i a d o c k e r ­ c o m p o s e # u s e d o c k e r ­ c o m p o s . y m l f r o m h t t p s : / / g i t h u b . c o m / C o n S o l / s a k u l i / t r e e / m a s t e r / d o c k e r d o c k e r ­ c o m p o s e u p 

Bakery Demo Setup 

Bakery Demo Setup 

Bakery Demo g i t c l o n e h t t p s : / / g i t h u b . c o m / t o s c h n e c k / s a k u l i ­ e x a m p l e ­ b a k e r y ­ t e s t i n g . g i t # s t a r t j e n k i n s j e n k i n s / d e p l o y _ j e n k i n s . s h # s t a r t O M D m o n t i o r i n g o m d ­ n a g i o s / d e p l o y _ o m d . s h # s t a r t t h e b u i l d o f t h e a p p l i c a t i o n i m a g e s b a k e r y ­ a p p / a p p ­ d e p l o y m e n t ­ d o c k e r ­ c o m p o s e / d e p l o y _ a p p . s h # s t a r t t e s t s s a k u l i ­ t e s t s / e x e c u t e _ a l l . s h # s t a r t t e s t s f o r m o n i t o r i n g s a k u l i ­ t e s t s / e x e c u t e _ a l l _ 4 _ m o n i t o r i n g . s h 

What's next? Headless execution ­ Linux: VNC & Docker Windows: ? Cloud­ready container for OpenShift (security, source­2­image) Video recording of the test execution (error documentation) Web UI to handle Sakuli test suites Connect 3rd­party test management tools (HP QC, TestRail, ...) Improve test result presentation in CI tools Selenium as an alternative to Sahi Implement Junit 5 test runner 

Links ConSol/sakuli Consol/sakuli­examples toschneck/sakuli­example­bakery­testing [email protected] 

Thank you Mannheim! Tobias Schneck ConSol Software GmbH Franziskanerstraße 38 D­81669 München Tel: +49­89­45841­100 Fax: +49­89­45841­111 tobias.schneck@consol. toschneck [email protected] www.consol.de ConSol