Docker and Go: Integration Testing. by Aaron Lehmann

Uploaded for the 2/17 Go 1.6 release party, at Docker HQ.

Talk given at GoSF:
http://www.meetup.com/golangsf/

Transcript

1. Docker and Go: Integration Testing Aaron Lehmann Docker
2. Background Major Docker open-source projects are written in Go (Docker Engine, Docker Registry, Docker Swarm, etc.) Individual packages have unit tests (* _ t e s t . g o ) What about full application integration testing?
3. How Docker Engine does integration testing Integration testing is essentially a scripting task - run commands, check output Tempting to use a scripting language to implement unit tests Engine integration tests are written in Go, as unit tests for an i n t e g r a t i o n - c l i package Much easier for contributors to write tests for their PRs when there isn't a separate language involved! Can reuse existing components for tests when it makes sense
4. Simple Engine CLI integration test example f u n c ( s * D o c k e r S u i t e ) T e s t K i l l C o n t a i n e r ( c * c h e c k . C ) { t e s t R e q u i r e s ( c , D a e m o n I s L i n u x ) o u t , _ : = d o c k e r C m d ( c , " r u n " , " - d " , " b u s y b o x " , " t o p " ) c l e a n e d C o n t a i n e r I D : = s t r i n g s . T r i m S p a c e ( o u t ) c . A s s e r t ( w a i t R u n ( c l e a n e d C o n t a i n e r I D ) , c h e c k . I s N i l ) d o c k e r C m d ( c , " k i l l " , c l e a n e d C o n t a i n e r I D ) o u t , _ = d o c k e r C m d ( c , " p s " , " - q " ) c . A s s e r t ( o u t , c h e c k e r . N o t ( c h e c k e r . C o n t a i n s ) , c l e a n e d C o n t a i n e r I D , c h e c k . C o m m e n t f ( " k i l l e d c o n t a i n e r i s s t i l l r u n n i n g " ) ) }
5. Simple Engine API integration test example f u n c ( s * D o c k e r S u i t e ) T e s t V o l u m e s A p i C r e a t e ( c * c h e c k . C ) { c o n f i g : = t y p e s . V o l u m e C r e a t e R e q u e s t { N a m e : " t e s t " , } s t a t u s , b , e r r : = s o c k R e q u e s t ( " P O S T " , " / v o l u m e s / c r e a t e " , c o n f i g ) c . A s s e r t ( e r r , c h e c k . I s N i l ) c . A s s e r t ( s t a t u s , c h e c k . E q u a l s , h t t p . S t a t u s C r e a t e d , c h e c k . C o m m e n t f ( s t r i n g ( b ) ) ) v a r v o l t y p e s . V o l u m e e r r = j s o n . U n m a r s h a l ( b , & v o l ) c . A s s e r t ( e r r , c h e c k e r . I s N i l ) c . A s s e r t ( f i l e p a t h . B a s e ( f i l e p a t h . D i r ( v o l . M o u n t p o i n t ) ) , c h e c k e r . E q u a l s , c o n f i g . N a m e ) }
6. Engine integration test mechanics Tests are organized into suites using g o c h e c k The setup function for a suite starts up the Docker daemon and any dependencies for that suite (for example, D o c k e r R e g i s t r y S u i t e starts a local registry to test against) A teardown function runs after each test in the suite and cleans up after the test (very important) Tests can interact with either the HTTP API or the command line tool Many helper functions to simplify common tasks
7. End-to-end testing with different component versions In addition to the in-tree integration tests, we also want to: Test compatiblity between component versions; i.e. Engine 1.X against Registry 2.Y Test more complex setups with additional dependencies (proxies, databases, etc.)
8. golem test runner g o l e m is an integration test runner that runs complex tests on top of Docker. g o l e m creates containers for the services involved in the test using Docker Compose Each testsuite run happens inside its own container, for isolation Caches container images to avoid rebuilding them as part of test execution Open source, written in Go First use case: run complex end-to-end tests between Docker Registry and various versions of Docker Engine Future use case: matrix testing, running whole Docker integration-cli suite under various configurations
9. Example golem testsuite configuration [ [ s u i t e ] ] d i n d = t r u e i m a g e s = [ " n g i n x : 1 . 9 " , " g o l a n g : 1 . 4 " , " h e l l o - w o r l d : l a t e s t " , " d m c g o w a n / t o k e n - s e r v e r : s i m p l e " ] [ [ s u i t e . p r e t e s t ] ] c o m m a n d = " / b i n / s h . / i n s t a l l _ c e r t s . s h l o c a l r e g i s t r y " [ [ s u i t e . t e s t r u n n e r ] ] c o m m a n d = " b a t s - t . " f o r m a t = " t a p " e n v = [ " T E S T _ R E P O = h e l l o - w o r l d " , " T E S T _ T A G = l a t e s t " , " T E S T _ U S E R = t e s t u s e r " , " T E S T _ P A S S W O R D = p a s s p a s s w o r d " , " T E S T _ R E G I S T R Y = l o c a l r e g i s t r y " , " T E S T _ S K I P _ P U L L = t r u e " ] [ [ s u i t e . c u s t o m i m a g e ] ] t a g = " g o l e m - d i s t r i b u t i o n : l a t e s t " d e f a u l t = " r e g i s t r y : 2 . 2 . 1 " [ [ s u i t e . c u s t o m i m a g e ] ] t a g = " g o l e m - r e g i s t r y : l a t e s t " d e f a u l t = " r e g i s t r y : 0 . 9 . 1 " Each suite also includes a d o c k e r - c o m p o s e . y m l file to connect the containers.
10. References Docker Engine test documentation: https://docs.docker.com/opensource/project/test-and-docs/ Docker Engine integration-cli code on GitHub: https://github.com/docker/docker/tree/master/integration-cli gocheck: http://labix.org/gocheck Golem code on GitHub: https://github.com/docker/golem
11. Thank you Aaron Lehmann Docker aaron.lehmann@docker.com (mailto:aaron.lehmann@docker.com)