Slide 1

Slide 1 text

Integration and end-to-end testing with TestСontainers Nikolay Kuznetsov @nikolayk812 31 October 2019

Slide 2

Slide 2 text

About me ● Go developer at Zalando Helsinki ● Java developer at Infobip, DevExperts ● C developer at Samsung, Motorola

Slide 3

Slide 3 text

Why integration testing?

Slide 4

Slide 4 text

Trade-offs Test Runtime

Slide 5

Slide 5 text

Basic integration test

Slide 6

Slide 6 text

Integration testing evolution ● In-memory mocking ● Local DBs ● Vagrant ● Docker, Docker Compose ● Docker API

Slide 7

Slide 7 text

Docker advantages ● 100% compatible database ● Same version as production ● Empty or known state

Slide 8

Slide 8 text

Start Docker containers for integration test ● Shell scripts ● Maven plugin ● Docker Compose ● Docker API ● MiniKube, Kubernetes

Slide 9

Slide 9 text

Shell scripts

Slide 10

Slide 10 text

Maven plugins github.com/fabric8io/docker-maven-plugin

Slide 11

Slide 11 text

Docker Compose

Slide 12

Slide 12 text

Docker API docs.docker.com/engine/api/latest

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

TestContainers ● github.com/testcontainers/testcontainers-java ● Wraps docker-java library ● Docker environment discovery ● Host port randomization ● Containers clean up on JVM shutdown ● Readiness waiting strategies

Slide 15

Slide 15 text

As simple as static GenericContainer redis = new GenericContainer("redis:5.0.6") .withExposedPorts(6379); static PostgreSQLContainer postgres = new PostgreSQLContainer();

Slide 16

Slide 16 text

Docker environment discovery

Slide 17

Slide 17 text

Host port randomization ● To prevent port conflicts ● Enables parallel builds ● API to get a host port

Slide 18

Slide 18 text

Containers cleanup https://github.com/testcontainers/moby-ryuk

Slide 19

Slide 19 text

Waiting strategies ● Host port ● HTTP ● Log message ● Docker healthcheck ● Custom

Slide 20

Slide 20 text

Host port waiting strategy ● Default: at first exposed port with timeout of 60s ● Implementation checks both from outside and inside container

Slide 21

Slide 21 text

HTTP waiting strategy ● Status & response body predicate

Slide 22

Slide 22 text

Demo I github.com/nikolayk812/vdc-tc-demo User Service

Slide 23

Slide 23 text

Demo recap ● JUnit 5 integration via Extension API @TestContainers / @Container annotations ● Modules ○ wrappers on top of GenericContainer ○ setters translated to containers environment variables

Slide 24

Slide 24 text

JUnit 5 extensions ● Test execution lifecycle phases = extension points ● Extension logic ○ Implement interface(s) from o.j.j.api.extension.* ○ Register with @ExtendsWith annotation

Slide 25

Slide 25 text

Modules ● 14 database modules ● MockServer ● LocalStack = mocked AWS by Atlassian ● Kafka, Pulsar, RabbitMQ ● Toxiproxy

Slide 26

Slide 26 text

Demo II Extensions Modules

Slide 27

Slide 27 text

Demo III github.com/nikolayk812/vdc-tc-demo User Service Item Service

Slide 28

Slide 28 text

Why end-to-end testing? ● Test business flows across multiple services ● Regression when ○ a new service introduced ○ a legacy service removed

Slide 29

Slide 29 text

Testing environment cluster Spring Cloud Kubernetes

Slide 30

Slide 30 text

Test cluster for automated E2E tests ● Unexpected versions of dependencies ○ Teammates deploy from a development branch ○ Pinging other teams to stabilize their test environment ● Unexpected database states ○ Care to clear data generated by a test run?

Slide 31

Slide 31 text

On-demand cluster for E2E tests ● Locally and at CI machines ● Needs time to spin up a new cluster ● Resources: memory and CPU ● How actually to do it?

Slide 32

Slide 32 text

Kubernetes for E2E tests

Slide 33

Slide 33 text

Non-Kubernetes for E2E tests ● YAGNI = You aren’t gonna need it (Kubernetes for end-to-end tests) ● Unless, you really want to test specific Kubernetes features or manifests ● So, start services in containers within the same Docker network! (with TestContainers)

Slide 34

Slide 34 text

E2E setup Item Service User Service Spring Cloud Netflix Eureka

Slide 35

Slide 35 text

Docker network aliases item-alias :8084 user-alias :8083 postgres-alias :5432 redis-alias :6379 eureka-alias :8761

Slide 36

Slide 36 text

Spring Cloud Eureka service names item-eureka user-eureka eureka

Slide 37

Slide 37 text

Demo IV github.com/nikolayk812/vdc-tc-demo

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

E2E approach ● Implement JUnit 5 extension for each service/database

Slide 40

Slide 40 text

E2E approach (2) ● An extension implements BeforeAllCallback interface ● Starts a corresponding container via TestContainers API ● Configures it, provides a shared a network ● Perform tests by calling endpoints via host ports

Slide 41

Slide 41 text

Demo V github.com/nikolayk812/vdc-tc-demo

Slide 42

Slide 42 text

Hints ● Host port forwarding Testcontainers.exposeHostPorts() ● Fixed host port for remote debug GenericContainer.addFixedExposedPort() ●

Slide 43

Slide 43 text

Takeaways ● https://testcontainers.org ● Balance between flexibility, speed and features ● Works on Mac, Linux, Windows ● Great for integration tests ● Possible to use for end-to-end tests

Slide 44

Slide 44 text

Mulțumesc! @nikolayk812 nikolayk812