Testing like if you were in Prod, with Testcontainers

We all know about the importance of testing, we do our unit tests, we continue with our integration tests….. but in each case we are always testing isolated features and we use mocks for those things not important in those tests….. So what about testing everything with real layers and all together ? And, even better, having that big test encapsulated in our project test suits and runnable locally and from our CI… Here is where Testcontainers appear.

In this talk I will explain what was my experience introducing Testcontainers in our project that includes 8 external components (PostgreSQL, AWS S3, Kafka, company applications…) , and how we test the whole application including network, product versions, and taking into account particular configurations on each layer involved.

Transcript

1. Testing as you were in PROD , with Testcontainers
2. TABLE OF CONTENTS 1. whoami 2. the project 3. what do we want to test 4. what is testcontainers 5. code 6. references
3. $whoami JUG organiser at BarcelonaJUG Co-Founder of JBCNConf ( Barcelona ) Software Engineer at Red Hat App. Modernisation and Migration team @vilojona jvilalop@redhat.com aytartana.wordpress.com github.com/jonathanvila
4. the project
5. Ingress xavier KIE Server - jBPM - RBAC App - Security - S3 bucket A DB Postgres DB Postgres Kafka S3 bucket B The project 2 1 3 4 5 7 8 9 6 10
6. what do we want to test
7. What do we want to test End to End tests + clients network connections AWS S3 Kafka PostgreSQL
8. What do we want to test End to End tests + clients network connections + external services version PostgreSQL 9.6 KIE 7.3
9. What do we want to test End to End tests + clients network connections + external services version + timeouts,params, on clients and listeners Apache Camel Kafka AWS S3
10. What do we want to test End to End tests + network steps + external services version + timeouts,params, on clients and listeners + background async processes Insights Apache Camel KIE Server
11. What do we want to test End to End tests + network steps + external services version + timeouts,params, on clients and listeners + background async processes + external services specific configuration Apache Camel KIE Server
12. What do we want to test End to End tests + network steps + external services version + timeouts,params, on clients and listeners + background async processes + external services specific configuration Apache Camel KIE Server And everything executed in our CI process ( Github + TravisCI )
13. Have the whole test in CI
14. Testcontainers to the rescue Testcontainers ( not TestContainers ) www.testcontainers.org @testcontainers
15. what is Testcontainers
16. Testcontai...what ? Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. Your APP JUnit test Testcontainer module Docker for Java Docker Container
17. 12 Tools Modules • Docker Compose Module • Elasticsearch container • Kafka Containers • Localstack Module • Mockserver Module • Nginx Module • Apache Pulsar Module • RabbitMQ Module • Solr Container • Toxiproxy Module • Hashicorp Vault Module • Webdriver Containers OOTB Modules 18 DB Modules • JDBC support • R2DBC support • Cassandra Module • CockroachDB Module • Couchbase Module • Clickhouse Module • DB2 Module • Dynalite Module • InfluxDB Module • MariaDB Module • MongoDB Module • MS SQL Server Module • MySQL Module • Neo4j Module • Oracle-XE Module • OrientDB Module • Postgres Module • Presto Module
18. Ingress xavier KIE Server - jBPM - RBAC App - Security - Minio bucket A DB Postgres DB Postgres Kafka Localstack bucket B REAL SUBSTITUTE Components to have similar Prod env 2 1 3 4 5 7 8 9 6 10
19. Tasks with containers Definition OOTB Module, Docker image, Dockerfile Start JUnit rule, Manual, Manual parallel Logging Manually, SLF4J consumer Connection Shared network, IP & port Finish Manual, Automatic
20. Defining containers OOTB Module Dockerhub image Dockerfile file
21. Ingress xavier KIE Server - jBPM - RBAC App - Security - Minio bucket A DB Postgres DB Postgres Kafka Localstack bucket B REAL SUBSTITUTE Testcontainers modules 2 1 3 4 5 7 8 9 6 10 GenericContainer<>("jboss/kie-server-showcase:7.18.0.Final") PostgreSQLContainer () PostgreSQLContainer () LocalStackContainer() GenericContainer<>("minio/minio") GenericContainer<>("minio/mc") KafkaContainer () GenericContainer(new ImageFromDockerfile() GenericContainer<>(new ImageFromDockerfile()
22. Starting containers Manual Manual Parallel Automatically
23. Logging SLF4J Logger Manual LOG output sample
24. Connecting to a container Connecting containers to a same local network Getting container host and random exposed port
25. Waiting to be ready Wait strategy : specify the condition to consider a container as useful for testing vs Startup strategy : always always means wait until container is running
26. Passing properties to Spring Boot Overriding properties from config file with values from containers Setting the context initializer for Spring
27. Finishing the containers Manually Automatically
28. Ok, Ok, but …
29. The code
30. References
31. References Testcontainers : www.testcontainers.org @testcontainers Docker for Java : https://github.com/docker-java/docker-java Repository used in this presentation : https://github.com/project-xavier/xavier-integration.git ( class EndToEndTest.java )
32. Books ( devs not only consume tech ) Sci Fy Mind Novel
33. @vilojona jvilalop@redhat.com aytartana.wordpress.com github.com/jonathanvila Thank you :) THE END