DockerCon 2019 - Write Maintainable Integration Tests with Docker

Integration tests are extremely valuable but very hard to maintain and to write. You need to provision and manage the lifecycle for the dependencies that your application uses such as caches and databases. Testcontainer is an open source community all focused on solving this kind of problems across many languages. Gianluca is the maintainer of the Golang implementation that uses the Docker API to expose a test-friendly library that you can use in your test cases. During this talk, he will show you how to use testcontainer to write unit tests in Go with very low overhead.

Transcript

1. Gianluca Arbezzano
   SRE, InfluxData
   Write
   maintainable test
   with Docker
   

   View Slide

2. Developer, Docker Captain and CNCF
   Ambassador. I like to hang out with good
   people!
   You can discover a lot of about me here:
   ● https://gianarb.it
   ● @gianarb
   What I like:
   ● I make dirty hacks that look awesome
   ● I grow my vegetables
   ● Travel for fun and work
   Who am I?
   

   View Slide

3. At least not integration tests
   Testing is not a
   solved issue
   

   View Slide

4. Old school applications (the greaters)
   

   View Slide

5. Today’s applications
   

   View Slide

6. The “mock everything” era is over
   we have too many integration
   points...
   

   View Slide

7. This doesn’t mean we need to
   stop doing unit tests! They still
   matter a lot!
   Disclaimer
   

   View Slide

8. •jUnit
   •PHPUnit
   •unittest
   •go “testing”
   There are
   frameworks for
   unit tests
   

   View Slide

9. $ docker-compose up -d
   $ make test-integration
   1. Orchestration Issue
   2. Validation issue
   3. Flakiness
   4. No Parallelization
   What about
   integration
   tests?
   

   View Slide

10. We can do better
    

    View Slide

11. We need to
    remember that
    Docker has an API
    

    View Slide

12. View Slide

13. View Slide

14. DIND
    docker run \
    -v /var/run/docker.sock:/var/run/docker.sock ...
    

    View Slide

15. Over the DOCKER CLI
    dockerd -H tcp://10.120.0.12
    

    View Slide

16. The SDK
    ctx := context.Background()
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
    panic(err)
    }
    cli.NegotiateAPIVersion(ctx)
    reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{})
    if err != nil {
    panic(err)
    }
    io.Copy(os.Stdout, reader)
    

    View Slide

17. Programmatically provision your
    integration tests
    config = MySqlContainer('mysql:5.7.17')
    with config as mysql:
    e = sqlalchemy.create_engine(mysql.get_connection_url())
    result = e.execute("select version()")
    

    View Slide

18. github.com/testcontainers
    TestContainers
    

    View Slide

19. View Slide

20. Where to find us:
    ● https://testcontainers.slack.com
    ● @testcontainers on Twitter
    

    View Slide

21. testcontainers-go
    

    View Slide

22. func TestNginxLatestReturn(t *testing.T) {
    ctx := context.Background()
    req := testcontainers.ContainerRequest{
    Image: "nginx",
    ExposedPorts: []string{"80/tcp"},
    }
    nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started: true,
    })
    if err != nil {
    t.Error(err)
    }
    defer nginxC.Terminate(ctx)
    ip, err := nginxC.Host(ctx)
    if err != nil {
    t.Error(err)
    }
    port, err := nginxC.MappedPort(ctx, "80")
    if err != nil {
    t.Error(err)
    }
    resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port()))
    if resp.StatusCode != http.StatusOK {
    t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode)
    }
    }
    

    View Slide

23. ctx := context.Background()
    req := ContainerRequest{
    Image: "mysql:latest",
    ExposedPorts: []string{"3306/tcp", "33060/tcp"},
    Env: map[string]string{
    "MYSQL_ROOT_PASSWORD": "password",
    "MYSQL_DATABASE": "database",
    },
    WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"),
    }
    mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started: true,
    })
    connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify",
    "root", "password", host, port, "database")
    db, err := sql.Open("mysql", connectionString)
    

    View Slide

24. testcontainers-java
    

    View Slide

25. /** Integration test for Redis-backed cache implementation. */
    public class RedisBackedCacheTest {
    @Rule
    public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379);
    private Cache cache;
    @Before
    public void setUp() throws Exception {
    Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379));
    cache = new RedisBackedCache(jedis, "test");
    }
    @Test
    public void testFindingAnInsertedValue() {
    cache.put("foo", "FOO");
    Optional foundObject = cache.get("foo", String.class);
    assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent());
    assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get());
    }
    https://github.com/testcontainers/testcontainers-java/tree/master/examples
    

    View Slide

26. container.execInContainer("touch", "/somefile.txt");
    new GenericContainer(...).withEnv("API_TOKEN", "foo")
    new GenericContainer(...)
    .withClasspathResourceMapping("redis.conf",
    "/etc/redis.conf",
    BindMode.READ_ONLY)
    

    View Slide

27. public GenericContainer nginxWithHttpWait = new
    GenericContainer("nginx:1.9.4")
    .withExposedPorts(80)
    .waitingFor(Wait.forHttp("/"));
    Wait.forHttp("/")
    .forStatusCode(200)
    .forStatusCode(301)
    

    View Slide

28. @Rule
    public GenericContainer dslContainer = new GenericContainer(
    new ImageFromDockerfile()
    .withFileFromString("folder/someFile.txt", "hello")
    .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt")
    .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))
    

    View Slide

29. public class ElasticsearchStorageRule extends ExternalResource {
    static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class);
    static final int ELASTICSEARCH_PORT = 9200; final String image; final String index;
    GenericContainer container;
    Closer closer = Closer.create();
    public ElasticsearchStorageRule(String image, String index) {
    this.image = image;
    this.index = index;
    }
    @Override
    protected void before() {
    try {
    LOGGER.info("Starting docker image " + image);
    container =
    new GenericContainer(image)
    .withExposedPorts(ELASTICSEARCH_PORT)
    .waitingFor(new HttpWaitStrategy().forPath("/"));
    container.start();
    if (Boolean.valueOf(System.getenv("ES_DEBUG"))) {
    container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image)));
    }
    System.out.println("Starting docker image " + image);
    } catch (RuntimeException e) {
    LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e);
    } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte
    gration/ElasticsearchStorageRule.java#L30
    

    View Slide

30. testcontainers/moby-ryuk
    

    View Slide

31. View Slide

32. Write your own test util functions
    You can write your packages that include functions to spin up and configure
    your environments.
    

    View Slide

33. testcontainers-go has canned
    containers
    

    View Slide

34. demo
    https://github.com/gianarb/testcontainer-demo
    

    View Slide

35. Links and Credits
    • https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa
    • https://www.testcontainers.org/
    • https://github.com/bmuschko/testcontainers-demo
    • https://gianarb.it/blog/testcontainers-go
    • https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers
    • https://github.com/testcontainers
    

    View Slide

36. @gianarb
    Thanks
    

    View Slide