$30 off During Our Annual Pro Sale. View Details »

DockerCon 2019 - Write Maintainable Integration Tests with Docker

DockerCon 2019 - Write Maintainable Integration Tests with Docker

Integration tests are extremely valuable but very hard to maintain and to write. You need to provision and manage the lifecycle for the dependencies that your application uses such as caches and databases. Testcontainer is an open source community all focused on solving this kind of problems across many languages. Gianluca is the maintainer of the Golang implementation that uses the Docker API to expose a test-friendly library that you can use in your test cases. During this talk, he will show you how to use testcontainer to write unit tests in Go with very low overhead.

Gianluca Arbezzano

May 01, 2019
Tweet

More Decks by Gianluca Arbezzano

Other Decks in Technology

Transcript

  1. Gianluca Arbezzano
    SRE, InfluxData
    Write
    maintainable test
    with Docker

    View Slide

  2. Developer, Docker Captain and CNCF
    Ambassador. I like to hang out with good
    people!
    You can discover a lot of about me here:
    ● https://gianarb.it
    ● @gianarb
    What I like:
    ● I make dirty hacks that look awesome
    ● I grow my vegetables
    ● Travel for fun and work
    Who am I?

    View Slide

  3. At least not integration tests
    Testing is not a
    solved issue

    View Slide

  4. Old school applications (the greaters)

    View Slide

  5. Today’s applications

    View Slide

  6. The “mock everything” era is over
    we have too many integration
    points...

    View Slide

  7. This doesn’t mean we need to
    stop doing unit tests! They still
    matter a lot!
    Disclaimer

    View Slide

  8. •jUnit
    •PHPUnit
    •unittest
    •go “testing”
    There are
    frameworks for
    unit tests

    View Slide

  9. $ docker-compose up -d
    $ make test-integration
    1. Orchestration Issue
    2. Validation issue
    3. Flakiness
    4. No Parallelization
    What about
    integration
    tests?

    View Slide

  10. We can do better

    View Slide

  11. We need to
    remember that
    Docker has an API

    View Slide

  12. View Slide

  13. View Slide

  14. DIND
    docker run \
    -v /var/run/docker.sock:/var/run/docker.sock ...

    View Slide

  15. Over the DOCKER CLI
    dockerd -H tcp://10.120.0.12

    View Slide

  16. The SDK
    ctx := context.Background()
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
    panic(err)
    }
    cli.NegotiateAPIVersion(ctx)
    reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{})
    if err != nil {
    panic(err)
    }
    io.Copy(os.Stdout, reader)

    View Slide

  17. Programmatically provision your
    integration tests
    config = MySqlContainer('mysql:5.7.17')
    with config as mysql:
    e = sqlalchemy.create_engine(mysql.get_connection_url())
    result = e.execute("select version()")

    View Slide

  18. github.com/testcontainers
    TestContainers

    View Slide

  19. View Slide

  20. Where to find us:
    ● https://testcontainers.slack.com
    ● @testcontainers on Twitter

    View Slide

  21. testcontainers-go

    View Slide

  22. func TestNginxLatestReturn(t *testing.T) {
    ctx := context.Background()
    req := testcontainers.ContainerRequest{
    Image: "nginx",
    ExposedPorts: []string{"80/tcp"},
    }
    nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started: true,
    })
    if err != nil {
    t.Error(err)
    }
    defer nginxC.Terminate(ctx)
    ip, err := nginxC.Host(ctx)
    if err != nil {
    t.Error(err)
    }
    port, err := nginxC.MappedPort(ctx, "80")
    if err != nil {
    t.Error(err)
    }
    resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port()))
    if resp.StatusCode != http.StatusOK {
    t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode)
    }
    }

    View Slide

  23. ctx := context.Background()
    req := ContainerRequest{
    Image: "mysql:latest",
    ExposedPorts: []string{"3306/tcp", "33060/tcp"},
    Env: map[string]string{
    "MYSQL_ROOT_PASSWORD": "password",
    "MYSQL_DATABASE": "database",
    },
    WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"),
    }
    mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{
    ContainerRequest: req,
    Started: true,
    })
    connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify",
    "root", "password", host, port, "database")
    db, err := sql.Open("mysql", connectionString)

    View Slide

  24. testcontainers-java

    View Slide

  25. /** Integration test for Redis-backed cache implementation. */
    public class RedisBackedCacheTest {
    @Rule
    public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379);
    private Cache cache;
    @Before
    public void setUp() throws Exception {
    Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379));
    cache = new RedisBackedCache(jedis, "test");
    }
    @Test
    public void testFindingAnInsertedValue() {
    cache.put("foo", "FOO");
    Optional foundObject = cache.get("foo", String.class);
    assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent());
    assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get());
    }
    https://github.com/testcontainers/testcontainers-java/tree/master/examples

    View Slide

  26. container.execInContainer("touch", "/somefile.txt");
    new GenericContainer(...).withEnv("API_TOKEN", "foo")
    new GenericContainer(...)
    .withClasspathResourceMapping("redis.conf",
    "/etc/redis.conf",
    BindMode.READ_ONLY)

    View Slide

  27. public GenericContainer nginxWithHttpWait = new
    GenericContainer("nginx:1.9.4")
    .withExposedPorts(80)
    .waitingFor(Wait.forHttp("/"));
    Wait.forHttp("/")
    .forStatusCode(200)
    .forStatusCode(301)

    View Slide

  28. @Rule
    public GenericContainer dslContainer = new GenericContainer(
    new ImageFromDockerfile()
    .withFileFromString("folder/someFile.txt", "hello")
    .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt")
    .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))

    View Slide

  29. public class ElasticsearchStorageRule extends ExternalResource {
    static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class);
    static final int ELASTICSEARCH_PORT = 9200; final String image; final String index;
    GenericContainer container;
    Closer closer = Closer.create();
    public ElasticsearchStorageRule(String image, String index) {
    this.image = image;
    this.index = index;
    }
    @Override
    protected void before() {
    try {
    LOGGER.info("Starting docker image " + image);
    container =
    new GenericContainer(image)
    .withExposedPorts(ELASTICSEARCH_PORT)
    .waitingFor(new HttpWaitStrategy().forPath("/"));
    container.start();
    if (Boolean.valueOf(System.getenv("ES_DEBUG"))) {
    container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image)));
    }
    System.out.println("Starting docker image " + image);
    } catch (RuntimeException e) {
    LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e);
    } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte
    gration/ElasticsearchStorageRule.java#L30

    View Slide

  30. testcontainers/moby-ryuk

    View Slide

  31. View Slide

  32. Write your own test util functions
    You can write your packages that include functions to spin up and configure
    your environments.

    View Slide

  33. testcontainers-go has canned
    containers

    View Slide

  34. demo
    https://github.com/gianarb/testcontainer-demo

    View Slide

  35. Links and Credits
    • https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa
    • https://www.testcontainers.org/
    • https://github.com/bmuschko/testcontainers-demo
    • https://gianarb.it/blog/testcontainers-go
    • https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers
    • https://github.com/testcontainers

    View Slide

  36. @gianarb
    Thanks

    View Slide