DockerCon 2019 - Write Maintainable Integration Tests with Docker

DockerCon 2019 - Write Maintainable Integration Tests with Docker

Integration tests are extremely valuable but very hard to maintain and to write. You need to provision and manage the lifecycle for the dependencies that your application uses such as caches and databases. Testcontainer is an open source community all focused on solving this kind of problems across many languages. Gianluca is the maintainer of the Golang implementation that uses the Docker API to expose a test-friendly library that you can use in your test cases. During this talk, he will show you how to use testcontainer to write unit tests in Go with very low overhead.

Fa5fd3405808cc6a9fe4b126b1ec39bd?s=128

Gianluca Arbezzano

May 01, 2019
Tweet

Transcript

  1. Gianluca Arbezzano SRE, InfluxData Write maintainable test with Docker

  2. Developer, Docker Captain and CNCF Ambassador. I like to hang

    out with good people! You can discover a lot of about me here: • https://gianarb.it • @gianarb What I like: • I make dirty hacks that look awesome • I grow my vegetables • Travel for fun and work Who am I?
  3. At least not integration tests Testing is not a solved

    issue
  4. Old school applications (the greaters)

  5. Today’s applications

  6. The “mock everything” era is over we have too many

    integration points...
  7. This doesn’t mean we need to stop doing unit tests!

    They still matter a lot! Disclaimer
  8. •jUnit •PHPUnit •unittest •go “testing” There are frameworks for unit

    tests
  9. $ docker-compose up -d $ make test-integration 1. Orchestration Issue

    2. Validation issue 3. Flakiness 4. No Parallelization What about integration tests?
  10. We can do better

  11. We need to remember that Docker has an API

  12. None
  13. None
  14. DIND docker run \ -v /var/run/docker.sock:/var/run/docker.sock ...

  15. Over the DOCKER CLI dockerd -H tcp://10.120.0.12

  16. The SDK ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if

    err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)
  17. Programmatically provision your integration tests config = MySqlContainer('mysql:5.7.17') with config

    as mysql: e = sqlalchemy.create_engine(mysql.get_connection_url()) result = e.execute("select version()")
  18. github.com/testcontainers TestContainers

  19. None
  20. Where to find us: • https://testcontainers.slack.com • @testcontainers on Twitter

  21. testcontainers-go

  22. func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{

    Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) if err != nil { t.Error(err) } defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) if err != nil { t.Error(err) } port, err := nginxC.MappedPort(ctx, "80") if err != nil { t.Error(err) } resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }
  23. ctx := context.Background() req := ContainerRequest{ Image: "mysql:latest", ExposedPorts: []string{"3306/tcp",

    "33060/tcp"}, Env: map[string]string{ "MYSQL_ROOT_PASSWORD": "password", "MYSQL_DATABASE": "database", }, WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"), } mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify", "root", "password", host, port, "database") db, err := sql.Open("mysql", connectionString)
  24. testcontainers-java

  25. /** Integration test for Redis-backed cache implementation. */ public class

    RedisBackedCacheTest { @Rule public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379); private Cache cache; @Before public void setUp() throws Exception { Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379)); cache = new RedisBackedCache(jedis, "test"); } @Test public void testFindingAnInsertedValue() { cache.put("foo", "FOO"); Optional<String> foundObject = cache.get("foo", String.class); assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent()); assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get()); } https://github.com/testcontainers/testcontainers-java/tree/master/examples
  26. container.execInContainer("touch", "/somefile.txt"); new GenericContainer(...).withEnv("API_TOKEN", "foo") new GenericContainer(...) .withClasspathResourceMapping("redis.conf", "/etc/redis.conf", BindMode.READ_ONLY)

  27. public GenericContainer nginxWithHttpWait = new GenericContainer("nginx:1.9.4") .withExposedPorts(80) .waitingFor(Wait.forHttp("/")); Wait.forHttp("/") .forStatusCode(200)

    .forStatusCode(301)
  28. @Rule public GenericContainer dslContainer = new GenericContainer( new ImageFromDockerfile() .withFileFromString("folder/someFile.txt",

    "hello") .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt") .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))
  29. public class ElasticsearchStorageRule extends ExternalResource { static final Logger LOGGER

    = LoggerFactory.getLogger(ElasticsearchStorageRule.class); static final int ELASTICSEARCH_PORT = 9200; final String image; final String index; GenericContainer container; Closer closer = Closer.create(); public ElasticsearchStorageRule(String image, String index) { this.image = image; this.index = index; } @Override protected void before() { try { LOGGER.info("Starting docker image " + image); container = new GenericContainer(image) .withExposedPorts(ELASTICSEARCH_PORT) .waitingFor(new HttpWaitStrategy().forPath("/")); container.start(); if (Boolean.valueOf(System.getenv("ES_DEBUG"))) { container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image))); } System.out.println("Starting docker image " + image); } catch (RuntimeException e) { LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e); } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte gration/ElasticsearchStorageRule.java#L30
  30. testcontainers/moby-ryuk

  31. None
  32. Write your own test util functions You can write your

    packages that include functions to spin up and configure your environments.
  33. testcontainers-go has canned containers

  34. demo https://github.com/gianarb/testcontainer-demo

  35. Links and Credits • https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa • https://www.testcontainers.org/ • https://github.com/bmuschko/testcontainers-demo •

    https://gianarb.it/blog/testcontainers-go • https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers • https://github.com/testcontainers
  36. @gianarb Thanks