Gianluca Arbezzano Write maintainable test with Docker

© 2019 InfluxData. All rights reserved. 2 Who am I? Gianluca Arbezzano Site Reliability Engineer @InfluxData ● http:/ /gianarb.it ● @gianarb What I like: ● I make dirty hacks that look awesome ● I grow my vegetables ● Travel for fun and work

At least not integration tests Testing is not a solved issue

Old school applications (the greaters)

Today’s applications

The “mock everything” era is over we have too many integration points...

This doesn’t mean we need to stop doing unit tests! They still matter a lot! Disclaimer

●jUnit ●PHPUnit ●unittest ●go “testing” There are frameworks for unit tests

$ docker-compose up -d $ make test-integration 1. Orchestration Issue 2. Validation issue 3. Flakiness 4. No Parallelization What about integration tests?

We can do better

We need to remember that Docker has an API

No content

No content

DIND docker run \ -v /var/run/docker.sock:/var/run/docker.sock ...

Over the DOCKER CLI dockerd -H tcp://10.120.0.12

The SDK ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)

Programmatically provision your integration tests config = MySqlContainer('mysql:5.7.17') with config as mysql: e = sqlalchemy.create_engine(mysql.get_connection_url()) result = e.execute("select version()")

github.com/testcontaine rs TestContainers

No content

Where to find us: https://testcontainers.slack.com @testcontainers on Twitter

testcontainers-go

func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{ Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) if err != nil { t.Error(err) } defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) if err != nil { t.Error(err) } port, err := nginxC.MappedPort(ctx, "80") if err != nil { t.Error(err) } resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }

ctx := context.Background() req := ContainerRequest{ Image: "mysql:latest", ExposedPorts: []string{"3306/tcp", "33060/tcp"}, Env: map[string]string{ "MYSQL_ROOT_PASSWORD": "password", "MYSQL_DATABASE": "database", }, WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"), } mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify", "root", "password", host, port, "database") db, err := sql.Open("mysql", connectionString)

testcontainers-java

/** Integration test for Redis-backed cache implementation. */ public class RedisBackedCacheTest { @Rule public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379); private Cache cache; @Before public void setUp() throws Exception { Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379)); cache = new RedisBackedCache(jedis, "test"); } @Test public void testFindingAnInsertedValue() { cache.put("foo", "FOO"); Optional foundObject = cache.get("foo", String.class); assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent()); assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get()); } https://github.com/testcontainers/testcontainers-java/tree/master/examples

container.execInContainer("touch", "/somefile.txt"); new GenericContainer(...).withEnv("API_TOKEN", "foo") new GenericContainer(...) .withClasspathResourceMapping("redis.conf", "/etc/redis.conf", BindMode.READ_ONLY)

public GenericContainer nginxWithHttpWait = new GenericContainer("nginx:1.9.4") .withExposedPorts(80) .waitingFor(Wait.forHttp("/")); Wait.forHttp("/") .forStatusCode(200) .forStatusCode(301)

@Rule public GenericContainer dslContainer = new GenericContainer( new ImageFromDockerfile() .withFileFromString("folder/someFile.txt", "hello") .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt") .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))

public class ElasticsearchStorageRule extends ExternalResource { static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class); static final int ELASTICSEARCH_PORT = 9200; final String image; final String index; GenericContainer container; Closer closer = Closer.create(); public ElasticsearchStorageRule(String image, String index) { this.image = image; this.index = index; } @Override protected void before() { try { LOGGER.info("Starting docker image " + image); container = new GenericContainer(image) .withExposedPorts(ELASTICSEARCH_PORT) .waitingFor(new HttpWaitStrategy().forPath("/")); container.start(); if (Boolean.valueOf(System.getenv("ES_DEBUG"))) { container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image))); } System.out.println("Starting docker image " + image); } catch (RuntimeException e) { LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e); } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte gration/ElasticsearchStorageRule.java#L30

testcontainers/moby-ryuk

No content

Write your own test util functions You can write your packages that include functions to spin up and configure your environments.

testcontainers-go has canned containers

ctx := context.Background() k := &KubeKindContainer{} err := k.Start(ctx) if err != nil { t.Fatal(err.Error()) } defer k.Terminate(ctx) clientset, err := k.GetClientset() if err != nil { t.Fatal(err.Error()) } ns, err := clientset.CoreV1().Namespaces().Get("default", metav1.GetOptions{}) if err != nil { t.Fatal(err.Error()) } https://github.com/testcontainers/testcontainers-go/pull/67

Links and Credits ● https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa ● https://www.testcontainers.org/ ● https://github.com/bmuschko/testcontainers-demo ● https://gianarb.it/blog/testcontainers-go ● https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers ● https://github.com/testcontainers

@gianarb Thanks