Slide 1

Slide 1 text

Gianluca Arbezzano Write maintainable test with Docker

Slide 2

Slide 2 text

© 2019 InfluxData. All rights reserved. 2 Who am I? Gianluca Arbezzano Site Reliability Engineer @InfluxData ● http:/ /gianarb.it ● @gianarb What I like: ● I make dirty hacks that look awesome ● I grow my vegetables ● Travel for fun and work

Slide 3

Slide 3 text

At least not integration tests Testing is not a solved issue

Slide 4

Slide 4 text

Old school applications (the greaters)

Slide 5

Slide 5 text

Today’s applications

Slide 6

Slide 6 text

The “mock everything” era is over we have too many integration points...

Slide 7

Slide 7 text

This doesn’t mean we need to stop doing unit tests! They still matter a lot! Disclaimer

Slide 8

Slide 8 text

●jUnit ●PHPUnit ●unittest ●go “testing” There are frameworks for unit tests

Slide 9

Slide 9 text

$ docker-compose up -d $ make test-integration 1. Orchestration Issue 2. Validation issue 3. Flakiness 4. No Parallelization What about integration tests?

Slide 10

Slide 10 text

We can do better

Slide 11

Slide 11 text

We need to remember that Docker has an API

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

DIND docker run \ -v /var/run/docker.sock:/var/run/docker.sock ...

Slide 15

Slide 15 text

Over the DOCKER CLI dockerd -H tcp://10.120.0.12

Slide 16

Slide 16 text

The SDK ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)

Slide 17

Slide 17 text

Programmatically provision your integration tests config = MySqlContainer('mysql:5.7.17') with config as mysql: e = sqlalchemy.create_engine(mysql.get_connection_url()) result = e.execute("select version()")

Slide 18

Slide 18 text

github.com/testcontaine rs TestContainers

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Where to find us: https://testcontainers.slack.com @testcontainers on Twitter

Slide 21

Slide 21 text

testcontainers-go

Slide 22

Slide 22 text

func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{ Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) if err != nil { t.Error(err) } defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) if err != nil { t.Error(err) } port, err := nginxC.MappedPort(ctx, "80") if err != nil { t.Error(err) } resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }

Slide 23

Slide 23 text

ctx := context.Background() req := ContainerRequest{ Image: "mysql:latest", ExposedPorts: []string{"3306/tcp", "33060/tcp"}, Env: map[string]string{ "MYSQL_ROOT_PASSWORD": "password", "MYSQL_DATABASE": "database", }, WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"), } mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify", "root", "password", host, port, "database") db, err := sql.Open("mysql", connectionString)

Slide 24

Slide 24 text

testcontainers-java

Slide 25

Slide 25 text

/** Integration test for Redis-backed cache implementation. */ public class RedisBackedCacheTest { @Rule public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379); private Cache cache; @Before public void setUp() throws Exception { Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379)); cache = new RedisBackedCache(jedis, "test"); } @Test public void testFindingAnInsertedValue() { cache.put("foo", "FOO"); Optional foundObject = cache.get("foo", String.class); assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent()); assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get()); } https://github.com/testcontainers/testcontainers-java/tree/master/examples

Slide 26

Slide 26 text

container.execInContainer("touch", "/somefile.txt"); new GenericContainer(...).withEnv("API_TOKEN", "foo") new GenericContainer(...) .withClasspathResourceMapping("redis.conf", "/etc/redis.conf", BindMode.READ_ONLY)

Slide 27

Slide 27 text

public GenericContainer nginxWithHttpWait = new GenericContainer("nginx:1.9.4") .withExposedPorts(80) .waitingFor(Wait.forHttp("/")); Wait.forHttp("/") .forStatusCode(200) .forStatusCode(301)

Slide 28

Slide 28 text

@Rule public GenericContainer dslContainer = new GenericContainer( new ImageFromDockerfile() .withFileFromString("folder/someFile.txt", "hello") .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt") .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))

Slide 29

Slide 29 text

public class ElasticsearchStorageRule extends ExternalResource { static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchStorageRule.class); static final int ELASTICSEARCH_PORT = 9200; final String image; final String index; GenericContainer container; Closer closer = Closer.create(); public ElasticsearchStorageRule(String image, String index) { this.image = image; this.index = index; } @Override protected void before() { try { LOGGER.info("Starting docker image " + image); container = new GenericContainer(image) .withExposedPorts(ELASTICSEARCH_PORT) .waitingFor(new HttpWaitStrategy().forPath("/")); container.start(); if (Boolean.valueOf(System.getenv("ES_DEBUG"))) { container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image))); } System.out.println("Starting docker image " + image); } catch (RuntimeException e) { LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e); } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte gration/ElasticsearchStorageRule.java#L30

Slide 30

Slide 30 text

testcontainers/moby-ryuk

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Write your own test util functions You can write your packages that include functions to spin up and configure your environments.

Slide 33

Slide 33 text

testcontainers-go has canned containers

Slide 34

Slide 34 text

ctx := context.Background() k := &KubeKindContainer{} err := k.Start(ctx) if err != nil { t.Fatal(err.Error()) } defer k.Terminate(ctx) clientset, err := k.GetClientset() if err != nil { t.Fatal(err.Error()) } ns, err := clientset.CoreV1().Namespaces().Get("default", metav1.GetOptions{}) if err != nil { t.Fatal(err.Error()) } https://github.com/testcontainers/testcontainers-go/pull/67

Slide 35

Slide 35 text

Links and Credits ● https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa ● https://www.testcontainers.org/ ● https://github.com/bmuschko/testcontainers-demo ● https://gianarb.it/blog/testcontainers-go ● https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers ● https://github.com/testcontainers

Slide 36

Slide 36 text

@gianarb Thanks