Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker and Go: Integration Testing. by Aaron Lehmann

Paul Burt
February 18, 2016

Docker and Go: Integration Testing. by Aaron Lehmann

Uploaded for the 2/17 Go 1.6 release party, at Docker HQ.

Talk given at GoSF:
http://www.meetup.com/golangsf/

Paul Burt

February 18, 2016
Tweet

More Decks by Paul Burt

Other Decks in Technology

Transcript

  1. Docker and Go: Integration Testing
    Aaron Lehmann
    Docker

    View Slide

  2. Background
    Major Docker open-source projects are written in Go (Docker Engine, Docker
    Registry, Docker Swarm, etc.)
    Individual packages have unit tests (*
    _
    t
    e
    s
    t
    .
    g
    o
    )
    What about full application integration testing?

    View Slide

  3. How Docker Engine does integration testing
    Integration testing is essentially a scripting task - run commands, check output
    Tempting to use a scripting language to implement unit tests
    Engine integration tests are written in Go, as unit tests for an i
    n
    t
    e
    g
    r
    a
    t
    i
    o
    n
    -
    c
    l
    i
    package
    Much easier for contributors to write tests for their PRs when there isn't a separate
    language involved!
    Can reuse existing components for tests when it makes sense

    View Slide

  4. Simple Engine CLI integration test example
    f
    u
    n
    c (
    s *
    D
    o
    c
    k
    e
    r
    S
    u
    i
    t
    e
    ) T
    e
    s
    t
    K
    i
    l
    l
    C
    o
    n
    t
    a
    i
    n
    e
    r
    (
    c *
    c
    h
    e
    c
    k
    .
    C
    ) {
    t
    e
    s
    t
    R
    e
    q
    u
    i
    r
    e
    s
    (
    c
    , D
    a
    e
    m
    o
    n
    I
    s
    L
    i
    n
    u
    x
    )
    o
    u
    t
    , _ :
    = d
    o
    c
    k
    e
    r
    C
    m
    d
    (
    c
    , "
    r
    u
    n
    "
    , "
    -
    d
    "
    , "
    b
    u
    s
    y
    b
    o
    x
    "
    , "
    t
    o
    p
    "
    )
    c
    l
    e
    a
    n
    e
    d
    C
    o
    n
    t
    a
    i
    n
    e
    r
    I
    D :
    = s
    t
    r
    i
    n
    g
    s
    .
    T
    r
    i
    m
    S
    p
    a
    c
    e
    (
    o
    u
    t
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    w
    a
    i
    t
    R
    u
    n
    (
    c
    l
    e
    a
    n
    e
    d
    C
    o
    n
    t
    a
    i
    n
    e
    r
    I
    D
    )
    , c
    h
    e
    c
    k
    .
    I
    s
    N
    i
    l
    )
    d
    o
    c
    k
    e
    r
    C
    m
    d
    (
    c
    , "
    k
    i
    l
    l
    "
    , c
    l
    e
    a
    n
    e
    d
    C
    o
    n
    t
    a
    i
    n
    e
    r
    I
    D
    )
    o
    u
    t
    , _ = d
    o
    c
    k
    e
    r
    C
    m
    d
    (
    c
    , "
    p
    s
    "
    , "
    -
    q
    "
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    o
    u
    t
    , c
    h
    e
    c
    k
    e
    r
    .
    N
    o
    t
    (
    c
    h
    e
    c
    k
    e
    r
    .
    C
    o
    n
    t
    a
    i
    n
    s
    )
    , c
    l
    e
    a
    n
    e
    d
    C
    o
    n
    t
    a
    i
    n
    e
    r
    I
    D
    , c
    h
    e
    c
    k
    .
    C
    o
    m
    m
    e
    n
    t
    f
    (
    "
    k
    i
    l
    l
    e
    d c
    o
    n
    t
    a
    i
    n
    e
    r i
    s s
    t
    i
    l
    l r
    u
    n
    n
    i
    n
    g
    "
    )
    )
    }

    View Slide

  5. Simple Engine API integration test example
    f
    u
    n
    c (
    s *
    D
    o
    c
    k
    e
    r
    S
    u
    i
    t
    e
    ) T
    e
    s
    t
    V
    o
    l
    u
    m
    e
    s
    A
    p
    i
    C
    r
    e
    a
    t
    e
    (
    c *
    c
    h
    e
    c
    k
    .
    C
    ) {
    c
    o
    n
    f
    i
    g :
    = t
    y
    p
    e
    s
    .
    V
    o
    l
    u
    m
    e
    C
    r
    e
    a
    t
    e
    R
    e
    q
    u
    e
    s
    t
    {
    N
    a
    m
    e
    : "
    t
    e
    s
    t
    "
    ,
    }
    s
    t
    a
    t
    u
    s
    , b
    , e
    r
    r :
    = s
    o
    c
    k
    R
    e
    q
    u
    e
    s
    t
    (
    "
    P
    O
    S
    T
    "
    , "
    /
    v
    o
    l
    u
    m
    e
    s
    /
    c
    r
    e
    a
    t
    e
    "
    , c
    o
    n
    f
    i
    g
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    e
    r
    r
    , c
    h
    e
    c
    k
    .
    I
    s
    N
    i
    l
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    s
    t
    a
    t
    u
    s
    , c
    h
    e
    c
    k
    .
    E
    q
    u
    a
    l
    s
    , h
    t
    t
    p
    .
    S
    t
    a
    t
    u
    s
    C
    r
    e
    a
    t
    e
    d
    , c
    h
    e
    c
    k
    .
    C
    o
    m
    m
    e
    n
    t
    f
    (
    s
    t
    r
    i
    n
    g
    (
    b
    )
    )
    )
    v
    a
    r v
    o
    l t
    y
    p
    e
    s
    .
    V
    o
    l
    u
    m
    e
    e
    r
    r = j
    s
    o
    n
    .
    U
    n
    m
    a
    r
    s
    h
    a
    l
    (
    b
    , &
    v
    o
    l
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    e
    r
    r
    , c
    h
    e
    c
    k
    e
    r
    .
    I
    s
    N
    i
    l
    )
    c
    .
    A
    s
    s
    e
    r
    t
    (
    f
    i
    l
    e
    p
    a
    t
    h
    .
    B
    a
    s
    e
    (
    f
    i
    l
    e
    p
    a
    t
    h
    .
    D
    i
    r
    (
    v
    o
    l
    .
    M
    o
    u
    n
    t
    p
    o
    i
    n
    t
    )
    )
    , c
    h
    e
    c
    k
    e
    r
    .
    E
    q
    u
    a
    l
    s
    , c
    o
    n
    f
    i
    g
    .
    N
    a
    m
    e
    )
    }

    View Slide

  6. Engine integration test mechanics
    Tests are organized into suites using g
    o
    c
    h
    e
    c
    k
    The setup function for a suite starts up the Docker daemon and any dependencies
    for that suite (for example, D
    o
    c
    k
    e
    r
    R
    e
    g
    i
    s
    t
    r
    y
    S
    u
    i
    t
    e
    starts a local registry to test
    against)
    A teardown function runs after each test in the suite and cleans up after the test
    (very important)
    Tests can interact with either the HTTP API or the command line tool
    Many helper functions to simplify common tasks

    View Slide

  7. End-to-end testing with different component versions
    In addition to the in-tree integration tests, we also want to:
    Test compatiblity between component versions; i.e. Engine 1.X against Registry 2.Y
    Test more complex setups with additional dependencies (proxies, databases, etc.)

    View Slide

  8. golem test runner
    g
    o
    l
    e
    m
    is an integration test runner that runs complex tests on top of Docker.
    g
    o
    l
    e
    m
    creates containers for the services involved in the test using Docker
    Compose
    Each testsuite run happens inside its own container, for isolation
    Caches container images to avoid rebuilding them as part of test execution
    Open source, written in Go
    First use case: run complex end-to-end tests between Docker Registry and various
    versions of Docker Engine
    Future use case: matrix testing, running whole Docker integration-cli suite under
    various configurations

    View Slide

  9. Example golem testsuite configuration
    [
    [
    s
    u
    i
    t
    e
    ]
    ]
    d
    i
    n
    d
    =
    t
    r
    u
    e
    i
    m
    a
    g
    e
    s
    =
    [ "
    n
    g
    i
    n
    x
    :
    1
    .
    9
    "
    , "
    g
    o
    l
    a
    n
    g
    :
    1
    .
    4
    "
    , "
    h
    e
    l
    l
    o
    -
    w
    o
    r
    l
    d
    :
    l
    a
    t
    e
    s
    t
    "
    , "
    d
    m
    c
    g
    o
    w
    a
    n
    /
    t
    o
    k
    e
    n
    -
    s
    e
    r
    v
    e
    r
    :
    s
    i
    m
    p
    l
    e
    " ]
    [
    [
    s
    u
    i
    t
    e
    .
    p
    r
    e
    t
    e
    s
    t
    ]
    ]
    c
    o
    m
    m
    a
    n
    d
    =
    "
    /
    b
    i
    n
    /
    s
    h .
    /
    i
    n
    s
    t
    a
    l
    l
    _
    c
    e
    r
    t
    s
    .
    s
    h l
    o
    c
    a
    l
    r
    e
    g
    i
    s
    t
    r
    y
    "
    [
    [
    s
    u
    i
    t
    e
    .
    t
    e
    s
    t
    r
    u
    n
    n
    e
    r
    ]
    ]
    c
    o
    m
    m
    a
    n
    d
    =
    "
    b
    a
    t
    s -
    t .
    "
    f
    o
    r
    m
    a
    t
    =
    "
    t
    a
    p
    "
    e
    n
    v
    =
    [
    "
    T
    E
    S
    T
    _
    R
    E
    P
    O
    =
    h
    e
    l
    l
    o
    -
    w
    o
    r
    l
    d
    "
    , "
    T
    E
    S
    T
    _
    T
    A
    G
    =
    l
    a
    t
    e
    s
    t
    "
    , "
    T
    E
    S
    T
    _
    U
    S
    E
    R
    =
    t
    e
    s
    t
    u
    s
    e
    r
    "
    , "
    T
    E
    S
    T
    _
    P
    A
    S
    S
    W
    O
    R
    D
    =
    p
    a
    s
    s
    p
    a
    s
    s
    w
    o
    r
    d
    "
    , "
    T
    E
    S
    T
    _
    R
    E
    G
    I
    S
    T
    R
    Y
    =
    l
    o
    c
    a
    l
    r
    e
    g
    i
    s
    t
    r
    y
    "
    , "
    T
    E
    S
    T
    _
    S
    K
    I
    P
    _
    P
    U
    L
    L
    =
    t
    r
    u
    e
    "
    ]
    [
    [
    s
    u
    i
    t
    e
    .
    c
    u
    s
    t
    o
    m
    i
    m
    a
    g
    e
    ]
    ]
    t
    a
    g
    =
    "
    g
    o
    l
    e
    m
    -
    d
    i
    s
    t
    r
    i
    b
    u
    t
    i
    o
    n
    :
    l
    a
    t
    e
    s
    t
    "
    d
    e
    f
    a
    u
    l
    t
    =
    "
    r
    e
    g
    i
    s
    t
    r
    y
    :
    2
    .
    2
    .
    1
    "
    [
    [
    s
    u
    i
    t
    e
    .
    c
    u
    s
    t
    o
    m
    i
    m
    a
    g
    e
    ]
    ]
    t
    a
    g
    =
    "
    g
    o
    l
    e
    m
    -
    r
    e
    g
    i
    s
    t
    r
    y
    :
    l
    a
    t
    e
    s
    t
    "
    d
    e
    f
    a
    u
    l
    t
    =
    "
    r
    e
    g
    i
    s
    t
    r
    y
    :
    0
    .
    9
    .
    1
    "
    Each suite also includes a d
    o
    c
    k
    e
    r
    -
    c
    o
    m
    p
    o
    s
    e
    .
    y
    m
    l
    file to connect the containers.

    View Slide

  10. References
    Docker Engine test documentation:
    https://docs.docker.com/opensource/project/test-and-docs/
    Docker Engine integration-cli code on GitHub:
    https://github.com/docker/docker/tree/master/integration-cli
    gocheck: http://labix.org/gocheck
    Golem code on GitHub: https://github.com/docker/golem

    View Slide

  11. Thank you
    Aaron Lehmann
    Docker
    [email protected] (mailto:[email protected])

    View Slide

  12. View Slide