Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker @Viadeo

Viadeo
February 25, 2015

Docker @Viadeo

The whole Viadeo's technical stack on one dev station using Docker
by Nicolas Colomer and Quention Suire, Software Engineers at Viadeo

Viadeo

February 25, 2015
Tweet

More Decks by Viadeo

Other Decks in Technology

Transcript

  1. Docker @ Viadeo
    Docker Paris - 25/02/2015

    View full-size slide

  2. Who we are...
    Nicolas Colomer
    @n_colomer
    Software Engineer @Viadeo,
    DevOps, (Big) Data, Geomatics,
    Electronics and OSS
    Quentin Suire
    @Kuhess
    Software Engineer @ Viadeo,
    DevOps, Machine Learning,
    Robotic & Science

    View full-size slide

  3. - Problematic
    - Containerization
    - Industrialization
    - Feedbacks
    - So what?
    Agenda

    View full-size slide

  4. Mention supplémentaire
    Problematic
    Houston, we have a problem!

    View full-size slide

  5. Meet Viadeo’s architecture
    network-graph
    MySQL
    platform
    memcached
    HBase
    elasticsearch
    0.20.6
    RabbitMQ
    consult-profile
    legacy
    middle-end
    (web)
    graph-api
    JS client
    middle-end
    (mobile)
    mobile apps
    elasticsearch
    1.1.2
    Spark jobs
    (over Mesos) Hadoop
    thumbor
    BATCH
    AWS + SAAS
    CLOUD
    DBs
    BACK
    FRONT
    :)

    View full-size slide

  6. commits are everywhere in the stack!
    we develop new product features, not tech stuff
    share the same environment with everyone
    need to run integration tests locally
    Full-stack developments

    View full-size slide

  7. each dev install his env based on his needs
    1 backend = n README (install, conf, etc...)
    ideally, all dependencies are installed locally...
    ... or defaults to demo/integ environments
    The stone age

    View full-size slide

  8. concurrency on mutualized environments
    README are painful to maintain
    dev station installation and configuration is hell!
    backing services are stateful
    test results may be unpredictable
    Then comes the crap

    View full-size slide

  9. each build triggers its dependent backends
    cheap (well, not really) POC
    proved it runs!
    despite:
    - slower tests
    - unstable results (leak)
    - dirty (java triggers shell + compatibility)
    - backend specific
    Sweep the past away

    View full-size slide

  10. Docker to the rescue!
    1 backend = 1 container (+ initialization scripts)
    system & service configuration + initial state

    View full-size slide

  11. Mention supplémentaire
    Containerization
    Go! Go! Docker!

    View full-size slide

  12. What is containerization?
    Back end
    Service
    Deps/conf
    port

    View full-size slide

  13. Example with MySQL
    Application MySQL
    port

    View full-size slide

  14. ★ A back end image must work out of the box on
    localhost.
    ★ Connect to a back end image must respect the
    back end conventions.
    ★ A back end must not contain "homemade" data.
    Golden Rules

    View full-size slide

  15. Port is 3306
    User is root
    Password is ""
    Configure with “defaults”

    View full-size slide

  16. EXAMPLE : UTF-8 ENCODING
    /etc/mysql/my.cnf
    Add some custom configuration
    ...
    [mysqld]
    character-set-server = utf8
    collation-server = utf8_swedish_ci
    character-set-client = utf8
    ...

    View full-size slide

  17. FROM google/debian:wheezy
    # Install MySQL server 5.5
    RUN apt-get update && \
    apt-get install -y mysql-server-5.5 && \
    apt-get clean
    # Configure root user
    ADD ./configure_mysql.sh /tmp/configure_mysql.sh
    RUN /tmp/configure_mysql.sh
    # MySQL custom configuration
    ADD ./my.cnf /etc/mysql/my.cnf
    EXPOSE 3306
    CMD ["/usr/bin/mysqld_safe"]
    Resulting Dockerfile

    View full-size slide

  18. Let’s use it :)
    mysql -u root
    CONNECT TO THE SERVICE
    docker build -t viadeo/mysql .
    docker run -p 3306:3306 viadeo/mysql
    BUILD THE IMAGE AND
    RUN THE CONTAINER

    View full-size slide

  19. - simplicity
    - speed (LXC)
    - maintainability (Dockerfile ≈ sh)
    - known starting state
    Why Docker?

    View full-size slide

  20. Mention supplémentaire
    Industrialization
    1 + 1 + … + 1 = N

    View full-size slide

  21. 1 APP with 1 SERVICE
    But how to scale?
    1 APP with N SERVICES ?
    Scalability of services

    View full-size slide

  22. Scale with command-line?
    docker run [opt...] viadeo/service1
    docker run [opt...] viadeo/service2

    docker run [opt...] viadeo/serviceN
    Hard to read and maintain!
    How to deal with dependencies?

    View full-size slide

  23. Use Orchestration!
    name: local
    ships:
    local:
    ip: 127.0.0.1
    services:
    service1:
    image: "viadeo/service1"
    instances:
    service1: { … }
    [ … ]
    serviceN:
    image: "viadeo/serviceN"
    requires: [serviceX]
    instances:
    serviceN: { … }
    Maestro

    View full-size slide

  24. Images are shared using a Docker registry.
    Use a Docker Registry
    We use Quay.io on SaaS
    REGISTRY

    View full-size slide

  25. Push Tested Images
    Git
    Repository
    Github
    Docker
    Registry
    Quay.io
    Continuous
    Integration
    CircleCI

    View full-size slide

  26. Basic test with Elasticsearch
    How to test your images?
    Elasticsearch
    9200
    docker build -t viadeo/elasticsearch .
    docker run -d -p 9200:9200 viadeo/elasticsearch
    # Test that Elasticsearch is alive
    curl -s "http://localhost:9200"
    curl localhost:9200

    View full-size slide

  27. Advanced test with Memcached
    How to test your images?
    Memcached
    11211
    MEMCACHED Client
    Memcached
    Tester

    View full-size slide

  28. Advanced test with Memcached
    How to test your images?
    docker-memcached
    ├── circle.yml
    ├── Dockerfile
    ├── README.md
    ├── run_test.sh
    └── test
    ├── Dockerfile
    ├── main.py
    └── requirements.txt

    View full-size slide

  29. Advanced test with Memcached
    test/main.py
    How to test your images?
    def test_memcached(self):
    host = os.getenv('MEMCACHED_PORT_11211_TCP_ADDR')
    port = int(os.getenv('MEMCACHED_PORT_11211_TCP_PORT'))
    address = "%s:%d" % (host, port)
    client = memcache.Client([address], debug=0)
    client.set("some_key", "Some value")
    value = client.get("some_key")
    self.assertEqual("Some value", value)

    View full-size slide

  30. Advanced test with Memcached
    run_test.sh
    How to test your images?
    # Build images
    docker build -t viadeo/memcached .
    docker build -t memcached_tester test
    # Run main container
    docker run -d --name=memcached viadeo/memcached
    # Wait Memcached to launch
    sleep 1
    # Run tester container
    docker run --rm --link=memcached:memcached memcached_tester

    View full-size slide

  31. Continuous Integration Cycle
    App
    Integration
    Dockerfile
    Integration
    Docker Registry
    Continuous
    Integration
    Git Repository
    Dev Env

    View full-size slide

  32. Mention supplémentaire
    Feedbacks
    You know, man!

    View full-size slide

  33. Network and DNS resolution
    Zookeeper

    View full-size slide

  34. SaaS downtimes
    The Murphy’s Law

    View full-size slide

  35. CircleCI VM setup
    The Evil Twin

    View full-size slide

  36. Multi-OS integration
    Wrong way!

    View full-size slide

  37. Mention supplémentaire
    So what?
    The takeaway

    View full-size slide

  38. Where we are?
    network-graph
    MySQL
    platform
    memcached
    HBase
    elasticsearch
    0.20.6
    RabbitMQ
    consult-profile
    legacy
    middle-end
    (web)
    graph-api
    JS client
    middle-end
    (mobile)
    mobile apps
    elasticsearch
    1.1.2
    Spark jobs
    (over Mesos) Hadoop
    thumbor
    AWS + SAAS
    Containerized!

    View full-size slide

  39. ISO and multi-OS environments for our devs
    easy to deploy and auto-configured
    ╭─me@v581 /tmp
    ╰─$ python -m maestro start
    # INSTANCE SERVICE SHIP CONTAINER STATUS
    1. elasticsearch elasticsearch local latest:984f444 up for 4d11h20m
    2. elasticsearch_0206 elasticsearch_0206 local 0.20.6:27ebcb9 up for 4d11h20m
    3. hbase hbase local latest:c9d87d6 up for 9m28s
    4. memcached memcached local latest:ef7d0bf up for 4d11h20m
    5. mysql mysql local latest:10ff53f up for 4d11h20m
    6. rabbitmq rabbitmq local latest:bed5aec up for 4d11h20m
    7. thumbor thumbor local latest:c27d00f up for 4d11h20m
    8. consultprofile consultprofile local master:9027f9e up for 4d11h20m
    9. platform platform local develop:aebab5b up for 4d11h20m

    View full-size slide

  40. apps can easily bind together (provided they listen envvars)
    ╭─me@v581 /tmp
    ╰─$ docker run -t -i --rm --link hbase:hbase quay.
    io/viadeo/viadeo-consultprofile /bin/bash
    root@7dd746618873:/app# env | grep HBASE
    HBASE_ENV_HBASE_HBASE_MASTER_UI_INTERNAL_PORT=60010
    HBASE_PORT_2181_TCP_PROTO=tcp
    HBASE_PORT_60020_TCP_PORT=60020
    HBASE_PORT_60010_TCP_PROTO=tcp
    HBASE_PORT_9090_TCP_PORT=9090
    HBASE_PORT_60020_TCP=tcp://172.17.0.41:60020
    HBASE_PORT=tcp://172.17.0.41:2181
    HBASE_ENV_HBASE_INSTANCES=hbase

    View full-size slide

  41. peaceful integration with our CI (CircleCI)
    container fast startup reduces build time
    machine:
    services:
    - docker
    dependencies:
    post:
    # stop CircleCI services
    - sudo /etc/init.d/rabbitmq-server stop
    # start Docker containers
    - docker run -d -p 9200:9200 -p 9300:9300 quay.io/viadeo/docker-elasticsearch:latest
    - docker run -d -p 9201:9200 -p 9301:9300 quay.io/viadeo/docker-elasticsearch:0.20.6
    - docker run -d -p 5672:5672 -p 15672:15672 quay.io/viadeo/docker-rabbitmq:latest

    View full-size slide

  42. easy to debug at runtime since we can tail logs of
    each container (provided they write logs to STDOUT)
    ╭─me@v581 /tmp
    ╰─$ docker logs -f platform
    [main] INFO c.v.e.o.e.plugins - [Dragon Lord] loaded [], sites []
    [main] INFO o.r.Reflections - Reflections took 9 ms to scan 1 urls, producing 1
    keys and 1 values
    [main] WARN c.v.p.r.b.DescriptorProvider - 10 resources are marked as expired
    (@legacy annotation)
    [main] INFO c.v.p.r.s.HttpConfiguration - Exposing 229 query handlers
    [main] INFO c.v.p.r.s.HttpConfiguration - Exposing 114 event listeners
    [main] INFO c.v.p.r.s.HttpConfiguration - Exposing 185 command handlers
    [main] INFO c.v.p.r.s.HttpConfiguration - Documentation loaded in 377 ms
    [main] INFO o.e.j.s.Server - jetty-8.1.11.v20130520
    Feb 20, 2015 10:05:30 AM

    View full-size slide

  43. we can pop trashable containers to make tests
    ╭─me@v581 /tmp
    ╰─$ docker run -t -i --rm -p 2181:2181 quay.io/viadeo/docker-
    hbase /bin/bash
    root@6e747d5e0782:/# supervisord -c /etc/supervisor.conf
    2015-02-24 21:02:59,144 CRIT Supervisor running as root (no user in config file)
    2015-02-24 21:02:59,146 INFO supervisord started with pid 7
    2015-02-24 21:03:00,149 INFO spawned: 'zookeeper-server' with pid 8
    2015-02-24 21:03:00,150 INFO spawned: 'hbase_master' with pid 9
    2015-02-24 21:03:00,151 INFO spawned: 'hbase-regionserver' with pid 10
    2015-02-24 21:03:00,152 INFO spawned: 'hbase_thrift' with pid 11
    2015-02-24 21:03:01,175 INFO success: zookeeper-server entered RUNNING state,
    process has stayed up for > than 1 seconds (startsecs)

    View full-size slide

  44. methodology for building SaaS apps
    by Adam Wiggins (Heroku) and friends.
    declarative setup, clean contract, portability, cloud friendly, continuous
    deployment, scalable, languages & backing services agnostic
    The DevOps holy bible
    APPROVED!

    View full-size slide

  45. auto-initialize our containers with data
    our homemade containers are dev-friendly
    multi-service container / one-node cluster
    deploy Docker containers up to the prod?
    Mesos in the pipeline, all-cloud ambitions
    containerize all backends/applications?
    lots of effort for legacy stuff…
    dockerization of SaaS dependencies? S3, Dynamo
    Where we go?

    View full-size slide

  46. Mention supplémentaire
    Questions?
    Please ask :)

    View full-size slide