Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Migrating to containers

Migrating to containers

Talk at OpenR&Day — OODrive

Matthieu Moquet

June 22, 2016
Tweet

More Decks by Matthieu Moquet

Other Decks in Programming

Transcript

  1. MIGRATING TO
    CONTAINERS
    How BlaBlaCar moved to a full container architecture

    View full-size slide

  2. Servers convergence via Chef

    View full-size slide

  3. DATA CENTER
    INDUSTRIALIZATION

    View full-size slide

  4. What’s a container?

    View full-size slide

  5. Process
    { }
    namespace
    Filesystem
    (chroot)
    Network

    View full-size slide

  6. metal servers
    services
    run. containers
    120
    200+
    3000+

    View full-size slide

  7. github.com/coreos/rkt

    View full-size slide

  8. $ (sudo) rkt run ./image.aci
    Archive of filesystem
    & instructions of
    what to run
    Foreground process
    Require root

    View full-size slide

  9. How to build an ACI?

    View full-size slide

  10. App Container
    Open specification defining how to run containers

    View full-size slide

  11. Pronounced « digg-er »
    github.com/blablacar/dgr

    View full-size slide

  12. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl

    View full-size slide

  13. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    name: aci-redis
    aci:
    app:
    exec:
    - /usr/bin/redis-server
    - /etc/redis/redis.conf

    View full-size slide

  14. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    name: aci-redis
    aci:
    app:
    exec:
    - /usr/bin/redis-server
    - /etc/redis/redis.conf
    dependencies:
    - aci-debian
    Not included in
    redis image

    View full-size slide

  15. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    #!/bin/bash
    apt install -y \
    redis-server

    View full-size slide

  16. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    Initialize database/user
    Enable/Disable modules
    Change mod/owner
    Dump configuration files

    View full-size slide

  17. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    daemonize no
    port {{ .redis.port }}
    timeout {{ .redis.timeout }}
    loglevel {{ .redis.loglevel }}
    databases {{ .redis.databases }}
    {{ if .redis.maxmemory }}
    maxmemory {{ .redis.maxmemory }}
    {{ endif }}
    ...

    View full-size slide

  18. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    default:
    redis:
    port: 6379
    timeout: 0
    loglevel: notice
    databases: 1
    Attributes are
    resolved at runtime

    View full-size slide

  19. POD
    Running several ACIs in the
    same context

    View full-size slide

  20. nginx
    php-fpm
    Linux kernel
    other process
    ...
    1.2.3.4
    172.16.8.8
    :80
    :9000
    mysql
    172.16.8.9
    :3306

    View full-size slide

  21. Services discovery with
    smartstack
    synapse + nerve
    by Airbnb

    View full-size slide

  22. Zookeeper
    haproxy
    php
    healtcheck
    rabbitmq
    client
    server
    get /services/rabbitmq
    add /services/rabbitmq ip:port

    View full-size slide

  23. systemd
    [Unit]
    Description={{.hostname}}
    [Service]
    KillMode=mixed
    Restart=always
    ExecStart=/opt/bin/rkt run \
    --hostname={{.hostname}} \
    --set-env=TEMPLATER_OVERRIDE='{{.attributes}}' \
    --volume=data,kind=host,source=/data/{{.hostname}} \
    {{.acis}}

    fleet

    ggn

    View full-size slide

  24. PRIVATE CLOUD
    BUILD, DEPLOY & RUN
    RESILIENT TO FAILURES

    View full-size slide