Migrating to containers

Talk at OpenR&Day — OODrive

Transcript

1. MIGRATING TO
   CONTAINERS
   How BlaBlaCar moved to a full container architecture
   

   View Slide

2. @MattKetmo
   

   View Slide

3. View Slide

4. Servers convergence via Chef
   

   View Slide

5. DATA CENTER
   INDUSTRIALIZATION
   

   View Slide

6. CONTAINERS
   

   View Slide

7. What’s a container?
   

   View Slide

8. Process
   

   View Slide

9. Process
   { }
   namespace
   Filesystem
   (chroot)
   Network
   

   View Slide

10. metal servers
    services
    run. containers
    120
    200+
    3000+
    

    View Slide

11. View Slide

12. github.com/coreos/rkt
    

    View Slide

13. $ (sudo) rkt run ./image.aci
    Archive of filesystem
    & instructions of
    what to run
    Foreground process
    Require root
    

    View Slide

14. How to build an ACI?
    

    View Slide

15. App Container
    Open specification defining how to run containers
    

    View Slide

16. Pronounced « digg-er »
    github.com/blablacar/dgr
    

    View Slide

17. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    

    View Slide

18. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    name: aci-redis
    aci:
    app:
    exec:
    - /usr/bin/redis-server
    - /etc/redis/redis.conf
    

    View Slide

19. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    name: aci-redis
    aci:
    app:
    exec:
    - /usr/bin/redis-server
    - /etc/redis/redis.conf
    dependencies:
    - aci-debian
    Not included in
    redis image
    

    View Slide

20. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    #!/bin/bash
    apt install -y \
    redis-server
    

    View Slide

21. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    Initialize database/user
    Enable/Disable modules
    Change mod/owner
    Dump configuration files
    …
    

    View Slide

22. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    daemonize no
    port {{ .redis.port }}
    timeout {{ .redis.timeout }}
    loglevel {{ .redis.loglevel }}
    databases {{ .redis.databases }}
    {{ if .redis.maxmemory }}
    maxmemory {{ .redis.maxmemory }}
    {{ endif }}
    ...
    

    View Slide

23. !" aci-manifest.yml
    !" attributes
    # $" redis.yml
    !" runlevels
    # $" build
    # $" install.sh
    # $" prestart-early
    # $" init.sh
    $" templates
    $" etc
    $" redis
    $" redis.conf.tmpl
    default:
    redis:
    port: 6379
    timeout: 0
    loglevel: notice
    databases: 1
    Attributes are
    resolved at runtime
    

    View Slide

24. POD
    Running several ACIs in the
    same context
    

    View Slide

25. nginx
    php-fpm
    Linux kernel
    other process
    ...
    1.2.3.4
    172.16.8.8
    :80
    :9000
    mysql
    172.16.8.9
    :3306
    

    View Slide

26. Services discovery with
    smartstack
    synapse + nerve
    by Airbnb
    

    View Slide

27. Zookeeper
    haproxy
    php
    healtcheck
    rabbitmq
    client
    server
    get /services/rabbitmq
    add /services/rabbitmq ip:port
    

    View Slide

28. systemd
    [Unit]
    Description={{.hostname}}
    [Service]
    KillMode=mixed
    Restart=always
    ExecStart=/opt/bin/rkt run \
    --hostname={{.hostname}} \
    --set-env=TEMPLATER_OVERRIDE='{{.attributes}}' \
    --volume=data,kind=host,source=/data/{{.hostname}} \
    {{.acis}}
    ▾
    fleet
    ▾
    ggn
    

    View Slide

29. PRIVATE CLOUD
    BUILD, DEPLOY & RUN
    RESILIENT TO FAILURES
    

    View Slide

30. THANK YOU
    

    View Slide