Migrating to containers

Slide 1

Slide 1 text

MIGRATING TO CONTAINERS How BlaBlaCar moved to a full container architecture

Slide 2

Slide 2 text

@MattKetmo

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Servers convergence via Chef

Slide 5

Slide 5 text

DATA CENTER INDUSTRIALIZATION

Slide 6

Slide 6 text

CONTAINERS

Slide 7

Slide 7 text

What’s a container?

Slide 8

Slide 8 text

Process

Slide 9

Slide 9 text

Process { } namespace Filesystem (chroot) Network

Slide 10

Slide 10 text

metal servers services run. containers 120 200+ 3000+

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

github.com/coreos/rkt

Slide 13

Slide 13 text

$ (sudo) rkt run ./image.aci Archive of filesystem & instructions of what to run Foreground process Require root

Slide 14

Slide 14 text

How to build an ACI?

Slide 15

Slide 15 text

App Container Open speciﬁcation deﬁning how to run containers

Slide 16

Slide 16 text

Pronounced « digg-er » github.com/blablacar/dgr

Slide 17

Slide 17 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl

Slide 18

Slide 18 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl name: aci-redis aci: app: exec: - /usr/bin/redis-server - /etc/redis/redis.conf

Slide 19

Slide 19 text

Slide 20

Slide 20 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl #!/bin/bash apt install -y \ redis-server

Slide 21

Slide 21 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl Initialize database/user Enable/Disable modules Change mod/owner Dump conﬁguration ﬁles …

Slide 22

Slide 22 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl daemonize no port {{ .redis.port }} timeout {{ .redis.timeout }} loglevel {{ .redis.loglevel }} databases {{ .redis.databases }} {{ if .redis.maxmemory }} maxmemory {{ .redis.maxmemory }} {{ endif }} ...

Slide 23

Slide 23 text

!" aci-manifest.yml !" attributes # $" redis.yml !" runlevels # $" build # $" install.sh # $" prestart-early # $" init.sh $" templates $" etc $" redis $" redis.conf.tmpl default: redis: port: 6379 timeout: 0 loglevel: notice databases: 1 Attributes are resolved at runtime

Slide 24

Slide 24 text

POD Running several ACIs in the same context

Slide 25

Slide 25 text

nginx php-fpm Linux kernel other process ... 1.2.3.4 172.16.8.8 :80 :9000 mysql 172.16.8.9 :3306

Slide 26

Slide 26 text

Services discovery with smartstack synapse + nerve by Airbnb

Slide 27

Slide 27 text

Zookeeper haproxy php healtcheck rabbitmq client server get /services/rabbitmq add /services/rabbitmq ip:port

Slide 28

Slide 28 text

systemd [Unit] Description={{.hostname}} [Service] KillMode=mixed Restart=always ExecStart=/opt/bin/rkt run \ --hostname={{.hostname}} \ --set-env=TEMPLATER_OVERRIDE='{{.attributes}}' \ --volume=data,kind=host,source=/data/{{.hostname}} \ {{.acis}} ▾ fleet ▾ ggn