Docking your services with Docker

Slide 1

Slide 1 text

FullStack Developers Israel Hosted by: Docking micro services with Haggai Philip Zagury 28.1.2014 Google Campus T.A

Slide 2

Slide 2 text

WHO AM I ?

Slide 3

Slide 3 text

{ } Haggai Philip Zagury, DevOps Engineer over 10 years of DevOps expertise ●  Continuous integration ●  Continuous delivery ●  It Operations ●  Configuration management

Slide 4

Slide 4 text

“ “ I am a member of Tikal's DevOps/ALM group. With over 15 members, we meet, share, contribute and code together on a monthly basis

Slide 5

Slide 5 text

WHO WE ARE?

Slide 6

Slide 6 text

We help companies build, deliver, deploy, manage and optimize their products.

Slide 7

Slide 7 text

OUR EXPERTISE

Slide 8

Slide 8 text

text Where are we going today

Slide 9

Slide 9 text

How CM & Deployment changed Between Monolithic & SOA / MSA

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Stack => Stacks

Slide 12

Slide 12 text

More Technologies

Slide 13

Slide 13 text

More Technologies More Services (API’s)

Slide 14

Slide 14 text

More Technologies More Teams More Services (API’s)

Slide 15

Slide 15 text

Each team with it’s own “madness”

Slide 16

Slide 16 text

MONOLITHIC app deployment 1 … n 1 … n } FABRIC * n

Slide 17

Slide 17 text

} FABRIC * n MONOLITHIC style for SOA/MSA service A service B service C

Slide 18

Slide 18 text

MONOLITHIC style for SOA/MSA Team / Service A

Slide 19

Slide 19 text

MONOLITHIC style for SOA/MSA Team / Service B

Slide 20

Slide 20 text

MONOLITHIC style for SOA/MSA Team / Service C

Slide 21

Slide 21 text

} FABRIC * n MONOLITHIC style for SOA/MSA

Slide 22

Slide 22 text

text MONOLITHIC “style” •  Much more “base images” in order to save time •  Deployment takes much longer (ad hoc configuration) •  Consolidate in order to save time •  Backup & Restore ? doesn’t save time :( •  Security ? System provisioning (& OS provisioning)

Slide 23

Slide 23 text

text MONOLITHIC “style” • Kernel version not supported • Other component's depend on that • Wait for next release / OS upgrade I need xyz installed

Slide 24

Slide 24 text

text More images == GB/$$/PERF Between 100MB & nGB Cost in storage … [ e.g. S3 ] Cost in performance [ VMware …]

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

From library dependency)

Slide 28

Slide 28 text

From library dependency runtime directory (encapsulation)

Slide 29

Slide 29 text

From library dependency runtime directory (encapsulation) “.service” (hybrid)

Slide 30

Slide 30 text

} FABRIC * n Choose 1 tool for the job ?! service A service B service C

Slide 31

Slide 31 text

Containers •  OSLV -‐ OperaJng System Level VirtualizaJon (link) •  API & tooling, which enable *nix users to easily create and manage system or applicaJon containers.

Slide 32

Slide 32 text

What are containers anyway ?

Slide 33

Slide 33 text

text Linux Containers (LXC) - Why ? Why now ? •  Solaris Zones (containers - link) •  Vserver •  Openvz •  Chroot Isn't there enough container tech ? •  Solaris not widely used as linux/freebsd … •  Linux kernel support ( >= 2.6.27 ) •  Application segmentation •  We really need it !!! => “.service” era

Slide 34

Slide 34 text

text Linux Containers (LXC) - Why ? Limitation •  Kernel namespaces [ isolated processes, network etc ] •  Chroot & Seccomp (isolation) •  Control groups (a.k.a cgroups) Features • Only Linux !

Slide 35

Slide 35 text

text Revolution – Hard/Software From Rack servers => Blade

Slide 36

Slide 36 text

So what’s this DOCKER and why do I need it ?

Slide 37

Slide 37 text

Why DOCKER ? Why? •  A wrapper for LXC •  An abstraction layer for LXC + features So Why not “plain old” LXC ? •  Portable deployments across machines •  LXC alone doesn't guarantee that ! •  Docker build - a “build tool” designed for portability •  Application centric / OS centric [ Docker’s API ] •  SHA-1 (git like) based versioning •  DRY / Reuse - 1 base image for many applications •  Sharing - index (global) or registry (private / on prem)

Slide 38

Slide 38 text

text Docker ( & LXC ) Solve ! •  Daemon per container ISOLATION Any version is supported

Slide 39

Slide 39 text

text Docker ( & LXC ) Solve ! SECURITY •  Daemon per container ISOLATION •  Container == Independent ( user/group/service etc) •  New version == new container ( not toe trading …)

Slide 40

Slide 40 text

text Docker ( & LXC ) Solve ! SECURITY •  Daemon per container ISOLATION •  Container == Independent ( user/group/service etc) •  New version == new container ( not toe trading …) PORTABILITY •  Container on DEV machine => to production •  Deploy from private registry •  Rollback == latest -1

Slide 41

Slide 41 text

VM vs Container •  No hypervisor layer •  No lib duplication •  Shared kernel •  VMS are “heavy” •  5-10 x Faster •  Startup time •  VMS are “heavy” •  Better utilize HW (cloud)

Slide 42

Slide 42 text

Docker - lightweight •  Reuse kernel •  Add functionality to a container, version it, share it

Slide 43

Slide 43 text

Docker Micro service example Host / VM •  ROR front end •  Key-‐value store

Slide 44

Slide 44 text

Workflow(s)

Slide 45

Slide 45 text

The developer workflow ●  How do we test locally ? { if running on windows / OSX } ●  Define an interface with operations ?

Slide 46

Slide 46 text

Vagrant & Docker Vagrant.configure("2") do |config| config.vm.box = "dummy" config.vm.provider :docker do |docker| docker.image = "your/image:tag" docker.cmd = ["/path/to/your", "command"] end end vagrant plugin install docker-‐provider -‐ docker friendly vagrant image

Slide 47

Slide 47 text

Fast, isolated development environments using Docker. •  Define your application’s environment •  OS •  Packages •  Configuration ! etc •  Number of machines ? •  Define a container via Dockerfile •  Use that Dockerfile to define your environment (via yaml file) web: build: . links: -‐ db ports: -‐ 8000:8000 db: image: hagzag/pgsql workﬂow

Slide 48

Slide 48 text

Search & Get an image docker search root@docker-‐poc:/tmp# docker search centos*6 NAME DESCRIPTION STARS OFFICIAL TRUSTED saltstack/centos-‐6 0 [OK] salgest/centos-‐6 0 [OK] saltstack/centos-‐6-‐minimal 1 [OK] leifw/tokumx-‐buildslave-‐centos-‐6 0 [OK] tenforward/centos-‐i386 CentOS 6 32bit image 0 hansode/rpmbuilder-‐rhel6 CentOS-‐6 with rpmdevtools 0 ... hgp://index.Docker.io

Slide 49

Slide 49 text

Define your own Dockerﬁle -‐> Redis server running in a container # Docker Image/tag FROM ubuntu:12.10 # command(s) to execute on container RUN apt-‐get update RUN apt-‐get -‐y install redis-‐server # what port to listen on EXPOSE 6379 # once container is acJve what binary to run ENTRYPOINT ["/usr/bin/redis-‐server"]

Slide 50

Slide 50 text

Docker - Choose base docker pull user/container-‐name root@docker-‐poc:/tmp# docker pull saltstack/centos-‐6-‐minimal Pulling repository saltstack/centos-‐6-‐minimal aca320b373f2: Download complete f2f28f99c5fd: Download complete bf9724189396: Download complete e7adb01c55f6: Download complete a3f13a39bbbe: Download complete Git style “tags” Salt – inside …

Slide 51

Slide 51 text

Docker build build from Dockerﬁle docker build . Step 1 : FROM ubuntu:12.10 -‐-‐-‐> b750fe79269d Step 2 : RUN apt-‐get update -‐-‐-‐> Running in 0d768rc284d Fetched 9813 kB in 20s (481 kB/s) -‐-‐-‐> 46a6f0556e96 Step 3 : RUN apt-‐get -‐y install redis-‐server -‐-‐-‐> Running in 5ea88c37d21f The following extra packages will be installed: libjemalloc1 The following NEW packages will be installed: libjemalloc1 redis-‐server 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 319 kB of archives.

Slide 52

Slide 52 text

Docker build … Processing triggers for ureadahead ... -‐-‐-‐> ba4030995701 Step 4 : EXPOSE 6379 -‐-‐-‐> Running in 24720beda74b -‐-‐-‐> 6fdf06372117 Step 5 : ENTRYPOINT ["/usr/bin/redis-‐server"] -‐-‐-‐> Running in c9b9480840ad -‐-‐-‐> a6dd4adbb425 Successfully built a6dd4adbb425 docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE a6dd4adbb425 8 minutes ago 297.2 MB

Slide 53

Slide 53 text

Docker tag & push docker tag a6dd4adbb425 localhost:5000/redis_hagzag docker push localhost:5000/redis_hagzag The push refers to a repository [localhost:5000/redis_hagzag] (len: 1) Sending image list Pushing repository localhost:5000/redis_hagzag (1 tags) 27cf78414709: Image successfully pushed b750fe79269d: Image successfully pushed 46a6f0556e96: Image successfully pushed ba4030995701: Image successfully pushed 6fdf06372117: Image successfully pushed a6dd4adbb425: Image successfully pushed Pushing tags for rev [a6dd4adbb425] on {hgp://localhost:5000/v1/repositories/ redis_hagzag/tags/latest}

Slide 54

Slide 54 text

The Deployment workflow ●  Provide docker-registry service / interface ●  Monitoring & Logging facilities ●  Data binding / persistent configuration

Slide 55

Slide 55 text

Docker Our service Host / VM •  Using –name & -‐link •  Linking containers by reference (not ip) build run + -‐name, build run + -‐link tag = complete “.service” on a single node

Slide 56

Slide 56 text

Docker run & ps docker run -‐name redis -‐d a6dd4adbb425 docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9026507ef675 a6dd4adbb425 /usr/bin/redis-‐serve 12 minutes ago Up 12 minutes 6379/tcp redis 7e88dcb96856 registry:0.6.1 /bin/sh -‐c cd /docke 9 days ago Up 40 minutes 0.0.0.0:5000-‐>5000/ tcp condescending_thompson

Slide 57

Slide 57 text

Docker inspect docker inspect redis

Slide 58

Slide 58 text

What we achieved ? In container responsibility ●  Latest code ●  Dependencies Out container responsibility ●  Security & Remote access ●  Logging ●  Monitoring ●  Networking take tag “latest” of app A docker push /app-‐1 docker pull /app-‐1 Immutability ? - not just yet … but we are getting close

Slide 59

Slide 59 text

Evolving with Docker OpsEnv •  FIG •  Vagrant – buggy •  Chef-‐docker (hgps://github.com/bﬂad/chef-‐docker) •  Chef Docker registry ( hgp://community.opscode.com/cookbooks/docker-‐registry) DevEnv •  Chef-‐docker (hgps://github.com/bﬂad/chef-‐docker) •  Chef Docker registry ( hgp://community.opscode.com/cookbooks/docker-‐registry) •  Puppet docker (hgp://forge.puppetlabs.com/garethr/docker) •  DOTCLOUDS (focke authors) – About to base PASS based on Docker

Slide 60

Slide 60 text

Search for “Dockerfile”

Slide 61

Slide 61 text

A nodejs container …

Slide 62

Slide 62 text

Heroku like with Docker = Dokku hgps://github.com/progrium/dokku

Slide 63

Slide 63 text

Heroku like with LXC + Chef = Diez hgps://github.com/opdemand/deis hgp://deis.io/

Slide 64

Slide 64 text

text To Summarize •  Very promising & almost J production ready •  A great complementary to existing CM tooling •  Simplifies deployment (I know it doesn’t seem so)

Slide 65

Slide 65 text

Thank You Haggai Philip Zagury Email: hagzag@Jkalk.com