Slide 1

Slide 1 text

Deploying Docker to production Lessons learnt

Slide 2

Slide 2 text

A clever man learns from his mistakes…

Slide 3

Slide 3 text

A clever man learns from his mistakes… a wise man learns from other people’s

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.

Slide 6

Slide 6 text

Docker provides a beautiful API that hides the real complexity.

Slide 7

Slide 7 text

Why do we call them containers?

Slide 8

Slide 8 text

ISO standards for containers were published between 1968 and 1970.

Slide 9

Slide 9 text

Software containers try to be the standard for Applications distribution Applications runtime

Slide 10

Slide 10 text

Distribution: Container Images

Slide 11

Slide 11 text

Packaging applications for distribution is not a new problem.

Slide 12

Slide 12 text

❏ Disk images .iso ❏ VMware .vdmk ❏ Vagrant .box ❏ Amazon Machine Images AMI Systems Packaging

Slide 13

Slide 13 text

❏ zip/tgz ❏ Java jar/war ❏ Debian deb ❏ RedHat rpm Application Packaging

Slide 14

Slide 14 text

Container images combine both system and application packaging. Old best practices still apply.

Slide 15

Slide 15 text

Container images can be easily built using manifests.

Slide 16

Slide 16 text

Building the same manifest twice could produce different images.

Slide 17

Slide 17 text

Build once. Promote images to different environments.

Slide 18

Slide 18 text

The difference between how you think something works and how it actually works risks hard-to-debug production issues. Gareth Rushgrove @garethr

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container

Slide 21

Slide 21 text

❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container

Slide 22

Slide 22 text

Operating System Which Alpine? FROM  alpine CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 23

Slide 23 text

Operating System Is this better? FROM  alpine:3.4 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 24

Slide 24 text

Operating System Tags can be overwritten! 3.4 won’t be the same in two weeks, probably FROM  alpine:3.4 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 25

Slide 25 text

Operating System Always the same version… but please kill me now FROM  alpine@sha256:e4c425e28a3cfe41efdfceda7ccce6… CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 26

Slide 26 text

❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container

Slide 27

Slide 27 text

Packages Which pip? FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 28

Slide 28 text

Versions Specify the version… and let’s hope developers respect versioning FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]

Slide 29

Slide 29 text

❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container

Slide 30

Slide 30 text

Application Which version of our application? FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 COPY  app.py  /app.py CMD  [“python”,  “/app.py”]

Slide 31

Slide 31 text

Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref  \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py  /app.py CMD  [“python”,  “/app.py”]

Slide 32

Slide 32 text

Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py /app.py CMD  [“python”,  “/app.py”]

Slide 33

Slide 33 text

Standard for Docker labels

Slide 34

Slide 34 text

Use labels to extract info

Slide 35

Slide 35 text

Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref  \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py  /app.py CMD  [“python”,  “/app.py”]

Slide 36

Slide 36 text

Metadata Calculate the values for the labels $  docker  build  \ -­‐-­‐build-­‐arg  vcs_ref=`git  rev-­‐parse  HEAD`  \ -­‐-­‐build-­‐arg  date=`date  -­‐u  +  "%Y-­‐%m-­‐%dT%H:%MZ"`  \ -­‐t  your_image_name  .

Slide 37

Slide 37 text

Open Source Docker Registries

Slide 38

Slide 38 text

Docker Hub

Slide 39

Slide 39 text

Official Docker Registry

Slide 40

Slide 40 text

Harbor (VMware)

Slide 41

Slide 41 text

Port.us (Suse)

Slide 42

Slide 42 text

Paid Docker Registries

Slide 43

Slide 43 text

Docker DataCenter

Slide 44

Slide 44 text

AWS ECR

Slide 45

Slide 45 text

JFrog Artifactory

Slide 46

Slide 46 text

Container Runtime

Slide 47

Slide 47 text

Build once. Promote images to different environments.

Slide 48

Slide 48 text

Jenkins Workflow 1. Detect merge to repository

Slide 49

Slide 49 text

Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry

Slide 50

Slide 50 text

Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment

Slide 51

Slide 51 text

Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment 4. If tests pass, push image to pro registry

Slide 52

Slide 52 text

Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment 4. If tests pass, push image to pro registry 5. Deploy to production

Slide 53

Slide 53 text

Keep In Mind ❏ Be clear on which versions of docker/docker-compose you allow ❏ Use Jenkins build number or timestamp as image tag ❏ Seek a Generic Build process ❏ Clean old images/containers

Slide 54

Slide 54 text

Clean old images/containers

Slide 55

Slide 55 text

Clean volumes

Slide 56

Slide 56 text

What does deploy mean?

Slide 57

Slide 57 text

Microservices architecture

Slide 58

Slide 58 text

❏ Harder to test before production ❏ Harder to build/deploy different languages ❏ More and more servers needed Microservices architecture

Slide 59

Slide 59 text

Mesos Container orchestration k8s Swarm

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

❏ Harder to test before production ❏ Harder to build/deploy different languages ❏ More and more servers needed Microservices architecture

Slide 62

Slide 62 text

Migrate to docker one step at a time.

Slide 63

Slide 63 text

Simplest scheduling you can get. 1 server = 1 container

Slide 64

Slide 64 text

❏ Start adding Dockerfile to your projects ❏ Easier testing using project’s images ❏ Deploying and building projects gets simpler ❏ Get used to Docker (logs/signals/…) Forget about orchestration for now

Slide 65

Slide 65 text

Automate everything.

Slide 66

Slide 66 text

Ansible Automation - Wimpy

Slide 67

Slide 67 text

❏ Builds and pushes docker image to registry ❏ Auto Scaling Group with CoreOS instances ❏ ELB in front of instances accessible through DNS ❏ Hooks to execute your own Ansible tasks ❏ Cloud Formation contains all the resources Deployment using Wimpy

Slide 68

Slide 68 text

$  ansible-­‐playbook deploy.yml \ –extra-­‐vars “wimpy_release_version=2.3  \ wimpy_deployment_environment=pre”

Slide 69

Slide 69 text

❏ Services used by other internal services ❏ Services exposed to the internet Two types of services / deploys

Slide 70

Slide 70 text

❏ Services used by other internal services ❏ Services exposed to the internet Two types of services / deploys

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

❏ Services used by other internal services ❏ Services exposed to the internet Two types of services / deploys

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

Re-Use configuration

Slide 77

Slide 77 text

Wrapping Up

Slide 78

Slide 78 text

We need formal pipelines to promote images from development to production. Build once and promote.

Slide 79

Slide 79 text

Choose your battles. Automate everything, but focusing on the important parts.

Slide 80

Slide 80 text

Docker is not a toy. Your Kubernetes cluster on raspberry pi is not production ready.

Slide 81

Slide 81 text

No content

Slide 82

Slide 82 text

@fiunchinho Schibsted Spain Jose Armesto