Using Fabric and Docker for deployment testing

Slide 1

Slide 1 text

Using Fabric and Docker for deployment testing Cheng-Lung Sung (clsung@)

Slide 2

Slide 2 text

about.me/clsung

Slide 3

Slide 3 text

Working at Studio Engineering, hTC

Slide 4

Slide 4 text

Lots of Golang Few Python

Slide 5

Slide 5 text

因為缺⼈人，所以很忙 (這絕不是徵⼈人廣告)

Slide 6

Slide 6 text

Love Testing

Slide 7

Slide 7 text

Code with no test is like turning with no direction signal

Slide 8

Slide 8 text

You don’t meet trouble Trouble meets you.

Slide 9

Slide 9 text

• Motivation • Fabric Introduction • Introduction to Docker • Fabric + Docker for deployment testing Outline

Slide 10

Slide 10 text

Motivation

Slide 11

Slide 11 text

有⼀一天我夢到我做了⼀一個夢

Slide 12

Slide 12 text

(most) Developers use Mac (圖⽚片引⾃自 pycontw fb)

Slide 13

Slide 13 text

• Developers use Mac • brew install ... • Development site is Ubuntu • apt-get install ...

Slide 14

Slide 14 text

• Developer use Mac • Development site is Ubuntu • Production site is CentOS • yum install ...

Slide 15

Slide 15 text

如果這不是惡夢什麼才是？

Slide 16

Slide 16 text

• Developer use Mac • Development site is Ubuntu • Production site is CentOS • Demo site is Windows • Windows Update

Slide 17

Slide 17 text

夢到這我就醒了

Slide 18

Slide 18 text

Fabric Introduction

Slide 19

Slide 19 text

Fabric from README • Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

Slide 20

Slide 20 text

--- a/pep-0373.txt +++ b/pep-0373.txt +Update +====== + +The End Of Life date (EOL, sunset date) for Python 2.7 has been moved +five years into the future, to 2020. This decision was made to +clarify the status of Python 2.7 and relieve worries for those users +who cannot yet migrate to Python 3. See also PEP 466. + +This declaration does not guarantee that bugfix releases will be made +on a regular basis, but it should enable volunteers who want to +contribute bugfixes for Python 2.7 and it should satisfy vendors who +still have to support Python 2 for years to come. + +There will be no Python 2.8. + Release Manager and Crew ======================== @@ -50,15 +65,16 @@ Maintenance releases

Slide 21

Slide 21 text

Fabric is • A tool that lets you execute arbitrary Python functions via the command line; • A library of subroutines (built on top of a lower-level library) to make executing shell commands over SSH easy and Pythonic.

Slide 22

Slide 22 text

Fabric Features • execute local() or run() remote shell commands (via ssh) • parallel execution (via multiprocessing) • prompt input and managed output • abort execution • deﬁne tasks

Slide 23

Slide 23 text

from fabric.contrib.console import confirm def coverage(): local('coverage erase') local('coverage run ./manage.py test') local("coverage report --omit='*/site-packages/*'") local("coverage html --omit='*/site-packages/*'") def pre_commit(): create_folder() with settings(warn_only=True): result = local('./manage.py test', capture=True) if result.failed and \ not confirm("Tests failed. Continue anyway?"): abort("Aborting at user request.") coverage()

Slide 24

Slide 24 text

Fabric Environment • Environment as conﬁguration (dict, fabric.state.env) • env.host, env.password • env.hosts (list), env.passwords (dict) • env.port • env.local_user, env.user • env.roledefs (dict of name to host list mapping) • The settings context manager • with settings(warn_only=true):

Slide 25

Slide 25 text

Fabric Execution model • Execution strategy • A list of tasks is created • For each task, a task-speciﬁc host list is generated from various sources • The task list is walked through in order, and each task if run once per host in its hosts list • Tasks with no hosts in their host list are considered local-only, and will alway run once and only onces

Slide 26

Slide 26 text

from fabric.api import run, task, env def set_hosts(): env.hosts = ['dev.clsung.tw', 'linode.clsung.tw'] def task(): run('uname -s') $ fab task No hosts found. Please specify (single) host string for connection: $ fab set_hosts task [dev.clsung.tw] Executing task 'task' [dev.clsung.tw] run: uname -s [dev.clsung.tw] out: Linux [dev.clsung.tw] out: [linode.clsung.tw] Executing task 'task' [linode.clsung.tw] run: uname -s [linode.clsung.tw] out: Linux [linode.clsung.tw] out:

Slide 27

Slide 27 text

Fabric @roles from fabric.api import task, env, run, roles env.roledefs = { 'db': ['linode.clsung.tw'], 'dev': ['dev.clsung.tw'], } @roles('db') def task(): run('uname -s')

Slide 28

Slide 28 text

Fabric 總結

Slide 29

Slide 29 text

Fabric 是⽤用 python 寫的

Slide 30

Slide 30 text

也是寫成 .py

Slide 31

Slide 31 text

但是你不⽤用很懂 python

Slide 32

Slide 32 text

但是你要很懂你要做什麼

Slide 33

Slide 33 text

Fabric 是讓懶⼈人⽤用的

Slide 34

Slide 34 text

懶，也是進步的原動⼒力

Slide 35

Slide 35 text

因為我懶

Slide 36

Slide 36 text

所以我⽤用 Fabric

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Developer 的夢靨

Slide 39

Slide 39 text

"It worked yesterday." “昨天還是好的啊“ "You must have the wrong version." “你⼀一定是⽤用到舊版了“ "THIS can't be the source of THAT." “這個不可能是那個的原始碼” "It works, but it hasn't been tested." “這程式應該是會動的，只是我寫好後還沒做測試。” "It works on my machine.” “在我的機器明明就是好的”

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Developer 的夢想

Slide 42

Slide 42 text

Build once...run anywhere

Slide 43

Slide 43 text

Devops 的夢靨

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

Devops 的夢想

Slide 46

Slide 46 text

Conﬁgure once...run anything

Slide 47

Slide 47 text

地⽅方的 Devops 需要 Docker

Slide 48

Slide 48 text

為什麼不⽤用 VM 就好？

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

• Resource Allocation in VM setting • CPU, RAM • Preallocated disk, .. • The modiﬁcation is recorded • Boot once, test twice, result changes • If always use fresh image • And always re-run the whole setup process

Slide 51

Slide 51 text

LinuX Container(LXC) • Run a Linux system within another Linux system • Isolated environment for processes in the container • From inside, like a VM • From outside, like a normal process • “chroot on steroids”

Slide 52

Slide 52 text

Docker ⽤用 golang 寫的所以不要介紹太多

Slide 53

Slide 53 text

簡單地說，下載 docker image

Slide 54

Slide 54 text

Just run it!

Slide 55

Slide 55 text

# A basic apache server. To use either add or bind mount content under /var/www FROM ubuntu:12.04 MAINTAINER Kimbro Staken version: 0.1 RUN apt-get update && \ apt-get install -y apache2 && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* ENV APACHE_RUN_USER www-data ENV APACHE_RUN_GROUP www-data ENV APACHE_LOG_DIR /var/log/apache2 EXPOSE 80 CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]

Slide 56

Slide 56 text

Dockerﬁle rules!

Slide 57

Slide 57 text

⾃自已的 image ⾃自已 build

Slide 58

Slide 58 text

Docker caches

Slide 59

Slide 59 text

$ fab build [localhost] local: docker build . Uploading context 3.584 kB Uploading context Step 0 : FROM phusion/baseimage ---> 35c2d5a269ff Step 1 : ENV HOME /root ---> Using cache ---> 50937eb28ee8 .... Successfully built 532a5f458a34

Slide 60

Slide 60 text

Docker refreshes

Slide 61

Slide 61 text

# VERSION 0.0.1 FROM centos:latest MAINTAINER Cheng-Lung Sung # Enviroment ENV HOME /root ENV GOPATH /root/go ENV GOROOT /root/hg/go ENV PATH /root/go/bin:/root/hg/go/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/ bin # for source code RUN mkdir -p /root/go/src/clsung.tw # Development tools RUN rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm RUN yum -y groupinstall "Development Tools" RUN yum -y python pip # build Go RUN cd /root/go/src/clsung.tw && git clone https://github.com/clsung/django_poster RUN cd /root/go/src/clsung.tw && pip -r requirement.txt # set working directory (entrypoint root) WORKDIR /root/go/src/clsung.tw/django_poster ENTRYPOINT ['./manage.py', 'runserver']

Slide 62

Slide 62 text

Lesson learned: Put ﬁxed RUN cmds earlier

Slide 63

Slide 63 text

Docker repos

Slide 64

Slide 64 text

index.docker.io

Slide 65

Slide 65 text

Notes on MacOSX • Require boot2docker (linux) • VOLUME is useless on MacOSX • docker cp in running container • use vagrant

Slide 66

Slide 66 text

Fabric + Docker for deployment testing

Slide 67

Slide 67 text

• Replace virtualenv with docker in testing • Use docker to test backend (web and db) • After testing, convert Dockerﬁle to Salt File • Before docker 1.0 released Goal

Slide 68

Slide 68 text

run docker command in fabric @task def docker(cmd): """ Run Docker cmd """ return run("docker %s" % cmd)

Slide 69

Slide 69 text

clean container by name @task def clean_env(): with settings(warn_only=True): docker('kill simplehttpd') docker('rm simplehttpd') docker('rm client')

Slide 70

Slide 70 text

build images @task def build(): docker_ids = {} for d in ['httpd', 'client']: with lcd(d): result =local('docker build .', capture=True) # handy hacks: get the image id docker_ids[d] = result.stdout.split()[-1] return docker_ids

Slide 71

Slide 71 text

@task def pre_deploy(): clean_env() d_ids = build() docker('run --name simplehttpd -d {0}'.format( d_ids['httpd'])) with settings(warn_only=True): result = docker('run -i --rm --link simplehttpd:httpd ' '--name "client" {0}'.format( d_ids['client']), capture=True) if result.failed and not confirm("Tests failed. Continue anyway?"): abort("Aborting at user request.") logger.info(result.stdout) docker('kill simplehttpd')

Slide 72

Slide 72 text

FROM phusion/baseimage ENV HOME /root RUN /etc/my_init.d/00_regen_ssh_host_keys.sh CMD ["/sbin/my_init"] RUN apt-get update RUN apt-get install -y python EXPOSE 8000 ENTRYPOINT ["python", "-m", "SimpleHTTPServer", "8000"] httpd/Dockerﬁle

Slide 73

Slide 73 text

client/Dockerﬁle FROM phusion/baseimage ENV HOME /root RUN apt-get update RUN apt-get install -y python python-pip RUN pip install requests ADD ping.py /opt/ping.py CMD ["/usr/bin/python", "/opt/ping.py"]

Slide 74

Slide 74 text

db/ping.py #!/usr/bin/env python import requests resp = requests.get('http://httpd:8001/') resp.raise_for_status() print resp.status_code

Slide 75

Slide 75 text

$ fab pre_deploy [localhost] local: docker build . [localhost] local: docker run --name simplehttpd -d b6c6ab6afc4d 9c2515094c384099a4d867747e0a9de2f72b0244e86f0263b559e555 98dd4239 [localhost] local: docker run -i --rm --link simplehttpd:httpd --name "client" f546ae5530c5 INFO:fabfile:200 [localhost] local: docker kill simplehttpd simplehttpd Done.

Slide 76

Slide 76 text

if any error..... [localhost] local: docker build . [localhost] local: docker run --name simplehttpd -d b6c6ab6afc4d 97da8851573380116f83b9f11f7a884591ebfffa53875ce91defd7810359ba7b [localhost] local: docker run -i --rm --link simplehttpd:httpd --name "client" cc5951661843 Warning: local() encountered an error (return code 1) while executing 'docker run -i --rm --link simplehttpd:httpd --name "client" cc5951661843' Tests failed. Continue anyway? [Y/n] n Fatal error: Aborting at user request. Aborting.

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Other • Shipper - fabric for docker • https://github.com/mailgun/shipper • easy to move containers from a developer environment to a staging environment

Slide 79

Slide 79 text

Thank you!