Docker Cluster based on Raspberry Pi

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

23 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ wget http://downloads.hypriot.com/hypriot-rpi-20150416-201537.img.zip $ unzip hypriot-rpi-20150416-201537.img.zip # insert SD CARD $ diskutil list ... /dev/disk3 #: TYPE NAME SIZE IDENTIFIER 0: FDisk_partition_scheme *31.9 GB disk3 1: Windows_FAT_32 NO NAME 31.9 GB disk3s1 $ diskutil unmountdisk /dev/disk3 $ sudo dd if=hypriot-rpi-20150416-201537.img of=/dev/rdisk3 bs=1M 1220+1 records in 1220+1 records out 1280000000 bytes (1,3 GB) copied, 81,6065 s, 15,7 MB/s # 1,3 GB warten... 80 Sec $ diskutil unmountdisk /dev/disk3 # eject SD card, insert at your pi and boot your pi...

Slide 24

Slide 24 text

24 / 107 © 2015 , @PRossbach, IOT Berlin 2015 wget https://raw.githubusercontent.com/hypriot/flash/master/$(uname -s)/flas chmod +x flash sudo mv flash /usr/local/bin/flash brew install pv brew install awscli $ flash help ... --config|-c Copy this config file to /boot/occidentalis.txt --hostname|-n Set hostname for this SD image --ssid|-s Set WiFi SSID for this SD image --password|-p Set WiFI password for this SD image ... $ flash --hostname rpi-001 \ http://downloads.hypriot.com/hypriot-rpi-20150416-201537.img.zip

Slide 25

Slide 25 text

25 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ ssh root@rpi-001 > mkdir -f .ssh > chmod 700 .ssh > exit $ scp ${HOME}/.ssh/id_rsa.pub root@rpi-001:/root/.ssh/authorized_keys $ ssh root@rpi-001 > chmod 600 .ssh/authorized_keys

Slide 26

Slide 26 text

26 / 107 © 2015 , @PRossbach, IOT Berlin 2015 # update to current docker version $ wget http://downloads.hypriot.com/docker-hypriot_1.8.1-1_armhf.deb $ dpkg -i docker-hypriot_1.8.1-1_armhf.deb $ service docker restart $ docker version Client: Version: 1.8.1 API version: 1.20 Go version: go1.4.2 Git commit: d12ea79-dirty Built: Thu Aug 13 07:53:24 UTC 2015 OS/Arch: linux/arm Server: Version: 1.8.1 API version: 1.20 Go version: go1.4.2 Git commit: d12ea79-dirty Built: Thu Aug 13 07:53:24 UTC 2015 OS/Arch: linux/arm

Slide 27

Slide 27 text

27 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker info Containers: 2 Images: 84 Storage Driver: overlay Backing Filesystem: extfs Execution Driver: native-0.2 Logging Driver: json-file Kernel Version: 3.18.11-hypriotos-v7+ Operating System: Raspbian GNU/Linux 7 (wheezy) CPUs: 4 Total Memory: 925.1 MiB Name: black-pearl ID: V57A:JOG4:5GRL:WKXF:BL2H:ZCFS:MGIF:6M3V:XD2I:FS6M:TXGU:6HRF Labels: provider=hypriot

Slide 28

Slide 28 text

28 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker pull resin/rpi-raspbian:jessie $ docker run --rm -ti resin/rpi-raspbian > cat /etc/os-release PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)" NAME="Raspbian GNU/Linux" VERSION_ID="8" VERSION="8 (jessie)" ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Slide 29

Slide 29 text

29 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker search hypriot hypriot/rpi-node RPi-compatible Docker Image with Node.js hypriot/rpi-iojs RPi-compatible Docker Image with io.js hypriot/rpi-java RPi-compatible Docker Image with Java hypriot/rpi-python RPi-compatible Docker Image with Python hypriot/rpi-golang hypriot/rpi-busybox-httpd Raspberry Pi compatible Docker Image with ... hypriot/rpi-gpio Dockerized gpio binary of wiringPi for the ... hypriot/rpi-ruby RPi-compatible Docker Image with Ruby hypriot/rpi-hugo Raspberry Pi compatible Docker Image with ... hypriot/rpi-crate RPi-compatible Docker Image with Crate.io hypriot/rpi-drone Raspberry Pi compatible Docker Image with ... hypriot/rpi-alpine-scratch Raspberry Pi compatible Docker Image with ... hypriot/rpi-haproxy Haproxy Dockerfile hypriot/rpi-gogs-raspbian Raspberry Pi compatible Docker Image with ... hypriot/rpi-swarm Raspberry Pi compatible Docker image with ... hypriot/rpi-mysql RPi-compatible Docker Image with Mysql hypriot/rpi-gogs-alpine Raspberry Pi compatible Docker Image with ... gdelpu/rpi-hypriot-node-hapi NodeJs with HapiJs framework hypriot/rpi-nano-httpd Raspberry Pi nano sized HTTP server hypriot/skydns hypriot/rpi-redis hypriot/rpi-skydns

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

38 / 107 © 2015 , @PRossbach, IOT Berlin 2015 aptitude install python python-pip git clone https://github.com/docker/compose.git cd compose git checkout 1.4.0 pip install . $ cd /etc/bash-completion.d $ rm docker $ URL_BASE=https://raw.githubusercontent.com/docker $ wget $(URL_BASE)/docker/master/contrib/completion/bash/docker $ wget $(URL_BASE)/compose/master/contrib/completion/bash/docker-compose # login again! $ exit $ ssh root@rpi-001

Slide 39

Slide 39 text

39 / 107 © 2015 , @PRossbach, IOT Berlin 2015 FROM hypriot/rpi-java MAINTAINER Peter Rossbach @PRossbach ENV TOMCAT_MINOR_VERSION=8.0.26 \ CATALINA_HOME=/opt/tomcat \ TOMCAT_URL=http://archive.apache.org/dist/tomcat/tomcat-8 RUN DEBIAN_FRONTEND=noninteractive apt-get update \ && apt-get install -y \ curl \ --no-install-recommends \ && rm -rf /var/lib/{apt,dpkg,cache,log}/ ...

Slide 40

Slide 40 text

40 / 107 © 2015 , @PRossbach, IOT Berlin 2015 ... RUN curl -O ${TOMCAT_URL}/v${TOMCAT_MINOR_VERSION}/bin/apache-tomcat-\ ${TOMCAT_MINOR_VERSION}.tar.gz \ && curl ${TOMCAT_URL}/v${TOMCAT_MINOR_VERSION}/bin/apache-tomcat-\ ${TOMCAT_MINOR_VERSION}.tar.gz.md5 | md5sum -c - \ && tar xzf apache-tomcat-*.tar.gz \ && rm apache-tomcat-*.tar.gz* && mv apache-tomcat* ${CATALINA_HOME} \ && rm -rf ${CATALINA_HOME}/webapps/* \ ${CATALINA_HOME}/RELEASE-NOTES ${CATALINA_HOME}/RUNNING.txt \ ${CATALINA_HOME}/bin/*.bat ${CATALINA_HOME}/bin/*.tar.gz WORKDIR /opt/tomcat EXPOSE 8080 EXPOSE 8009 ENTRYPOINT [ "/opt/tomcat/bin/catalina.sh" ] CMD ["run"]

Slide 41

Slide 41 text

Slide 42

Slide 42 text

42 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ IP=$(docker inspect --format \ "{{ .NetworkSettings.IPAddress }}" tomcat_tomcat_1) $ curl -sL http://${IP}:8080/status hello Mon Aug 31 09:58:37 UTC 2015 $ docker exec -ti tomcat_tomcat_1 java -version java version "1.7.0_65" OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-2~deb7u1+rpi1) OpenJDK Zero VM (build 24.65-b04, mixed mode) $ sudo ntpdate -u ptbtime2.ptb.de docker pull infrabricks/tomcat:rpi-8.0.24

Slide 43

Slide 43 text

43 / 107 © 2015 , @PRossbach, IOT Berlin 2015 FROM resin/rpi-raspbian:jessie MAINTAINER Peter Rossbach @PRossbach RUN DEBIAN_FRONTEND=noninteractive apt-get update \ && apt-get install -y \ wget pwgen curl openjdk-8-jre \ --no-install-recommends \ && apt-get clean autoclean \ && apt-get autoremove -y \ && rm -rf /var/lib/{apt,dpkg,cache,log}/ WORKDIR /data ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-armhf ENTRYPOINT [ "/usr/bin/java" ] CMD ["-version"]

Slide 44

Slide 44 text

44 / 107 © 2015 , @PRossbach, IOT Berlin 2015 FROM resin/rpi-raspbian:jessie MAINTAINER Peter Rossbach @PRossbach RUN DEBIAN_FRONTEND=noninteractive \ echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" \ | tee -a /etc/apt/sources.list \ && echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" \ | tee -a /etc/apt/sources.list \ && apt-key adv --keyserver keyserver.ubuntu.com \ --recv-keys EEA14886 && apt-get update \ && echo oracle-java8-installer shared/accepted-oracle-license-v1-1 \ select true | debconf-set-selections \ && apt-get install -y oracle-java8-installer wget pwgen curl oracle-java8-installer \ --no-install-recommends \ && rm /usr/lib/jvm/java-8-oracle/src.zip \ && apt-get clean autoclean \ && apt-get autoremove -y \ && rm -rf /var/lib/{apt,dpkg,cache,log}/ RUN locale-gen en_US.UTF-8 ENV LANG en_US.UTF-8 ENV LC_CTYPE en_US.UTF-8 WORKDIR /data ENV JAVA_HOME /usr/lib/jvm/java-8-oracle ENTRYPOINT [ "/usr/bin/java" ] CMD ["-version"]

Slide 45

Slide 45 text

Slide 46

Slide 46 text

46 / 107 © 2015 , @PRossbach, IOT Berlin 2015 tomcat1: image: tomcat_tomcat volumes: - "./status:/opt/tomcat/webapps/status" tomcat2: image: tomcat_tomcat volumes: - "./status:/opt/tomcat/webapps/status" haproxy: image: hypriot/rpi-haproxy volumes: - "./haproxy:/haproxy-override" links: - tomcat1 - tomcat2 ports: - "8080:80" - "7070:70"

Slide 47

Slide 47 text

47 / 107 © 2015 , @PRossbach, IOT Berlin 2015 global log 127.0.0.1 local0 log 127.0.0.1 local1 notice defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 10000 timeout server 10000 listen stats :70 stats enable stats uri / ...

Slide 48

Slide 48 text

48 / 107 © 2015 , @PRossbach, IOT Berlin 2015 ... frontend balancer bind 0.0.0.0:80 mode http default_backend tc_backends backend tc_backends mode http option forwardfor # http-request set-header X-Forwarded-Port %[dst_port] balance roundrobin server tomcat1 tomcat1:8080 check server tomcat2 tomcat2:8080 check # option httpchk OPTIONS * HTTP/1.1\r\nHost:\ localhost option httpchk GET /status/index.jsp http-check expect status 200

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

55 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ wget http://downloads.hypriot.com/docker-machine_0.4.0-dev_darwin-amd64 $ chmod +x docker-machine_0.4.0-dev_darwin-amd64 $ ln -sf docker-machine_0.4.0-dev_darwin-amd64 docker-machine $ export PATH=$(pwd):$PATH

Slide 56

Slide 56 text

56 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker-machine create -d hypriot \ --hypriot-ip-address=192.168.178.109 \ rpi-001 # Update to newest docker engine! $ ssh [email protected] > wget http://downloads.hypriot.com/docker-hypriot_1.8.1-1_armhf.deb > dpkg -i docker-hypriot_1.8.1-1_armhf.deb > service docker restart > exit

Slide 57

Slide 57 text

57 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ cd ~/bin $ docker-machine ls INFO[0000] Creating CA: /Users/peter/.docker/machine/certs/ca.pem INFO[0001] Creating client certificate: /Users/peter/.docker/machine/certs/c NAME ACTIVE DRIVER STATE URL $ ls -a .docker/machine/certs ca-key.pem ca.pem cert.pem key.pem server-key.pem server.pem

Slide 58

Slide 58 text

58 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker-machine ls NAME ACTIVE DRIVER STATE URL rpi-master virtualbox Running tcp://192.168.99.100:2376 (master) rpi-001 * hypriot Running tcp://192.168.99.109:2376 $ docker-machine config rpi-001 --tlsverify --tlscacert="/Users/peter/.docker/machine/machines/rpi-001/ca.pem" --tlscert="/Users/peter/.docke $ docker $(docker-machine config rpi-001) ps $ docker $(docker-machine config rpi-001) run \ --rm hypriot/rpi-alpine-scratch /bin/sh -c "echo hello machine \$(hostname)" Unable to find image 'hypriot/rpi-alpine-scratch:latest' locally hypriot/rpi-alpine-scratch:latest: The image you are pulling has been verified 511136ea3c5a: Pull complete df7546f9f060: Pull complete ea13149945cb: Pull complete 4986bf8c1536: Pull complete Status: Downloaded newer image for busybox:latest hello machine 117ade53e1d6 $ docker $(docker-machine config rpi-001) images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE hypriot/rpi-alpine-scratch latest 9566b98023b5 8 weeks ago 4.971 MB

Slide 59

Slide 59 text

59 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker-machine ssh rpi-001 Linux rpi-001 3.18.11-hypriotos-v7+ #2 SMP PREEMPT Sun Apr 12 16:34:20 UTC 2 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Mon Aug 31 12:47:02 2015 from heisenberg-2.fritz.box HypriotOS: root@rpi-001 in ~

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Slide 65

Slide 65 text

Slide 66

Slide 66 text

66 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ docker pull swarm # docker pull hypriot/rpi-swarm $ docker run --rm swarm create $ docker run -d -p 3376:2376 swarm manage \ token:// $ docker run -d swarm join \ --addr= token://

Slide 67

Slide 67 text

67 / 107 © 2015 , @PRossbach, IOT Berlin 2015 # TOKEN=$(docker run --rm -ti swarm create) $ docker-machine create -d hypriot \ --hypriot-ip-address=192.168.178.94 \ --swarm \ --swarm-image "hypriot/rpi-swarm" \ --swarm-discovery token://${TOKEN} \ rpi-master $ ssh [email protected] > wget http://downloads.hypriot.com/docker-hypriot_1.8.1-1_armhf.deb > dpkg -i docker-hypriot_1.8.1-1_armhf.deb > service docker restart > exit

Slide 68

Slide 68 text

68 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ ssh root@rpi-001 $ docker pull hypriot/rpi-golang $ mkdir -p swarm $ docker run -ti -v $(pwd)/swarm:/app hypriot/rpi-golang $ go get -d github.com/docker/swarm $ go get github.com/tools/godep $ cd $GOPATH/src/github.com/docker/swarm $ CGO_ENABLED=0 GOOS=linux godep go build -a -tags netgo -ldflags '-w' . $ cp swarm /app/swarm # if work.. $ exit

Slide 69

Slide 69 text

Slide 70

Slide 70 text

Slide 71

Slide 71 text

Slide 72

Slide 72 text

Slide 73

Slide 73 text

73 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ eval $(docker-machine env rpi-master) $ docker run -c 1 -p 9082:8080 -d infrabricks/tomcat:rpi-8.0.24 35d50a29ed0fde780c0826ef16ff24d2582609f877ee092c780d9aeb6f27fc75 $ docker ps CONTAINER ID IMAGE COMMAND CREATED \ STATUS PORTS NAMES 35d50a29ed0f infrabricks/tomcat:rpi-8.0.24 "catalina.sh run" 16 minutes ago\ Up Less than a second 192.168.33.89:9082->8080/tcp \ rpi-001/insane_brattain $ docker run -c 1 -p 9083:8080 -d infrabricks/tomcat:rpi-8.0.24 f5bc6a274e82426df98eaca5e88fb01f43e03455dfebdb9ee4a3b00a7eab202e $ docker ps CONTAINER ID IMAGE COMMAND CREATED\ STATUS PORTS NAMES f5bc6a274e82 infrabricks/tomcat:rpi-8.0.24 "catalina.sh run" 16 minutes ago\ Up Less than a second 192.168.33.90:9083->8080/tcp \ rpi-002/stupefied_albattani 35d50a29ed0f infrabricks/tomcat:rpi-8.0.24 "catalina.sh run" 17 minutes ago\ Up Less than a second 192.168.33.89:9082->8080/tcp \ rpi-001/insane_brattain ...

Slide 74

Slide 74 text

74 / 107 © 2015 , @PRossbach, IOT Berlin 2015 lsb_release -a DOCKER_OPTS="--label region=de-berlin \ --label storage=ssd-vdi \ --label role=loadbalancer" docker -d ${DOCKER_OPTS} --label stage=prod docker run -e "constraint:provider==hypriot" ... docker run -e "constraint:storagedriver==aufs" ... docker run -e "constraint:region==de-berlin" ... docker run -e "constraint:node!=rpi-master" ... constraint:node==/rpi-00[12]/ constraint:node!=app* constraint:node==/(?i)rpi-001/

Slide 75

Slide 75 text

75 / 107 © 2015 , @PRossbach, IOT Berlin 2015 docker run -d -p 80:80 --name front nginx docker run -e "affinity:container==front" logger docker run -e "affinity:image==redis" redis docker run -d -p 80:80 --label com.example.type=frontend nginx docker run -d -e affinity:com.example.type==frontend logger docker run -d --name redis1 -e affinity:image==~redis redis docker run -d --name redis2 -e constraint:region==~us* redis docker run -d --name redis5 -e affinity:container!=~redis* redis

Slide 76

Slide 76 text

Slide 77

Slide 77 text

77 / 107 © 2015 , @PRossbach, IOT Berlin 2015 DOT=XYZ docker pull swarm SWARM_TOKEN=$(docker run --rm swarm create) echo $SWARM_TOKEN >token.txt docker-machine create \ --driver digitalocean \ --digitalocean-region fra1 \ --digitalocean-size 1gb \ --digitalocean-access-token $DOT \ --digitalocean-private-networking \ --swarm \ --swarm-master \ --swarm-discovery token://$SWARM_TOKEN \ --engine-label zone=prod \ --engine-label swarm=master \ swarm-master

Slide 78

Slide 78 text

78 / 107 © 2015 , @PRossbach, IOT Berlin 2015 docker-machine create \ --driver digitalocean \ --digitalocean-region fra1 \ --digitalocean-size 1gb \ --digitalocean-access-token $DOT \ --digitalocean-private-networking \ --swarm \ --swarm-discovery token://$SWARM_TOKEN \ --engine-label zone=prod \ --engine-label swarm=node \ swarm-node-001

Slide 79

Slide 79 text

Slide 80

Slide 80 text

Slide 81

Slide 81 text

81 / 107 © 2015 , @PRossbach, IOT Berlin 2015 $ ssh root@rpi-master $ git clone https://github.com/hypriot/rpi-consul $ cd rpi-consul $ make $ cd .. $ mkdir -p orchestration $ cat >docker-compose.yml <

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

84 / 107 © 2015 , @PRossbach, IOT Berlin 2015 FROM hypriot/rpi-alpine-scratch ENTRYPOINT ["/bin/registrator"] RUN apk update \ && apk add build-base go git mercurial \ && mkdir -p /go/src/github.com/gliderlabs && cd /go/src/github.com/glide && git clone https://github.com/gliderlabs/registrator \ && cd registrator \ && export GOPATH=/go \ && go get \ && go build -ldflags "-X main.Version $(cat VERSION)" -o /bin/registrato && rm -rf /go \ && apk del --purge build-base go git mercurial

Slide 85

Slide 85 text

Slide 86

Slide 86 text

86 / 107 © 2015 , @PRossbach, IOT Berlin 2015 tomcat: image: infrabricks/tomcat:rpi-8.0.24 ports: - "8080" volumes: - tomcat-users.xml:/opt/tomcat/conf/tomcat-users.xml environment: - constraint:zone==dev - constraint:swarm==node SERVICE_8080_NAME: status-http SERVICE_REGION: bee42.1 SERVICE_8080_CHECK_HTTP: /status/ping.jsp SERVICE_8080_CHECK_INTERVAL: 30s