Tinkering To Production

n this talk, Martin will explain how over the last two years I have made Docker and Kubernetes part of my web development workflow. I will show with examples and code, the lessons I have learned and how I settled on a DevOps workflow using a build tool called Wercker to put Docker Based Web applications into production.

Transcript

1. Martin Beeby - @thebeebs TO PRODUCTION FROM TINKERING
2. Martin Beeby @thebeebs Five years Community Sharing DOCKER
3. Martin Beeby @thebeebs Software/Web developer Since I was 16 Joined Oracle 6 months ago MARTIN BEEBY
4. Martin Beeby @thebeebs Tim Berners-Lee's At CERN Web Invented 0 DAYS DECEMBER 20, 1990
5. Martin Beeby @thebeebs 9960 DAYS 28 MARCH 2018
6. Martin Beeby @thebeebs 10,000 DAYS 7 MAY 2018
7. Martin Beeby @thebeebs CAR DAY 10,000
8. Martin Beeby @thebeebs TV DAY 10,000
9. Martin Beeby @thebeebs ME DAY 10,000
10. Martin Beeby @thebeebs DOCKER IS DAY 1825
11. Martin Beeby @thebeebs
12. Martin Beeby @thebeebs KUBERNETES IS DAY 1348
13. Martin Beeby @thebeebs Open Source, Container-based system Wraps up the application & all dependencies into a software container Automates the deployment of applications WHAT IS DOCKER Independent of underlying OS of the host system
14. Martin Beeby @thebeebs FTP > VMS > CI/CD > DOCKER
15. Martin Beeby @thebeebs Scale up containers Container Cluster Management System DevOps focused WHAT IS KUBERNETES? Originally designed by Google @NLad95
16. Martin Beeby @thebeebs DOCKERFILE Instruction Description COPY Copies a file from the host system onto the container CMD The command that runs when the container starts ENV Sets an environment variable in the new container EXPOSE Opens a port for linked containers FROM The base image to use in the build. Mandatory and must be the first command ONBUILD A command that is triggered when the image in the Dockerfile is used as a base for another image RUN Executes a command and saves the results as a new layer VOLUME Creates a shared volume that can be shared among containers by host machine WORKDIR Set the default working directory for the container
17. Martin Beeby @thebeebs # comment INSTRUCTIONS arguments # our base image FROM python:3-onbuild # specify the port number the container should expose EXPOSE 5000 # run the application! CMD [“python”, “./myapp.py”] BASIC FILE
18. Martin Beeby @thebeebs $ docker build –t <userName>/<appName> Sending build context to Docker daemon 8.704 kB Step 1 : FROM python:3-onbuild # Executing 3 build triggers…… Step 1 : COPY requirements.txt /usr/src/app/ ---> Using cache Step 1 : COPY ./usr/src/app ---> 1d6e32hrwie35 Removing internmediate container 45jfedf7wegedd Step 2 : EXPOSE 5000 ---> Running in 12cfcewrij87eds ---> 7we62fehk89ke Removing intermediate container 12cfcewrij87eds Step 3 : CMD python ./app.py ---> Running in f06e63ij87eds ---> 9ghf8fehk89ke Removing intermediate container f06e63ij87eds Successfully built 13e8796fbc4 DOCKER BUILD
19. Martin Beeby @thebeebs $ docker run –p 8888:5000 <userName>/<appName> * Running on http://0.0.0.0:5000/ (Press CTRL+ to quit) DOCKER RUN
20. Martin Beeby @thebeebs EXAMPLE COMMANDS: docker pull python:2.7 Pull image from DockerHub docker images Shows all images docker rmi python:2.7 Remove an image docker ps Show only running containers docker ps -a Shows all containers docker build -t <image_name> Create your own image docker logs -f <container_name> Check the logs of container docker load -i <CONTAINER_FILE>.tar Restoring docker container docker rm $(docker ps –a -q) Deletes all containers but fails on the ones running docker history [OPTIONS] IMAGE Shows the effect of each command on overall size of file
21. Martin Beeby @thebeebs CONFIGURE DOCKER PS OUTPUT • Default output of docker ps • This output takes up too much room – can get annoying! $ docker ps CONTAINER ID IMAGE COMMAND …… 247wh3jddee0 nginx “nginx –g daemon” …… @NLad95
22. Martin Beeby @thebeebs • Make the output in permanent format by setting an alias in config.json $ cat ~/.docker/config.json {…… “psFormat”: “table {{.Names}}\\t{{.Image}}\\t{{.Status}}” } $ docker ps --format \ “table {{.Names}}\\t{{.Image}}\\t{{.Status}}” NAMES IMAGE STATUS web nginx Up 20 minutes • It can be fixed! • Solution is to use --format argument CONFIGURE DOCKER PS OUTPUT @NLad95
23. Martin Beeby @thebeebs USEFUL COMMANDS • Deleting “dangling” images • <none> images $ docker image prune WARNING! This will remove all dangling images Are you sure you want to continue? [y/N] y Deleted images: deleted: 23j2j3kmcerkeddne09eea3goed232nfks.. …… Total reclaimed space: 3GB @NLad95
24. Martin Beeby @thebeebs USEFUL COMMANDS • Delete stopped containers $ docker container prune WARNING! This will remove all stopped containers Are you sure you want to continue? [y/N] y Deleted Containers: deleted: 23j2j3kmcerkeddne09eea3goed232nfks.. …… Total reclaimed space: 300MB @NLad95
25. Martin Beeby @thebeebs STARTING UP A CONTAINER • Do not require containers to start in sequence • If a container depends on another service i.e. a database • It should wait for that service • The container should not crash • Container to wait • Do this in the application code/start up script @NLad95
26. Martin Beeby @thebeebs HEALTH CHECKS • Used by docker to determine the “health” of a container FROM nginx RUN apt-get update && apt-get install –y curl HEALTHCHECK --interval=10s –timeout=3s \ CMD curl –f http://localhost/ || exit 1 @NLad95
27. Martin Beeby @thebeebs AN APP WERCKER KUBERNETES
28. Martin Beeby @thebeebs
29. Martin Beeby @thebeebs HOW HAPPY?
30. Martin Beeby @thebeebs SIMPLE APP External API Microservice API x 2 Container – Node.JS Port 3000 Endpoint Service IP Load Balancer Port 80 Front End Container – Oracle Jet x 3 Port 80 Endpoint Service IP Load Balancer Port 80 Redis Master Redis Slave x 2
31. Martin Beeby @thebeebs apiVersion: v1 kind: Service metadata: name: frontend labels: app: audienceanalyser tier: frontend spec: type: LoadBalancer ports: - port: 80 selector: app: audienceanalyser tier: frontend --- KUBERNETES YML - SERVICE
32. Martin Beeby @thebeebs FRONT END Endpoint Service IP Load Balancer Port 80
33. Martin Beeby @thebeebs --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: frontend spec: replicas: 3 template: metadata: labels: app: audienceanalyser tier: frontend spec: containers: - name: beebs image: thebeebs/howareyou:THIS_STRING_IS_REPLACED_DURING_BUILD imagePullPolicy: Always resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns ports: - containerPort: 80 KUBERNETES YML
34. Martin Beeby @thebeebs FRONT END Front End Container – Oracle Jet x 3 Port 80 Endpoint Service IP Load Balancer Port 80
35. Martin Beeby @thebeebs SIMPLE APP AI Emotion Service Microservice API x 2 Container – Node.JS Port 3000 Endpoint Service IP Load Balancer Port 80 Front End Container – Oracle Jet x 3 Port 80 Endpoint Service IP Load Balancer Port 80 Redis Master Redis Slave x 2
36. Martin Beeby @thebeebs Simple Integration and deployment Free for basic use WHY WERCKER
37. Martin Beeby @thebeebs FRONT END Front End Container – Oracle Jet x 3 Port 80
38. Martin Beeby @thebeebs http://github.com/thebeebs/howareyou ORACLE JET APP
39. Martin Beeby @thebeebs http://www.oracle.com/technetwork/developer-tools/jet/ SINGLE PAGE APP FRAMEWORK
40. Martin Beeby @thebeebs ADD APPLICATION
41. Martin Beeby @thebeebs Applications have workflows Workflows connect pipelines Pipelines contain steps CONNECTING PIPES
42. Martin Beeby @thebeebs box: node WERCKER YAML
43. Martin Beeby @thebeebs build: steps: - npm-install - npm-test - script: name: oracle jet build code: | npm install -g @oracle/ojet-cli ojet build - script: name: copy code to output code: cp -r web "$WERCKER_OUTPUT_DIR" - script: name: copy yml file to output code: cp service.yml "$WERCKER_OUTPUT_DIR" BUILD - PIPELINE
44. Martin Beeby @thebeebs steps: - npm-install - npm-test - script: name: oracle jet build code: | npm install -g @oracle/ojet-cli ojet build - script: name: copy code to output code: cp -r web "$WERCKER_OUTPUT_DIR" STEPS
45. Martin Beeby @thebeebs push-release: # Override the node:4-slim box box: id: nginx:alpine cmd: /bin/sh steps: - script: name: list whats on disk code: | echo "Listings files on disk" ls cd web echo "Checing whats in index" cat index.html - script: name: mv static files code: | echo "CAT Index file" cat $WERCKER_SOURCE_DIR/web/index.html ls $WERCKER_SOURCE_DIR/web/ rm -rf /usr/share/nginx/html/* mv web/* /usr/share/nginx/html PUSH-RELEASE - PIPELINE - script: name: list file code: | echo "ls files:" ls - internal/docker-push: disable-sync: true repository: thebeebs/howareyou username: $DOCKER_USERNAME password: $DOCKER_PASSWORD registry: https://registry.hub.docker.com/v2 tag: $WERCKER_GIT_COMMIT cmd: nginx -g 'daemon off;'
46. Martin Beeby @thebeebs - internal/docker-push: disable-sync: true repository: thebeebs/howareyou username: $DOCKER_USERNAME password: $DOCKER_PASSWORD registry: https://registry.hub.docker.com/v2 tag: $WERCKER_GIT_COMMIT cmd: nginx -g 'daemon off;' VARIABLES
47. Martin Beeby @thebeebs https://app.wercker.com DECLARED IN THE APP
48. Martin Beeby @thebeebs deploy: steps: - script: name: list file code: | echo "ls files:" ls - script: name: update image build code: sed -ie "s/THIS_STRING_IS_REPLACED_DURING_BUILD/$WERCKER_GIT_COMMIT/g" service.yml - script: name: show file code: cat service.yml - kubectl: server: $KUBERNETES_MASTER username: $KUBERNETES_USERNAME password: $KUBERNETES_PASSWORD insecure-skip-tls-verify: true command: apply -f service.yml DEPLOY - PIPELINE
49. Martin Beeby @thebeebs --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: frontend spec: replicas: 3 template: metadata: labels: app: audienceanalyser tier: frontend spec: containers: - name: beebs image: thebeebs/howareyou:THIS_STRING_IS_REPLACED_DURING_BUILD imagePullPolicy: Always resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns ports: - containerPort: 80 KUBERNETES YML
50. Martin Beeby @thebeebs ADD PIPELINE
51. Martin Beeby @thebeebs WORKFLOWS
52. Martin Beeby @thebeebs CHECK IN TO DEPLOY
53. Martin Beeby @thebeebs Kubernetes Bare Metal / VM / Hybrid Highly Available CLUSTERS TECHNICAL PREVIEW
54. Martin Beeby @thebeebs HTTP://THEBEEBS.UK/WERCKER
55. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Call For Papers: developer.oracle.com/code/cfp Registration Now Open: https://developer.oracle.com/code ORACLE CODE STEP UP TO MODERN CLOUD DEVELOPMENT JUNE 12TH, BERLIN #OracleCode
56. Martin Beeby @thebeebs THEBEEBS.UK/TRIAL
57. Martin Beeby @thebeebs CONTACT MARTIN BEEBY @thebeebs https://www.thebeebs.co.uk martin.beeby@oracle.com