Docker Use Case in Production

Slide 1

Slide 1 text

DOCKER USE CASE IN PRODUCTION (USING COREOS AND DOCKER COMPOSE) 商用DOCKER 利用事例 (COREOS/COMPOSE を利用して) 2016-03-05 Docker Meetup Tokyo #6 by / Takuya Noguchi @tn961ir

Slide 2

Slide 2 text

@TNIR Takuya Noguchi ( 野口卓也) Software Engineer iRidge, Inc. Twitter: @tn961ir Facebook: takuya.noguchi.961 LinkedIn: takuyanoguchi

Slide 3

Slide 3 text

IRIDGE, INC. an O2O solution company established in 2008 50+ members (incl. 20+ engineers)

Slide 4

Slide 4 text

MY WORKS OEM to private IaaS (a Telecom) BI system development for geo-location data API service development for partners' integration Mobile payment service "C-less" Operations & Monitoring PyCon APAC & JP 2013/14/15 (sponsor) Resource management (cost, code, security, performance)

Slide 5

Slide 5 text

PROBLEMS ON DEVELOPMENT Cost, Ops, QA & security Development process (e.g. multiple deployments on a single codebase) Multicloud platforms (AWS, GCP, CloudStack, OpenStack, other IaaSs)

Slide 6

Slide 6 text

[SOL] OPS & COST 0 infra enginneers (vs. 12 server-side eng) Started to use Google App Engine (2009) Moved to AWS (Tokyo) (2011) Moved to GCP (APAC) (2014) GCP が支える、次世代 O2O ソリューション [cloud.google.com/?hl=ja]

Slide 7

Slide 7 text

[PROBLEM] DEVELOPMENT PROCESS Repositories: Subversion (Trac), Mercurial (hg) -> Git Code review, CI/CD (Jenkins) Multiple deployments on a single codebase

Slide 8

Slide 8 text

CANDIDATES FOR CODE & CI Code BitBucket GitLab GitHub Enterprise CI Jenkins wercker GitLab CI

Slide 9

Slide 9 text

ALL-IN-ONE SOLUTION GitLab (OSS) fork code review CI/CD Docker support

Slide 10

Slide 10 text

ALL-IN-ONE SOLUTION (2) Says "GitHub Enterprise for the poor" Good oppotunities for us: to unify development/deployment process to customize for me to catch up Rails stack :-)

Slide 11

Slide 11 text

CI WITH DOCKER . g i t l a b - c i . y m l : p y t e s t : 　i m a g e : u b u n t u : 1 4 . 0 4 　s e r v i c e s : 　　- m y s q l : 5 . 6 . 2 8 　s c r i p t : 　　- s c r i p t s / b o o t s t r a p . s h 　　- p y . t e s t p y t e s t : d j a n g o 1 9 m y s q l 5 7 : 　i m a g e : u b u n t u : 1 4 . 0 4 　s e r v i c e s : 　　- m y s q l : 5 . 7 . 1 1 　s c r i p t : 　　- s c r i p t s / b o o t s t r a p . s h 　　- p i p i n s t a l l D j a n g o = = 1 . 9 . 3 　　- p y . t e s t

Slide 12

Slide 12 text

DOCKER FOR ALL ENGINEERS Advertise Docker for all our engineers CI status notifications to Slack TODO: in-house seminars

Slide 13

Slide 13 text

DOCKER (CI) FOR ANDROID/IOS Docker (CI) for iOS/Android Engineers QA enmtknt posted in Feb 26, 2016 TODO: Docker for our Android projects GitLab CI でXcode プロジェクトをCI する手順

Slide 14

Slide 14 text

DOCKER IN PRODUCTION

Slide 15

Slide 15 text

[PROBLEM] CONVENTIONAL ARCHITECUTRE Microservices Several services a person (previously) web2.py on GAE (no Asia region!!!) (current) Python 2/Django on Ubuntu 12.04/14.04 deb/pip package dependency vagrant/VirtualBox development environment problems Fabric/Ansible for deployment EC2 on AWS used like on-premise All eng in charge doesn't have time for/interest in Docker

Slide 16

Slide 16 text

[SOLUTION] NEW ARCHITECUTRE Golang latest (1.4.x) Docker Cloud native Google Cloud Platform

Slide 17

Slide 17 text

CONSIDERATIONS Build (CI) Deployment (CD) OS/Platform Image Management Container Management Logging/Monitoring

Slide 18

Slide 18 text

DEPLOYMENT

Slide 19

Slide 19 text

WHEN TO DEPLOY midnight (once a week) daytime (every day we want)

Slide 20

Slide 20 text

BUILD & DEPLOYMENT Fig (Docker Compose) with Ansible CI: GitLab CI with Docker support

Slide 21

Slide 21 text

OS/PLATFORM CANDIDATES Ubuntu CoreOS k8s on GCE Amazon ECS (Apr 2015 GA in Tokyo) GCP GKE (Aug 2015 GA) (Project Atomic) (Snappy Ubuntu)

Slide 22

Slide 22 text

ADVANTAGES ON COREOS minimal OS footprint (lightweight!) cluster -> unused etcd -> unused fleet -> unused automatic update -> off

Slide 23

Slide 23 text

COREOS RELEASE CHANNELS 3 Channels Stable (1.8) // Beta (1.9) // Alpha (1.10)

Slide 24

Slide 24 text

coreos.com/os/docs/latest/update-strategies.html

Slide 25

Slide 25 text

COREOS RELEASE CHANNELS (2) The Company prefer stability(!) while I prefer the cutting edge. Mixed channels I chose: Beta (1.9) for 50% hosts Stable (1.8) for 50% hosts Alpha (1.10) for 0%

Slide 26

Slide 26 text

REBOOT STRATEGIES Update is fully manual. # c l o u d - c o n f i g c o r e o s : 　u p d a t e : 　　r e b o o t - s t r a t e g y : o f f

Slide 27

Slide 27 text

PART OF ARCHITECTURE OVEWVIEW

Slide 28

Slide 28 text

IMAGE MANAGEMENT Private image management Docker Registry Docker Hub Quay.io ✓ G Container Repository ✓ Build on each node with Dockerfile + Blue-Green deployment

Slide 29

Slide 29 text

CONTAINER MANAGEMENT Docker Compose + Ansible num of containers a VM ✓ 1 container a VM ✓ 2 containers a VM Connections via HTTPS over VMs

Slide 30

Slide 30 text

OPS AFTER DEPLOYMENTS ON PRODUCTION Did almost nothing after the deployment Except for CoreOS update Manual CoreOS update a month Logging: default (j s o n - f i l e ) Heavy disk usage d o c k e r l o g s is slow TODO: f l u n t d driver to forward to G Cloud Logging

Slide 31

Slide 31 text

MONITORING Datadog Mackerel ✓ Google Cloud Monitoring (Stackdriver before 2014) ✓ Munin etc.

Slide 32

Slide 32 text

DOCKER SUMMARY Docker was simply introduced and it has been served in production for more than 1.5 yrs as well. Docker is also utilized for development and CI. Cloud vendor-unlocked! Docker Compose for production still works for our service.

Slide 33

Slide 33 text

ARCHIVED GOALS?

Slide 34

Slide 34 text

SERVERLESS ARCHITECTURE Again, 0 infra/ops engineers in our company. PaaS? App Engine (Asia or Tokyo) AWS Lambda G Cloud Functions (Alpha)