Better DevOps With Docker & MySQL Sunny Gleason Distributed Systems Engineer, SunnyCloud April 15, 2015 

Who am I? 2 • Sunny Gleason – Distributed Systems Engineer – SunnyCloud, Boston MA • Prior Web Services Work – Amazon – Ning • Focus: Scalable, Reliable Storage Systems for Structured & Unstructured Data 

What’s this all about? • Why care about Databases & DevOps? • What benefits does Docker provide? • How can we use Docker with MySQL? 3 

How does DevOps relate to Databases? • Development-Oriented: DB is an empty shell / skeleton schema that enables features to provide new business value • Operations-Oriented: DB is a living organism that provides existing business value 4 

Development-Focused Perspective 5 Sources: http://support.smartbear.com/ http://www.docbyte.com/en/blog/integrating-gwt-with-spring-and-hibernate • Business/Product defines data needs • Developer maps it into a schema • Developer implements features using whatever app implementation technologies • “Somebody else” takes care of releases, monitoring, escalation 

Development-Focused Perspective 6 Sources: http://support.smartbear.com/ • Model a schema • Define tables & relations • Make queries easy 

Development-Focused Perspective 7 Sources: http://www.docbyte.com/en/blog/integrating-gwt-with-spring-and-hibernate • Write the feature • Create Value Objects and
 Data Access Objects • Minimize lines of code 

Development-Focused Perspective 8 Source: http://www.cumulogic.com/why-a-microservice-architecture-needs-dbaas/ • Create logical/physical
 services • Encapsulate based on
 domain area / vertical • Minimize number of
 systems 

What’s missing? 9 Source: http://stayhard.typepad.com/development/2013/08/octopus-deployments-in-new-relic.html • Deployment • Release Management
 & Change Management • Monitoring / Alerting • Scaling & Performance 

Operations-Focused Perspective 10 Source: https://www.mysql.com/products/workbench/ • Instantiate & Deploy
 systems • Manage environments • Hosts & Databases • Storage management • Replication & Backups 

Operations-Focused Perspective 11 Source: http://www.slideshare.net/aca_it/modularity-ddd • Coordinate schema
 migrations • Manage replication • Minimize downtime • Maximize performance 

Operations-Focused Perspective 12 Source: http://ronaldbradford.com/images/blog/newrelic-summary1.png • Create views • Monitor & Alert on
 key metrics • Inform scaling &
 perf optimization 

Operations-Focused Perspective 13 Source: https://databasesincloud.wordpress.com/2011/11/12/scaling-oracle-databases-on-amazon-rds/ • Do initial scale plan • Monitor capacity
 constraints • Perform capacity
 updates • Continuous feedback
 loop 

Where does DevOps fit in? 14 Source: http://blog.appdynamics.com/devops/devops-scares-me-part-2/ • “Developer folks” should be
 more operations-aware • “Systems folks” should be
 more development-aware • Goal: Everyone should be more
 responsive to the business & market 

How can Docker help? 15 • Docker is not DevOps • Docker will not do your laundry • Docker will let you create standard images • Docker will let you deploy & run versioned images • Docker is lightweight enough for developers to use • Docker is powerful enough to solve many needs • Docker will not coordinate your systems 

What is the core idea of Docker? 16 Traditional Virtualization
 (Xen, VMWare, etc.) Source: https://www.docker.com/ 

What does Docker provide? 17 • Container System • Container Format • Container Buildfiles • Container Versioning • Repository API • Deployment Mechanism • Container Runtime • Virtualization via libvirt • Container Library • Cross-platform support 

How do we work with Docker? 18 Source: http://blog.octo.com/en/docker-registry-first-steps/ 

Docker Gaps / Gotchas 19 • Still maturing • Rapid release cycle • Network configuration • No Easy Introspection • No “dom0” standard • Cross-platform is new • Security • Troubleshooting • I/O-Intensive Deploy • No Service Discovery • Differing Integrations • Orchestration 

What’s Docker’s value proposition? 20 • Ability to collaborate & version standard images • Easier deployment across environments / stages • Developer-friendly installation & usage • Operations-friendly controls & management 

Getting Started with Docker 21 • Check out: http://docs.docker.com/installation/ • Install Docker on Linux using something like:
 $ wget -qO- https://get.docker.com/ | sh • (Install Boot2docker or Kitematic on OS X) • Deploy Containers using “docker run” • A somewhat-authoritative MySQL image is at:
 https://registry.hub.docker.com/_/mysql/ 

Docker Summed Up in 1 Command 22 
 docker run --name some-mysql 
 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag • Downloads image with version “mysql:tag” • Creates a new container called “some-mysql” • Starts the container process using the default endpoint 

First Commands With Docker 23 • docker pull : downloads an image (makes available for use) • docker run : creates a container from an image • docker start / stop / restart : container process control • docker save : saves container state as a tar file • docker load : loads container state from a tar file • docker build : builds a new image from a Dockerfile • docker commit : creates a new revision of an image from a container • docker tag : associates a tag with a commit (think: git) • docker attach : attach console to running container • docker logs : fetch the logs of a container 

FROM debian:wheezy 
 RUN groupadd -r mysql && useradd -r -g mysql mysql RUN apt-get update && apt-get install -y perl --no-install-recommends 
 && rm -rf /var/lib/apt/lists/* RUN apt-key adv --keyserver pool.sks-keyservers.net \
 --recv-keys A4A9406876FCBD3C456770C88C718D3B5072E1F5
 ENV MYSQL_MAJOR 5.7 ENV MYSQL_VERSION 5.7.7-rc 
 RUN echo "deb http://repo.mysql.com/apt/debian/ wheezy mysql-$ {MYSQL_MAJOR}-dmr"
 > /etc/apt/sources.list.d/mysql.list 
 ...
 MySQL 5.7 Dockerfile (part 1) 24 Source: https://raw.githubusercontent.com/docker-library/mysql/master/5.7/Dockerfile 

MySQL 5.7 Dockerfile (part 2) 25 Source: https://raw.githubusercontent.com/docker-library/mysql/master/5.7/Dockerfile ... 
 # comment out a few problematic configuration values RUN sed -Ei 's/^(bind-address|log)/#&/' /etc/mysql/my.cnf VOLUME /var/lib/mysql COPY docker-entrypoint.sh /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] EXPOSE 3306 CMD ["mysqld"] 

Keep in mind: file locations • Docker stores files in /var/lib/docker • Containers live in “containers” dir • Images live in “graph” • You can inspect/modify file systems
 (at own peril) 26 

Keep in mind: users & permissions • Docker containers tend to run entry points as root • In mysql case, uses mysql user • User-level security is not 100% 27 

Keep in mind: externalize volumes • Docker containers are self-contained by default • External volumes can be mounted using the
 -v /source/external:/source/internal
 argument to docker run • This is important for: data dir, log dir, possibly config dir 28 

Keep in mind: network ports • Each docker container has its own virtual ip • Ports are mapped from internal to external using 
 -P 3306:3333
 argument to docker run • There will be port conflicts to manage 29 

Keep in mind: entry points • Each docker container starts with a default entry point • This is usually the only process running in the container • To do more, need to use a supervisor process or / bin/bash as entry point • This affects your ability to inspect inside the container using the default attach command 30 

Keep in mind: storage management • The default docker directory is
 /var/lib/docker • Consider mounting /var/lib/docker as an independent device using /etc/fstab or configure a docker directory for the docker daemon • It’s tricky to analyze the storage requirements of images since revisions are shared 31 

What does this mean for development? • Ability to use Docker on Linux, OS X • Provides mechanism for image versioning and easier provisioning / running of containers • Development environments are less unique, more throwaway • Ability to run multiple container instances 32 

What does this mean for staging/test? • Staging and test can be closer to development • Staging and test environments are less unique, more throwaway • Ability to run multiple container instances • Point in time activation / deactivation of containers 33 

What does this mean for production? • Production can be closer to development • Production environments are more throwaway • Ability to run multiple container instances • Ability to deploy software updates more easily on same hardware (different containers using same data volume) 34 

What’s next? • Better cross-platform support • Better service discovery & network config • Better orchestration • Wider range of integrations (AWS, Google, …) 35 

What’s going to be tricky for a while? • Security • Performance & process isolation • Resource management • “dom0” Provisioning • Network configuration 36 

Thank You! 37