Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Better DevOps with MySQL and Docker

Better DevOps with MySQL and Docker

Docker is a linux container technology that is taking the world by storm. It has enabled a vibrant ecosystem of open-source and private container images, including dozens of data storage solutions. However, there are still many tricky cases to manage with Docker, such as upgrading containers smoothly, accessing volumes inside and outside the container and bridging network ports from container to host. In this presentation, we survey techniques for managing Docker containers across many data stores and apply the best ones within our own Docker buildfile with the most current practices for database management. We look at MySQL & Docker in the big picture, from development to deployment, operations and maintenance.

Sunny Gleason

April 15, 2015

More Decks by Sunny Gleason

Other Decks in Technology


  1. Who am I? 2 • Sunny Gleason – Distributed Systems

    Engineer – SunnyCloud, Boston MA • Prior Web Services Work – Amazon – Ning • Focus: Scalable, Reliable Storage Systems for Structured & Unstructured Data
  2. What’s this all about? • Why care about Databases &

    DevOps? • What benefits does Docker provide? • How can we use Docker with MySQL? 3
  3. How does DevOps relate to Databases? • Development-Oriented: DB is

    an empty shell / skeleton schema that enables features to provide new business value • Operations-Oriented: DB is a living organism that provides existing business value 4
  4. Development-Focused Perspective 5 Sources: http://support.smartbear.com/ http://www.docbyte.com/en/blog/integrating-gwt-with-spring-and-hibernate • Business/Product defines data

    needs • Developer maps it into a schema • Developer implements features using whatever app implementation technologies • “Somebody else” takes care of releases, monitoring, escalation
  5. Operations-Focused Perspective 10 Source: https://www.mysql.com/products/workbench/ • Instantiate & Deploy

    • Manage environments • Hosts & Databases • Storage management • Replication & Backups
  6. Where does DevOps fit in? 14 Source: http://blog.appdynamics.com/devops/devops-scares-me-part-2/ • “Developer

    folks” should be
 more operations-aware • “Systems folks” should be
 more development-aware • Goal: Everyone should be more
 responsive to the business & market
  7. How can Docker help? 15 • Docker is not DevOps

    • Docker will not do your laundry • Docker will let you create standard images • Docker will let you deploy & run versioned images • Docker is lightweight enough for developers to use • Docker is powerful enough to solve many needs • Docker will not coordinate your systems
  8. What is the core idea of Docker? 16 Traditional Virtualization

    (Xen, VMWare, etc.) Source: https://www.docker.com/
  9. What does Docker provide? 17 • Container System • Container

    Format • Container Buildfiles • Container Versioning • Repository API • Deployment Mechanism • Container Runtime • Virtualization via libvirt • Container Library • Cross-platform support
  10. Docker Gaps / Gotchas 19 • Still maturing • Rapid

    release cycle • Network configuration • No Easy Introspection • No “dom0” standard • Cross-platform is new • Security • Troubleshooting • I/O-Intensive Deploy • No Service Discovery • Differing Integrations • Orchestration
  11. What’s Docker’s value proposition? 20 • Ability to collaborate &

    version standard images • Easier deployment across environments / stages • Developer-friendly installation & usage • Operations-friendly controls & management
  12. Getting Started with Docker 21 • Check out: http://docs.docker.com/installation/ •

    Install Docker on Linux using something like:
 $ wget -qO- https://get.docker.com/ | sh • (Install Boot2docker or Kitematic on OS X) • Deploy Containers using “docker run” • A somewhat-authoritative MySQL image is at:
  13. Docker Summed Up in 1 Command 22 
 docker run

    --name some-mysql 
 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag • Downloads image with version “mysql:tag” • Creates a new container called “some-mysql” • Starts the container process using the default endpoint
  14. First Commands With Docker 23 • docker pull : downloads

    an image (makes available for use) • docker run : creates a container from an image • docker start / stop / restart : container process control • docker save : saves container state as a tar file • docker load : loads container state from a tar file • docker build : builds a new image from a Dockerfile • docker commit : creates a new revision of an image from a container • docker tag : associates a tag with a commit (think: git) • docker attach : attach console to running container • docker logs : fetch the logs of a container
  15. FROM debian:wheezy 
 RUN groupadd -r mysql && useradd -r

    -g mysql mysql RUN apt-get update && apt-get install -y perl --no-install-recommends 
 && rm -rf /var/lib/apt/lists/* RUN apt-key adv --keyserver pool.sks-keyservers.net \
 --recv-keys A4A9406876FCBD3C456770C88C718D3B5072E1F5
 RUN echo "deb http://repo.mysql.com/apt/debian/ wheezy mysql-$ {MYSQL_MAJOR}-dmr"
 > /etc/apt/sources.list.d/mysql.list 
 MySQL 5.7 Dockerfile (part 1) 24 Source: https://raw.githubusercontent.com/docker-library/mysql/master/5.7/Dockerfile
  16. MySQL 5.7 Dockerfile (part 2) 25 Source: https://raw.githubusercontent.com/docker-library/mysql/master/5.7/Dockerfile ... 

    # comment out a few problematic configuration values RUN sed -Ei 's/^(bind-address|log)/#&/' /etc/mysql/my.cnf VOLUME /var/lib/mysql COPY docker-entrypoint.sh /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] EXPOSE 3306 CMD ["mysqld"]
  17. Keep in mind: file locations • Docker stores files in

    /var/lib/docker • Containers live in “containers” dir • Images live in “graph” • You can inspect/modify file systems
 (at own peril) 26
  18. Keep in mind: users & permissions • Docker containers tend

    to run entry points as root • In mysql case, uses mysql user • User-level security is not 100% 27
  19. Keep in mind: externalize volumes • Docker containers are self-contained

    by default • External volumes can be mounted using the
 -v /source/external:/source/internal
 argument to docker run • This is important for: data dir, log dir, possibly config dir 28
  20. Keep in mind: network ports • Each docker container has

    its own virtual ip • Ports are mapped from internal to external using 
 -P 3306:3333
 argument to docker run • There will be port conflicts to manage 29
  21. Keep in mind: entry points • Each docker container starts

    with a default entry point • This is usually the only process running in the container • To do more, need to use a supervisor process or / bin/bash as entry point • This affects your ability to inspect inside the container using the default attach command 30
  22. Keep in mind: storage management • The default docker directory

 /var/lib/docker • Consider mounting /var/lib/docker as an independent device using /etc/fstab or configure a docker directory for the docker daemon • It’s tricky to analyze the storage requirements of images since revisions are shared 31
  23. What does this mean for development? • Ability to use

    Docker on Linux, OS X • Provides mechanism for image versioning and easier provisioning / running of containers • Development environments are less unique, more throwaway • Ability to run multiple container instances 32
  24. What does this mean for staging/test? • Staging and test

    can be closer to development • Staging and test environments are less unique, more throwaway • Ability to run multiple container instances • Point in time activation / deactivation of containers 33
  25. What does this mean for production? • Production can be

    closer to development • Production environments are more throwaway • Ability to run multiple container instances • Ability to deploy software updates more easily on same hardware (different containers using same data volume) 34
  26. What’s next? • Better cross-platform support • Better service discovery

    & network config • Better orchestration • Wider range of integrations (AWS, Google, …) 35
  27. What’s going to be tricky for a while? • Security

    • Performance & process isolation • Resource management • “dom0” Provisioning • Network configuration 36