FlyingCloud: Docker containers built with SaltStack

Transcript

1. Copyright © MetaBrite, Inc. 2016 FlyingCloud Building Docker Containers with

   SaltStack Adam Feuer and George V. Reilly

2. Copyright © MetaBrite, Inc. 2016 The Story of Lectern •

   Lectern Web Service • Receives uploaded images of receipts from CookBrite mobile app • OCRs images using Tesseract + OpenCV • Extracts data about purchases • Deployed on AWS ElasticBeanstalk • Could not get app to build on Amazon Linux

3. Copyright © MetaBrite, Inc. 2016 Lectern v1 • ElasticBeanstalk also

   supports Docker • We built a Docker app on top of an Ubuntu image • Solved the build problem. Sort of. • Our build was a twisted, unmaintainable, monolithic mess of shell scripts. • Took an hour to build. Any failure reset the clock.

4. Copyright © MetaBrite, Inc. 2016 Lectern's old Docker image setup

   scripts

5. Copyright © MetaBrite, Inc. 2016 Lectern v2: with FlyingCloud

6. Copyright © MetaBrite, Inc. 2016 One Big Idea • Use

   Configuration Management to build Docker images • Configure rather than write code • More understandable • Reusable and repeatable

7. Copyright © MetaBrite, Inc. 2016 What is Docker? • Lightweight

   Virtualization • Run multiple containers on one host system • Isolated process group for each container • Shared kernel • Run a different Linux OS in each container • Image = filesystem snapshot • Container = running image

8. Copyright © MetaBrite, Inc. 2016 What is SaltStack? • Configuration

   management system • "Automated system administration" • Configuration, not scripts • Declarative, not imperative • Salt is implemented in Python • Though you usually write Salt States in YAML • We use Salt Masterless mode • Configure system at build time; no runtime dependency on Salt

9. Copyright © MetaBrite, Inc. 2016 Two Ways to Build Docker

   Images • Traditional • Dockerfile: static list of commands • Run commands inside container, then commit • FlyingCloud • Use SaltStack to update state of Docker image • Easy to make layers • Easy to run in Continuous Integration

10. Copyright © MetaBrite, Inc. 2016 Example App - layers overview

11. Copyright © MetaBrite, Inc. 2016 Example App - project and

    base layers

12. Copyright © MetaBrite, Inc. 2016 Example App - app layer

13. Copyright © MetaBrite, Inc. 2016 Example App - testrunner layer

14. Copyright © MetaBrite, Inc. 2016 FlyingCloud Demo See https://github.com/cookbrite/flyingcloud/tree/master/examples/flask-web-app

15. FlyingCloud • Use Configuration Management to build layered Docker images

    • An FC Layer can be made from any of these: ◦ A Dockerfile, Salt states, or custom Python code ◦ Do not confuse with Docker’s layer filesystem • Layers are buildable and/or runnable • Can be pushed to or pulled from a Docker repository

16. Copyright © MetaBrite, Inc. 2016 Kubernetes • Google’s open source

    platform that automates deployment, scaling, and operations of application containers built on Docker images across clusters of hosts • Our Kubernetes workflow for new services: • Build Docker images with FlyingCloud • Configure Kubernetes cluster • Test cluster in Continuous Integration • Deploy cluster to AWS

17. Copyright © MetaBrite, Inc. 2016 FlyingCloud is Open Source •

    Source: https://github.com/cookbrite/flyingcloud • Docs: http://flyingcloud.readthedocs.org • PyPI: https://pypi.python.org/pypi/flyingcloud/ • Mail: https://groups.google.com/group/flyingcloud-users • Apache License Contributors welcome! Especially: • SaltStack users • Ansible users

18. Copyright © MetaBrite, Inc. 2016 We are organizing the world's

    product information We’re Hiring! Located in Pioneer Square. • Back-end devs. We're a Python shop. • Android devs • iOS devs • Data scientists Contact • adam@metabrite.com or george@metabrite.com