FlyingCloud: Docker containers built with SaltStack

Transcript

1. Copyright © MetaBrite, Inc. 2016 FlyingCloud Building Docker Containers with SaltStack Adam Feuer and George V. Reilly
2. Copyright © MetaBrite, Inc. 2016 The Story of Lectern • Lectern Web Service • Receives uploaded images of receipts from CookBrite mobile app • OCRs images using Tesseract + OpenCV • Extracts data about purchases • Deployed on AWS ElasticBeanstalk • Could not get app to build on Amazon Linux
3. Copyright © MetaBrite, Inc. 2016 Lectern v1 • ElasticBeanstalk also supports Docker • We built a Docker app on top of an Ubuntu image • Solved the build problem. Sort of. • Our build was a twisted, unmaintainable, monolithic mess of shell scripts. • Took an hour to build. Any failure reset the clock.
4. Copyright © MetaBrite, Inc. 2016 Lectern's old Docker image setup scripts
5. Copyright © MetaBrite, Inc. 2016 Lectern v2: with FlyingCloud
6. Copyright © MetaBrite, Inc. 2016 One Big Idea • Use Configuration Management to build Docker images • Configure rather than write code • More understandable • Reusable and repeatable
7. Copyright © MetaBrite, Inc. 2016 What is Docker? • Lightweight Virtualization • Run multiple containers on one host system • Isolated process group for each container • Shared kernel • Run a different Linux OS in each container • Image = filesystem snapshot • Container = running image
8. Copyright © MetaBrite, Inc. 2016 What is SaltStack? • Configuration management system • "Automated system administration" • Configuration, not scripts • Declarative, not imperative • Salt is implemented in Python • Though you usually write Salt States in YAML • We use Salt Masterless mode • Configure system at build time; no runtime dependency on Salt
9. Copyright © MetaBrite, Inc. 2016 Two Ways to Build Docker Images • Traditional • Dockerfile: static list of commands • Run commands inside container, then commit • FlyingCloud • Use SaltStack to update state of Docker image • Easy to make layers • Easy to run in Continuous Integration
10. Copyright © MetaBrite, Inc. 2016 Example App - layers overview
11. Copyright © MetaBrite, Inc. 2016 Example App - project and base layers
12. Copyright © MetaBrite, Inc. 2016 Example App - app layer
13. Copyright © MetaBrite, Inc. 2016 Example App - testrunner layer
14. Copyright © MetaBrite, Inc. 2016 FlyingCloud Demo See https://github.com/cookbrite/flyingcloud/tree/master/examples/flask-web-app
15. FlyingCloud • Use Configuration Management to build layered Docker images • An FC Layer can be made from any of these: ◦ A Dockerfile, Salt states, or custom Python code ◦ Do not confuse with Docker’s layer filesystem • Layers are buildable and/or runnable • Can be pushed to or pulled from a Docker repository
16. Copyright © MetaBrite, Inc. 2016 Kubernetes • Google’s open source platform that automates deployment, scaling, and operations of application containers built on Docker images across clusters of hosts • Our Kubernetes workflow for new services: • Build Docker images with FlyingCloud • Configure Kubernetes cluster • Test cluster in Continuous Integration • Deploy cluster to AWS
17. Copyright © MetaBrite, Inc. 2016 FlyingCloud is Open Source • Source: https://github.com/cookbrite/flyingcloud • Docs: http://flyingcloud.readthedocs.org • PyPI: https://pypi.python.org/pypi/flyingcloud/ • Mail: https://groups.google.com/group/flyingcloud-users • Apache License Contributors welcome! Especially: • SaltStack users • Ansible users
18. Copyright © MetaBrite, Inc. 2016 We are organizing the world's product information We’re Hiring! Located in Pioneer Square. • Back-end devs. We're a Python shop. • Android devs • iOS devs • Data scientists Contact • adam@metabrite.com or george@metabrite.com