FlyingCloud: Docker containers built with SaltStack

FlyingCloud: Docker containers built with SaltStack

12253d48375c73fcfdda8688da6aa3a4?s=128

George V. Reilly

March 09, 2016
Tweet

Transcript

  1. Copyright © MetaBrite, Inc. 2016 FlyingCloud Building Docker Containers with

    SaltStack Adam Feuer and George V. Reilly
  2. Copyright © MetaBrite, Inc. 2016 The Story of Lectern •

    Lectern Web Service • Receives uploaded images of receipts from CookBrite mobile app • OCRs images using Tesseract + OpenCV • Extracts data about purchases • Deployed on AWS ElasticBeanstalk • Could not get app to build on Amazon Linux
  3. Copyright © MetaBrite, Inc. 2016 Lectern v1 • ElasticBeanstalk also

    supports Docker • We built a Docker app on top of an Ubuntu image • Solved the build problem. Sort of. • Our build was a twisted, unmaintainable, monolithic mess of shell scripts. • Took an hour to build. Any failure reset the clock.
  4. Copyright © MetaBrite, Inc. 2016 Lectern's old Docker image setup

    scripts
  5. Copyright © MetaBrite, Inc. 2016 Lectern v2: with FlyingCloud

  6. Copyright © MetaBrite, Inc. 2016 One Big Idea • Use

    Configuration Management to build Docker images • Configure rather than write code • More understandable • Reusable and repeatable
  7. Copyright © MetaBrite, Inc. 2016 What is Docker? • Lightweight

    Virtualization • Run multiple containers on one host system • Isolated process group for each container • Shared kernel • Run a different Linux OS in each container • Image = filesystem snapshot • Container = running image
  8. Copyright © MetaBrite, Inc. 2016 What is SaltStack? • Configuration

    management system • "Automated system administration" • Configuration, not scripts • Declarative, not imperative • Salt is implemented in Python • Though you usually write Salt States in YAML • We use Salt Masterless mode • Configure system at build time; no runtime dependency on Salt
  9. Copyright © MetaBrite, Inc. 2016 Two Ways to Build Docker

    Images • Traditional • Dockerfile: static list of commands • Run commands inside container, then commit • FlyingCloud • Use SaltStack to update state of Docker image • Easy to make layers • Easy to run in Continuous Integration
  10. Copyright © MetaBrite, Inc. 2016 Example App - layers overview

  11. Copyright © MetaBrite, Inc. 2016 Example App - project and

    base layers
  12. Copyright © MetaBrite, Inc. 2016 Example App - app layer

  13. Copyright © MetaBrite, Inc. 2016 Example App - testrunner layer

  14. Copyright © MetaBrite, Inc. 2016 FlyingCloud Demo See https://github.com/cookbrite/flyingcloud/tree/master/examples/flask-web-app

  15. FlyingCloud • Use Configuration Management to build layered Docker images

    • An FC Layer can be made from any of these: ◦ A Dockerfile, Salt states, or custom Python code ◦ Do not confuse with Docker’s layer filesystem • Layers are buildable and/or runnable • Can be pushed to or pulled from a Docker repository
  16. Copyright © MetaBrite, Inc. 2016 Kubernetes • Google’s open source

    platform that automates deployment, scaling, and operations of application containers built on Docker images across clusters of hosts • Our Kubernetes workflow for new services: • Build Docker images with FlyingCloud • Configure Kubernetes cluster • Test cluster in Continuous Integration • Deploy cluster to AWS
  17. Copyright © MetaBrite, Inc. 2016 FlyingCloud is Open Source •

    Source: https://github.com/cookbrite/flyingcloud • Docs: http://flyingcloud.readthedocs.org • PyPI: https://pypi.python.org/pypi/flyingcloud/ • Mail: https://groups.google.com/group/flyingcloud-users • Apache License Contributors welcome! Especially: • SaltStack users • Ansible users
  18. Copyright © MetaBrite, Inc. 2016 We are organizing the world's

    product information We’re Hiring! Located in Pioneer Square. • Back-end devs. We're a Python shop. • Android devs • iOS devs • Data scientists Contact • adam@metabrite.com or george@metabrite.com