Dan Callahan - Fire your supervisord: running Python apps on CoreOS

Dan Callahan - Fire your supervisord: running Python apps on CoreOS

CoreOS is a new Linux distribution that makes it easy to deploy applications on dynamically scaled clusters of computers, and which has recently been embraced by infrastructure providers like DigitalOcean, Rackspace, and Google Compute Engine. Come learn how to package and deploy a Python application on this new, Docker-based platform.

https://us.pycon.org/2015/schedule/presentation/334/

D5710b3bca38f1233274b4cbc523dc4b?s=128

PyCon 2015

April 18, 2015
Tweet

Transcript

  1. Python on CoreOS Dan Callahan — @callahad

  2. None
  3. This is not a talk about Docker There is one

    of those tomorrow
  4. This is a talk about servers And what containerization changes

  5. None
  6. Linux distribution designed for containers Preview best practices from the

    future
  7. What’s your ideal platform?

  8. 1. Stays Updated 2. Won’t Break Apps 3. Survives Outages

  9. We need something declarative “Always keep two of these running,

    but not on the same machine.”
  10. We need new technology 1. System Updates 2. Application Isolation

    3. Clustering 4. Task Distribution
  11. Technology in CoreOS 1. FastPatch (Updates) 2. Docker / rkt

    (Containers) 3. Etcd (Consensus) 4. Fleet / Kubernetes (Scheduling) All Free / Open Source Software
  12. System Updates (FastPatch)

  13. Staying up-to-date is key to good security

  14. Browser-like Update Channels Alpha → Beta → Stable Opportunistically downloaded

    Applied on next start
  15. Whole-system Updates

  16. None
  17. None
  18. None
  19. None
  20. None
  21. None
  22. My server is rebooting on its own, how do I

    keep my app online?
  23. None
  24. Both my servers reboot at the same time, how do

    I keep my app online?
  25. Consensus (etcd)

  26. etcd Key-value store Centralized place to store cluster metadata

  27. None
  28. locksmith Must acquire a lock from etcd before rebooting Release

    lock after successful boot
  29. Demo

  30. My etcd server is rebooting on its own, how do

    I keep my app online?
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. Etcd is Reusable Google Kubernetes Pivotal CloudFoundry Mailgun Vulcand

  38. Containerization (Docker / rkt)

  39. CoreOS is Minimal 140 MB compressed No Python, Perl, Ruby,

    or JavaScript No package manager No compiler
  40. How do you run anything?

  41. None
  42. Containers CoreOS includes Docker and rkt

  43. Demo

  44. Scheduling (Fleet / Kubernetes)

  45. Cluster-level init “Always keep two of these running, but not

    on the same machine.”
  46. Schedulers CoreOS includes Fleet, supports Kubernetes Both independent components Both

    built on etcd
  47. Fleet Clustered interface for systemd

  48. Systemd Unit Files [ U n i t ] D

    e s c r i p t i o n = M y A p p A f t e r = d o c k e r . s e r v i c e R e q u i r e s = d o c k e r . s e r v i c e [ S e r v i c e ] E x e c S t a r t P r e = - / u s r / b i n / d o c k e r k i l l m y - a p p - % i E x e c S t a r t P r e = - / u s r / b i n / d o c k e r r m m y - a p p - % i E x e c S t a r t = / u s r / b i n / d o c k e r r u n - r m - - n a m e m y - a p p - % i - p 8 0 : 8 0 8 0 c a l l a h a d / m y - a p p E x e c S t o p = / u s r / b i n / d o c k e r s t o p m y - a p p - % i [ X - F l e e t ] C o n f l i c t s = m y - a p p @ * . s e r v i c e
  49. X-Fleet attributes C o n f l i c t

    s M a c h i n e O f M a c h i n e I D M a c h i n e M e t a d a t a G l o b a l
  50. Demo

  51. Design Considerations Minimize state Build “Twelve-Factor Apps”

  52. What about Databases? Load balancers?

  53. We did it!

  54. We built a platform that is self-updating, self-organizing, and self-healing.

  55. We used 1. An OS with automatic, atomic, whole-system updates.

    2. Portable, isolated containers for our applications. 3. Multiple servers in a coordinated cluster. 4. A scheduler to distribute jobs across machines.
  56. Now it’s your turn!

  57. Many supported platforms Local VMs (Vagrant) Azure, EC2, GCE, RackSpace

    DigitalOcean $40 credit on DigitalOcean: “SAMMYLOVESPYCON”
  58. Questions? dcallahan@mozilla.com @callahad github.com/callahad/pycon2015-coreos “SAMMYLOVESPYCON”