Dan Callahan - Fire your supervisord: running Python apps on CoreOS

CoreOS is a new Linux distribution that makes it easy to deploy applications on dynamically scaled clusters of computers, and which has recently been embraced by infrastructure providers like DigitalOcean, Rackspace, and Google Compute Engine. Come learn how to package and deploy a Python application on this new, Docker-based platform.

https://us.pycon.org/2015/schedule/presentation/334/

Transcript

1. Python on CoreOS Dan Callahan — @callahad
3. This is not a talk about Docker There is one of those tomorrow
4. This is a talk about servers And what containerization changes
6. Linux distribution designed for containers Preview best practices from the future
7. What’s your ideal platform?
8. 1. Stays Updated 2. Won’t Break Apps 3. Survives Outages
9. We need something declarative “Always keep two of these running, but not on the same machine.”
10. We need new technology 1. System Updates 2. Application Isolation 3. Clustering 4. Task Distribution
11. Technology in CoreOS 1. FastPatch (Updates) 2. Docker / rkt (Containers) 3. Etcd (Consensus) 4. Fleet / Kubernetes (Scheduling) All Free / Open Source Software
12. System Updates (FastPatch)
13. Staying up-to-date is key to good security
14. Browser-like Update Channels Alpha → Beta → Stable Opportunistically downloaded Applied on next start
15. Whole-system Updates
22. My server is rebooting on its own, how do I keep my app online?
24. Both my servers reboot at the same time, how do I keep my app online?
25. Consensus (etcd)
26. etcd Key-value store Centralized place to store cluster metadata
28. locksmith Must acquire a lock from etcd before rebooting Release lock after successful boot
29. Demo
30. My etcd server is rebooting on its own, how do I keep my app online?
37. Etcd is Reusable Google Kubernetes Pivotal CloudFoundry Mailgun Vulcand
38. Containerization (Docker / rkt)
39. CoreOS is Minimal 140 MB compressed No Python, Perl, Ruby, or JavaScript No package manager No compiler
40. How do you run anything?
42. Containers CoreOS includes Docker and rkt
43. Demo
44. Scheduling (Fleet / Kubernetes)
45. Cluster-level init “Always keep two of these running, but not on the same machine.”
46. Schedulers CoreOS includes Fleet, supports Kubernetes Both independent components Both built on etcd
47. Fleet Clustered interface for systemd
48. Systemd Unit Files [ U n i t ] D e s c r i p t i o n = M y A p p A f t e r = d o c k e r . s e r v i c e R e q u i r e s = d o c k e r . s e r v i c e [ S e r v i c e ] E x e c S t a r t P r e = - / u s r / b i n / d o c k e r k i l l m y - a p p - % i E x e c S t a r t P r e = - / u s r / b i n / d o c k e r r m m y - a p p - % i E x e c S t a r t = / u s r / b i n / d o c k e r r u n - r m - - n a m e m y - a p p - % i - p 8 0 : 8 0 8 0 c a l l a h a d / m y - a p p E x e c S t o p = / u s r / b i n / d o c k e r s t o p m y - a p p - % i [ X - F l e e t ] C o n f l i c t s = m y - a p p @ * . s e r v i c e
49. X-Fleet attributes C o n f l i c t s M a c h i n e O f M a c h i n e I D M a c h i n e M e t a d a t a G l o b a l
50. Demo
51. Design Considerations Minimize state Build “Twelve-Factor Apps”
52. What about Databases? Load balancers?
53. We did it!
54. We built a platform that is self-updating, self-organizing, and self-healing.
55. We used 1. An OS with automatic, atomic, whole-system updates. 2. Portable, isolated containers for our applications. 3. Multiple servers in a coordinated cluster. 4. A scheduler to distribute jobs across machines.
56. Now it’s your turn!
57. Many supported platforms Local VMs (Vagrant) Azure, EC2, GCE, RackSpace DigitalOcean $40 credit on DigitalOcean: “SAMMYLOVESPYCON”
58. Questions? dcallahan@mozilla.com @callahad github.com/callahad/pycon2015-coreos “SAMMYLOVESPYCON”