PyBay 2018: Production-Ready Python applications

In 2016, Susan Fowler released the 'Production Ready Microservices' book. This book sets an industry benchmark on explaining how microservices should be conceived, all the way through to documentation. So how does this translate for Python applications? This session will explore how to expertly deploy your Python micro-service to production.

Transcript

1. Production-Ready Python applications Michael Kehoe Staff Site Reliability Engineer
2. Today’s agenda 1 Introduction 2 Tenets of Readiness 3 Building Production-Ready Python Applications 4 Recap
3. Introduction
4. Michael Kehoe $ /USR/BIN/WHOAMI • Staff Site Reliability Engineer @ LinkedIn • Production-SRE Team • Funny accent = Australian + 4 years American • Worked on: • Networks • Micro-services • Traffic Engineering • Databases • Satellites
5. Production-SRE Team @ LinkedIn $ /USR/BIN/WHOAMI • Disaster Recovery - Planning & Automation • Incident Response – Process & Automation • Visibility Engineering – Making use of operational data • Reliability Principles – Defining best practice & automating it
6. Production-Ready Python Applications • This talk is a high-level overview of what it takes to build a production-ready service • Each of the topics could be its own separate talk • Focus on standard open-source options • Number of options not mentioned
7. What makes an application Production-Ready?
8. O’Reilly 2017 Susan J. Fowler Production-Ready Microservices
9. “A production-ready application or service is one that can be trusted to serve production traffic…” S U S A N J . F O W L E R
10. “… We trust it to behave reasonably, we trust it to perform reliably, we trust it to get the job done and to do its job well with very little downtime.” S U S A N J . F O W L E R
11. Tenets of readiness
12. Tenets of Readiness 1 Stability 2 Reliability 3 Scalability 4 Performance 5 Fault Tolerance 6 Disaster Recovery 7 Monitoring 8 Documentation
13. Building Production-Ready Python applications
14. Stability
15. Tenets of Readiness STABILITY • Stable development cycle • Code Linting • Code Review • Central Repository • Build system • See Fabio Fleitas’s talk from Saturday
16. Tenets of Readiness STABILITY • Stable deployment cycle • Canary/ Staging environment • Reliable Deployment via: • Docker/ Kubernetes • Heroku • CD tools
17. Reliability
18. Tenets of Readiness RELIABILITY • Dependency Management • Onboarding + Deprecation procedures • See Documentation section • Routing + Discovery • Etcd • PyDiscover • Consul-service-discovery
19. Scalability
20. Tenets of Readiness SCALABILITY • Understanding growth-scales • Qualitative vs quantitative growth scale • Resource awareness • Dependency scaling • What services/ databases need to scale
21. Performance
22. Tenets of Readiness PERFORMANCE • Constant performance evaluation • Understand how to benchmark application • Traffic management • Understand traffic pattern performance • Capacity Planning • Have the right metrics
23. Fault Tolerance
24. Tenets of Readiness FAULT TOLERANCE • Avoiding Single Points of Failure (SPOF) • Use multiple instances behind a load balancer • Catch exceptions (meaningfully)
25. Tenets of Readiness FAULT TOLERANCE • Resiliency Engineering • Add testing to verify that non-standard behavior is handled correctly • Run chaos experiments: • DNS/ Network failures • Consume disk space/ IO • Consume CPU
26. Disaster Recovery
27. Tenets of Readiness DISASTER RECOVERY • Understand common failures • Have an application-specific disaster- recovery plan in place • Have a general incident management plan
28. Tenets of Readiness DISASTER RECOVERY: DAEMONS
29. Tenets of Readiness DISASTER RECOVERY: FLASK APP
30. Monitoring
31. Tenets of Readiness MONITORING • Logging • Tracing • Metrics à Dashboards/ Alerts
32. Tenets of Readiness MONITORING: LOGGING https://docs.python.org/2/library/logging.handlers.html#sysloghandler
33. Tenets of Readiness MONITORING: TRACING • Multiple (free/ open-source) Options • Opentracing • Jaeger • Zipkin (various community libraries)
34. Tenets of Readiness MONITORING: METRICS • Multiple (free/ open-source) Options • Statsd • Jaeger • prometheus
35. Monitoring: Metrics
36. Documentation
37. In-Code Documentation What you should be covering • Function/ API & Class docstrings • Code usage documentation • Project documentation • Onboarding • Contribution • Testing
38. Tenets of Readiness DOCUMENTATION How to do it? • Sphinx • JavaDoc • Doxygen • Etc….
39. Documentation RESTRUCTURED TEXT EXAMPLE https://github.com/requests/requests/blob/master/requests/api.py
40. Documentation RESTRUCTURED TEXT EXAMPLE http://docs.python-requests.org/en/master/api/
41. Documentation RESTRUCTURED TEXT EXAMPLE https://github.com/requests/requests/blob/master/requests/api.py
42. Documentation RESTRUCTURED TEXT EXAMPLE http://docs.python-requests.org/en/master/api/
43. Documentation https://oncall.tools/docs/
44. Documentation • Relevant PEP’s: • PEP -257: Docstring Conventions • PEP-287: reStructured Docstring Format (Official Python Documentation Standard) • Further info: • https://pythonhosted.org/an_example_p ypi_project/sphinx.html • https://realpython.com/documenting- python-code
45. Recap
46. Tenets of Readiness 1 Stability 2 Reliability 3 Scalability 4 Performance 5 Fault Tolerance 6 Disaster Recovery 7 Monitoring 8 Documentation
47. Further Resources • Find me at: • michael-kehoe.io • @matrixtek • linkedin.com/in/michaelkkehoe • Slides will be available in multiple locations shortly