PyBay 2018: Production-Ready Python applications

In 2016, Susan Fowler released the 'Production Ready Microservices' book. This book sets an industry benchmark on explaining how microservices should be conceived, all the way through to documentation. So how does this translate for Python applications? This session will explore how to expertly deploy your Python micro-service to production.

Transcript

1. Production-Ready Python applications
   Michael Kehoe
   Staff Site Reliability Engineer
   

   View Slide

2. Today’s
   agenda
   1 Introduction
   2 Tenets of Readiness
   3 Building Production-Ready Python Applications
   4 Recap
   

   View Slide

3. Introduction
   

   View Slide

4. Michael Kehoe
   $ /USR/BIN/WHOAMI
   • Staff Site Reliability Engineer @ LinkedIn
   • Production-SRE Team
   • Funny accent = Australian + 4 years American
   • Worked on:
   • Networks
   • Micro-services
   • Traffic Engineering
   • Databases
   • Satellites
   

   View Slide

5. Production-SRE Team @ LinkedIn
   $ /USR/BIN/WHOAMI
   • Disaster Recovery - Planning & Automation
   • Incident Response – Process & Automation
   • Visibility Engineering – Making use of
   operational data
   • Reliability Principles – Defining best practice
   & automating it
   

   View Slide

6. Production-Ready Python Applications
   • This talk is a high-level overview of what it
   takes to build a production-ready service
   • Each of the topics could be its own separate
   talk
   • Focus on standard open-source options
   • Number of options not mentioned
   

   View Slide

7. What makes an application
   Production-Ready?
   

   View Slide

8. O’Reilly 2017
   Susan J. Fowler
   Production-Ready
   Microservices
   

   View Slide

9. “A production-ready application or service is one
   that can be trusted to serve production traffic…”
   S U S A N J . F O W L E R
   

   View Slide

10. “… We trust it to behave reasonably, we trust it to
    perform reliably, we trust it to get the job done and
    to do its job well with very little downtime.”
    S U S A N J . F O W L E R
    

    View Slide

11. Tenets of readiness
    

    View Slide

12. Tenets of
    Readiness
    1 Stability
    2 Reliability
    3 Scalability
    4 Performance
    5 Fault Tolerance
    6 Disaster Recovery
    7 Monitoring
    8 Documentation
    

    View Slide

13. Building Production-Ready
    Python applications
    

    View Slide

14. Stability
    

    View Slide

15. Tenets of Readiness
    STABILITY
    • Stable development cycle
    • Code Linting
    • Code Review
    • Central Repository
    • Build system
    • See Fabio Fleitas’s talk from Saturday
    

    View Slide

16. Tenets of Readiness
    STABILITY
    • Stable deployment cycle
    • Canary/ Staging environment
    • Reliable Deployment via:
    • Docker/ Kubernetes
    • Heroku
    • CD tools
    

    View Slide

17. Reliability
    

    View Slide

18. Tenets of Readiness
    RELIABILITY
    • Dependency Management
    • Onboarding + Deprecation procedures
    • See Documentation section
    • Routing + Discovery
    • Etcd
    • PyDiscover
    • Consul-service-discovery
    

    View Slide

19. Scalability
    

    View Slide

20. Tenets of Readiness
    SCALABILITY
    • Understanding growth-scales
    • Qualitative vs quantitative growth scale
    • Resource awareness
    • Dependency scaling
    • What services/ databases need to scale
    

    View Slide

21. Performance
    

    View Slide

22. Tenets of Readiness
    PERFORMANCE
    • Constant performance evaluation
    • Understand how to benchmark
    application
    • Traffic management
    • Understand traffic pattern performance
    • Capacity Planning
    • Have the right metrics
    

    View Slide

23. Fault Tolerance
    

    View Slide

24. Tenets of Readiness
    FAULT TOLERANCE
    • Avoiding Single Points of Failure (SPOF)
    • Use multiple instances behind a load
    balancer
    • Catch exceptions (meaningfully)
    

    View Slide

25. Tenets of Readiness
    FAULT TOLERANCE
    • Resiliency Engineering
    • Add testing to verify that non-standard
    behavior is handled correctly
    • Run chaos experiments:
    • DNS/ Network failures
    • Consume disk space/ IO
    • Consume CPU
    

    View Slide

26. Disaster Recovery
    

    View Slide

27. Tenets of Readiness
    DISASTER RECOVERY
    • Understand common failures
    • Have an application-specific disaster-
    recovery plan in place
    • Have a general incident management plan
    

    View Slide

28. Tenets of Readiness
    DISASTER RECOVERY: DAEMONS
    

    View Slide

29. Tenets of Readiness
    DISASTER RECOVERY: FLASK APP
    

    View Slide

30. Monitoring
    

    View Slide

31. Tenets of Readiness
    MONITORING
    • Logging
    • Tracing
    • Metrics à Dashboards/ Alerts
    

    View Slide

32. Tenets of Readiness
    MONITORING: LOGGING
    https://docs.python.org/2/library/logging.handlers.html#sysloghandler
    

    View Slide

33. Tenets of Readiness
    MONITORING: TRACING
    • Multiple (free/ open-source) Options
    • Opentracing
    • Jaeger
    • Zipkin (various community libraries)
    

    View Slide

34. Tenets of Readiness
    MONITORING: METRICS
    • Multiple (free/ open-source) Options
    • Statsd
    • Jaeger
    • prometheus
    

    View Slide

35. Monitoring: Metrics
    

    View Slide

36. Documentation
    

    View Slide

37. In-Code Documentation
    What you should be covering
    • Function/ API & Class docstrings
    • Code usage documentation
    • Project documentation
    • Onboarding
    • Contribution
    • Testing
    

    View Slide

38. Tenets of Readiness
    DOCUMENTATION
    How to do it?
    • Sphinx
    • JavaDoc
    • Doxygen
    • Etc….
    

    View Slide

39. Documentation
    RESTRUCTURED TEXT EXAMPLE
    https://github.com/requests/requests/blob/master/requests/api.py
    

    View Slide

40. Documentation
    RESTRUCTURED TEXT EXAMPLE
    http://docs.python-requests.org/en/master/api/
    

    View Slide

41. Documentation
    RESTRUCTURED TEXT EXAMPLE
    https://github.com/requests/requests/blob/master/requests/api.py
    

    View Slide

42. Documentation
    RESTRUCTURED TEXT EXAMPLE
    http://docs.python-requests.org/en/master/api/
    

    View Slide

43. Documentation
    https://oncall.tools/docs/
    

    View Slide

44. Documentation
    • Relevant PEP’s:
    • PEP -257: Docstring Conventions
    • PEP-287: reStructured Docstring Format
    (Official Python Documentation Standard)
    • Further info:
    • https://pythonhosted.org/an_example_p
    ypi_project/sphinx.html
    • https://realpython.com/documenting-
    python-code
    

    View Slide

45. Recap
    

    View Slide

46. Tenets of
    Readiness
    1 Stability
    2 Reliability
    3 Scalability
    4 Performance
    5 Fault Tolerance
    6 Disaster Recovery
    7 Monitoring
    8 Documentation
    

    View Slide

47. Further Resources
    • Find me at:
    • michael-kehoe.io
    • @matrixtek
    • linkedin.com/in/michaelkkehoe
    • Slides will be available in multiple locations
    shortly
    

    View Slide

48. View Slide