Can We Deploy Yet

What happens when your features are done, your MVP is ready and you want to deploy your first production build? What do you do then? How do you make your first production build instead of re-using your development one? is your code ready to handle real user interactions?
This talk will show a production ready checklist for your Python code; what to look for when creating a production-ready Docker image; what are the differences between development and production environments and builds.

This talk will show a production ready checklist for your Python code:

- what to look for when creating a production-ready Docker image;
- what are the differences between development and production environments and builds;
- logging important information and production readiness;
- exception handling;
- documentation for your code.

Transcript

1. anastasiatymo by Anastasiia Tymoshchuk Can We Deploy Yet? Lead Engineer at Scoutbee
2. anastasiatymo • Lead Dev at Scoutbee in Berlin • PyBerlin organiser 
 https://www.meetup.com/PyBerlin/ • 10 years in so ft ware development • 7 years in Python • Happy Pythonista 🐍 😊 Few words about myself
3. anastasiatymo What does "Production" mean to you?
4. anastasiatymo "You’ve worked hard on your project. It looks like all the features are actually complete, and most even have tests. You can breathe a sigh of relief. You're done. Or are you?" Is my code ready for Production? from "Release It! Design and Deploy Production- ready Software" book by Michael T. Nygard
5. anastasiatymo There are couple of checks for you before going to production
6. anastasiatymo check your code now 🐍 😎 Ready for Production? • Exception handling • How to become a detective or meaningful logging • From code review to production or e ff ective CI/CD • Docker? No problem! • More hints and ideas
7. anastasiatymo Exceptions
8. anastasiatymo We don't want to return to the user "500 Internal Server Error" https://docs.python.org/3.8/library/exceptions.html try : ... # some code here might raise an exceptio n except Exception as e: # catching exception s print("Exception occurred:", repr(e) )
9. anastasiatymo Catching Exception and BaseException try : raise Exception("My custom exception" ) except Exception as e: # catching Exception is not the best ide a print("Exception occurred:", repr(e) ) except BaseException as e: # catching BaseException is even worse print("BaseException occurred:", repr(e) )
10. anastasiatymo Guess which one will be printed try : input() except Exception as e: print("Exception occurred:", repr(e) ) except BaseException as e: print("BaseException occurred:", repr(e)) > BaseException occurred: KeyboardInterrupt()
11. anastasiatymo Hierarchy of exceptions in Python BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception +-- StopIteration +-- StopAsyncIteration +-- ArithmeticError | +-- FloatingPointError | +-- OverflowError | +-- ZeroDivisionError +-- AssertionError +-- AttributeError (...)
12. anastasiatymo Handling exceptions try : raise Exception("Something custom happened!!!" ) except Exception as e : print("Printing exception", repr(e) ) raise Exception("I want my custom message!!!" )
13. anastasiatymo Handling exceptions Traceback (most recent call last) : File "my_awesome_code.py", line 2, in <module > raise Exception("Something custom happened!!!" ) Exception: Something custom happened!! ! During handling of the above exception, another exception occurred : Traceback (most recent call last) : File "my_awesome_code.py", line 5, in <module > raise Exception("I want my custom message!!!" ) Exception: I want my custom message!! ! Printing exception Exception('Something custom happened!!!')
14. anastasiatymo Handling exceptions - raising from try : raise Exception("Something custom happened!!!" ) except Exception as e : print("Printing exception", repr(e) ) raise Exception("I want my custom message!!!") from e
15. anastasiatymo Handling exceptions - raising from Traceback (most recent call last) : File "my_awesome_code.py", line 2, in <module > raise Exception("Something custom happened!!!" ) Exception: Something custom happened!! ! The above exception was the direct cause of the following exception : Traceback (most recent call last) : File "my_awesome_code.py", line 5, in <module > raise Exception("I want my custom message!!!") from e Exception: I want my custom message!! ! Printing exception Exception('Something custom happened!!!')
16. anastasiatymo Custom exceptions class MyCustomException(Exception) : pas s try : raise MyCustomException("Something custom happened!!!" ) except MyCustomException as e: print("We are handling this exception here!", repr(e) )
17. anastasiatymo More to learn
18. anastasiatymo Logging
19. anastasiatymo from The Twelve-Factor App " Treat logs as event s tr eam s Logs provide visibili ty in to th e behavior of a running app. (...) Logs are th e s tr eam of a gg rega te d, ti me-ordered events co ll ec te d fr om th e output s tr eams of a ll running processes and backing services. Logs in th eir raw form are ty pica ll y a te xt format wi th one event per line ( th ough back tr aces fr om excep ti ons may span mul ti ple lines). Logs have no fi xed beginning or end, but fl ow con ti nuously as long as th e app is opera ti ng. "
20. anastasiatymo • when • where • what • who • outcome Main logging attributes
21. anastasiatymo How do we usually log something? import loggin g my_logger = logging.getLogger(__name__ ) logging.basicCon fi g(level=logging.INFO ) my_logger.info ( "Hello Pythonista! Conference name %s, talk name %s, key_id = %s " % ("Python Web Conf", "Can we deploy yet?", "1234" ) ) INFO:__main__:Hello Pythonista! Conference name Python Web Conf, talk name Can we deploy yet?, key_id = 1234
22. anastasiatymo How can we improve? Maybe Structlog?
23. anastasiatymo Logging with Structlog import structlo g logger_structlog = structlog.get_logger( ) logger_structlog.info ( "Hello Pythonista!" , key_id="1234" , conference_name="Python Web Conf" , talk_name="Can we deploy yet?" , ) 2020-07-19 21:38.48 Hello Pythonista! conference_name= Python Web Conf key_id=1234 talk_name=Can we deploy yet?
24. anastasiatymo How can we improve? De fi nitely Structlog!
25. anastasiatymo Let's take a closer look
26. anastasiatymo import structlo g logger_structlog = structlog.get_logger(__name__ ) logger_structlog = logger_structlog.bind ( key_id="1234", conference_name="Python Web Conf", talk_name="Can we deploy yet? " ) try : raise Exception("Oh, something went wrong..." ) except Exception : logger_structlog.exception("logging exception" ) 2020-07-19 21:43.48 logging exception conference_name= Python Web Conf key_id=1234 talk_name=Can we deploy yet? Traceback (most recent call last): File "my_awesome_code.py", line 9, in <module> raise Exception("Oh, something went wrong...") Exception: Oh, something went wrong...
27. anastasiatymo { "event": "Hello Pythonista!", "level": "error", "logger": "test", "timestamp": "2020-07-19T19:47:03.514339Z" } Also in json format!
28. anastasiatymo Demo time!👩💻
29. anastasiatymo More to learn about Structlog
30. anastasiatymo E ff ective CI/CD
31. anastasiatymo • test coverage • reliability • fault isolation • transparency • code quality • faster development • code review improvements Continuous Integration
32. anastasiatymo • self-hosted solution • paid solution • free for open-source Continuous Integration
33. anastasiatymo
34. anastasiatymo
35. anastasiatymo Demo time!👩💻 https://github.com/atymoshchuk/can_we_deploy_yet/blob/master/.github/work f lows/python-app.yml
36. anastasiatymo Docker? No Problem!
37. anastasiatymo • do not use root user • use trusted and well-known images • use COPY instead of ADD • lint your Dockerfile • save images in your docker registry and maintain them • check authenticity of the docker image Secure your Docker images
38. anastasiatymo Let me ask you something Which user are you using for your production docker images?
39. anastasiatymo Do not use root user FROM ubunt u ENV USER python # Default valu e ENV GROUP python # Default valu e RUN mkdir /ap p # Create group, create user, add user to grou p RUN groupadd -r "$GROUP" && useradd -r -g "$GROUP" -s /bin/bash "$USER " WORKDIR /ap p COPY . /ap p # Change owner of directories and fi les to USER and GROU P RUN chown -R "$USER":"$GROUP" /app USER "$USER" # Change use r
40. anastasiatymo Demo time!👩💻
41. anastasiatymo More hints
42. anastasiatymo Do you document your code?
43. anastasiatymo "There is a secret that needs to be understood in order to write good so ft ware documentation: there isn’t one thing called documentation, there are four." • tutorials, • how-to guides, • technical reference • explanation from https://documentation.divio.com/
44. anastasiatymo • start as simple as possible • go to version controlled docs How to start?
45. anastasiatymo • start with Sphinx • try Read The Docs How to start?
46. anastasiatymo Demo time!👩💻
47. anastasiatymo Books for further reading ... and many more referenced on https://atymo.me/
48. anastasiatymo I would love to hear back from you! https://www.meetup.com/PyBerlin/ https://atymo.me/ https://github.com/atymoshchuk/can_we_deploy_yet
49. anastasiatymo Thank you! Questions?