How to Write Deployment-friendly Applications

The DevOps movement gave us many ways to put Python applications into production. But how can you practically structure and configure your applications to make them indifferent to the environment they run in? How do secrets fit into the picture? And where do you put that log file?

Transcript

1. How to Write
   
   Deployment-friendly
   
   Applications
   Hynek Schlawack
   

   View Slide

2. View Slide

3. View Slide

4. views.py
   from pyramid.response import Response
   from pyramid.view import view_config
   @view_config(route_name="hello")
   def hello_world(request):
   return Response("Hello World!")
   

   View Slide

5. from pyramid.config import Configurator
   from .views import hello_world
   def make_app():
   config = Configurator()
   config.add_route("hello", "/hello")
   config.scan()
   return config.make_wsgi_app()
   app_maker.py
   

   View Slide

6. from pyramid.config import Configurator
   from .views import hello_world
   def make_app():
   config = Configurator()
   config.add_route("hello", "/hello")
   config.scan()
   return config.make_wsgi_app()
   app_maker.py
   

   View Slide

7. $ gunicorn \
   --access-logfile - \
   "sample.app_maker:make_app()"
   

   View Slide

8. $ gunicorn \
   --access-logfile - \
   "sample.app_maker:make_app()"
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Starting gunicorn 19.7.1
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Listening at: http://127.0.0.1:8000 (54343)
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Using worker: sync
   [2018-04-05 16:28:06 +0200] [54346] [INFO] Booting worker with pid: 54346
   

   View Slide

9. $ gunicorn \
   --access-logfile - \
   "sample.app_maker:make_app()"
   $ curl http://127.0.0.1:8000/hello
   Hello World!
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Starting gunicorn 19.7.1
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Listening at: http://127.0.0.1:8000 (54343)
   [2018-04-05 16:28:06 +0200] [54343] [INFO] Using worker: sync
   [2018-04-05 16:28:06 +0200] [54346] [INFO] Booting worker with pid: 54346
   

   View Slide

10. $ gunicorn \
    --access-logfile - \
    "sample.app_maker:make_app()"
    $ curl http://127.0.0.1:8000/hello
    Hello World!
    [2018-04-05 16:28:06 +0200] [54343] [INFO] Starting gunicorn 19.7.1
    [2018-04-05 16:28:06 +0200] [54343] [INFO] Listening at: http://127.0.0.1:8000 (54343)
    [2018-04-05 16:28:06 +0200] [54343] [INFO] Using worker: sync
    [2018-04-05 16:28:06 +0200] [54346] [INFO] Booting worker with pid: 54346
    127.0.0.1 - - [05/Apr/2018:16:28:08 +0200] "GET /hello HTTP/1.1" 200 12 "-" "curl/7.54.0"
    

    View Slide

11. Zoom Out
    

    View Slide

12. View Slide

13. #!/bin/bash
    exec 2>&1 \
    gunicorn \
    --access-logfile - \
    "sample.app_maker:make_app()"
    run-app.sh
    

    View Slide

14. #!/bin/bash
    exec 2>&1 \
    gunicorn \
    --access-logfile - \
    "sample.app_maker:make_app()"
    run-app.sh
    

    View Slide

15. #!/bin/bash
    exec 2>&1 \
    gunicorn \
    --access-logfile - \
    "sample.app_maker:make_app()"
    run-app.sh
    

    View Slide

16. ExecStart=/app/run-app.sh
    systemd
    

    View Slide

17. ENTRYPOINT ["/app/run-app.sh"]
    Dockerfile
    

    View Slide

18. web: ./run-app.sh
    Procfile
    

    View Slide

19. App
    

    View Slide

20. ./run-app.sh App
    

    View Slide

21. 127.0.0.1:8000
    Exposes
    ./run-app.sh App
    

    View Slide

22. 127.0.0.1:8000
    Exposes
    ./run-app.sh Logs
    stdout
    App
    

    View Slide

23. Configuration
    

    View Slide

24. What Varies?
    

    View Slide

25. What Varies?
    Very Little
    

    View Slide

26. [server]
    host=127.0.0.1
    port=8000
    log_level=INFO
    log_format=human
    

    View Slide

27. Environment
    
    Variables
    

    View Slide

28. Environment=LOG_FORMAT=human
    Environment=LOG_LEVEL=INFO
    ExecStart=/app/run-app.sh
    systemd
    

    View Slide

29. ENV LOG_FORMAT=human
    ENV LOG_LEVEL=INFO
    ENTRYPOINT ["/app/run-app.sh"]
    Dockerfile
    

    View Slide

30. View Slide

31. direnv
    

    View Slide

32. envconsul direnv
    etcdenv
    

    View Slide

33. envconsul direnv
    etcdenv
    os.environ
    

    View Slide

34. envconsul direnv
    etcdenv
    envsubst
    os.environ
    

    View Slide

35. envconsul direnv
    etcdenv
    envsubst
    consul-template
    confd
    os.environ
    

    View Slide

36. $ env HOST=0.0.0.0 PORT=8888 LOG_LEVEL=INFO \
    ./run-app.sh
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Starting gunicorn 19.7.1
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Listening at: http://0.0.0.0:8888
    

    View Slide

37. $ env HOST=0.0.0.0 PORT=8888 LOG_LEVEL=INFO \
    ./run-app.sh
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Starting gunicorn 19.7.1
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Listening at: http://0.0.0.0:8888
    

    View Slide

38. $ env HOST=0.0.0.0 PORT=8888 LOG_LEVEL=INFO \
    ./run-app.sh
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Starting gunicorn 19.7.1
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Listening at: http://0.0.0.0:8888
    

    View Slide

39. $ env HOST=0.0.0.0 PORT=8888 LOG_LEVEL=INFO \
    ./run-app.sh
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Starting gunicorn 19.7.1
    [2018-04-09 15:59:29 +0200] [35323] [INFO] Listening at: http://0.0.0.0:8888
    logging.basicConfig(
    level=getattr(
    logging,
    os.environ["LOG_LEVEL"],
    ),
    format="%(message)s",
    stream=sys.stdout,
    )
    

    View Slide

40. import environ
    @environ.config(prefix="APP")
    class AppConfig:
    @environ.config
    class Log:
    level = environ.var()
    format = environ.var()
    log = environ.group(Log)
    config.py
    

    View Slide

41. APP_LOG_LEVEL → app_cfg.log.level
    import environ
    @environ.config(prefix="APP")
    class AppConfig:
    @environ.config
    class Log:
    level = environ.var()
    format = environ.var()
    log = environ.group(Log)
    config.py
    

    View Slide

42. import environ
    from .config import AppConfig
    from .app_maker import make_app
    app_cfg = environ.to_config(AppConfig)
    application = make_app(app_cfg)
    wsgi.py
    

    View Slide

43. import environ
    from .config import AppConfig
    from .app_maker import make_app
    app_cfg = environ.to_config(AppConfig)
    application = make_app(app_cfg)
    wsgi.py
    

    View Slide

44. #!/bin/bash
    exec 2>&1 \
    gunicorn \
    --access-logfile - \
    "sample.app"
    run-app.sh
    

    View Slide

45. View Slide

46. Don’t Put
    
    Sensitive Data
    
    Into Env Variables
    

    View Slide

47. View Slide

48. HashiCorp Vault
    AWS Secrets Manager
    Google Cloud KMS
    Microsoft Azure Key Vault
    Docker Secrets
    

    View Slide

49. HashiCorp Vault
    AWS Secrets Manager
    Google Cloud KMS
    Microsoft Azure Key Vault
    Docker Secrets
    

    View Slide

50. Open Space: Secrets
    Room 10 @ 5pm
    

    View Slide

51. secrets.py
    

    View Slide

52. class VaultSecrets:
    # ...
    def get_db_url(self):
    return self.vault.read(
    f"secret/{self.env}/app-name"
    )["data"]["db_url"])
    secrets.py
    

    View Slide

53. class FakeSecrets:
    def get_db_url(self):
    return (
    "postgresql://user@localhost/db"
    )
    secrets.py
    

    View Slide

54. App
    

    View Slide

55. env HOST=0.0.0.0 \
    PORT=8000 \
    LOG_LEVEL=INFO \
    ./run-app.sh App
    

    View Slide

56. env HOST=0.0.0.0 \
    PORT=8000 \
    LOG_LEVEL=INFO \
    ./run-app.sh
    Secrets
    App
    

    View Slide

57. 0.0.0.0:8000
    Exposes
    env HOST=0.0.0.0 \
    PORT=8000 \
    LOG_LEVEL=INFO \
    ./run-app.sh
    Secrets
    App
    

    View Slide

58. 0.0.0.0:8000
    Exposes
    env HOST=0.0.0.0 \
    PORT=8000 \
    LOG_LEVEL=INFO \
    ./run-app.sh
    Secrets
    stdout
    @ INFO
    Logs
    @ INFO
    App
    

    View Slide

59. 127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    

    View Slide

60. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    

    View Slide

61. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

62. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    Offline
    

    View Slide

63. SIGTERM
    

    View Slide

64. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

65. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

66. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

67. View Slide

68. Readiness
    

    View Slide

69. Readiness
    •/healthz
    

    View Slide

70. Readiness
    •/healthz
    •/__heartbeat__
    

    View Slide

71. Readiness
    •/healthz
    •/__heartbeat__
    •/-/ready
    

    View Slide

72. Readiness
    •/healthz
    •/__heartbeat__
    •/-/ready
    •/-/readiness
    

    View Slide

73. HAProxy
    http-request deny if { path_beg /-/ }
    

    View Slide

74. Liveness
    

    View Slide

75. Liveness
    •/-/healthy
    

    View Slide

76. Liveness
    •/-/healthy
    •/-/liveness
    

    View Slide

77. Liveness
    •/-/healthy
    •/-/liveness
    •__lbheartbeat__
    

    View Slide

78. @view_config(
    route_name="ready",
    permission=NO_PERMISSION_REQUIRED,
    )
    def ready(request):
    return Response("yep")
    

    View Slide

79. @view_config(
    route_name="ready",
    permission=NO_PERMISSION_REQUIRED,
    )
    def ready(request):
    return Response("yep")
    config.add_route("ready", "/-/ready")
    

    View Slide

80. Moar
    

    View Slide

81. Moar
    •__version__
    

    View Slide

82. Moar
    •__version__
    •/-/metrics
    

    View Slide

83. Moar
    •__version__
    •/-/metrics
    •/-/log-level
    

    View Slide

84. 127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

85. App
    127.0.0.1:8003
    Exposes
    env HOST=127.0.0.1 \
    PORT=8002 \
    ./run-app.sh
    127.0.0.1:8002
    Exposes
    env HOST=127.0.0.1 \
    PORT=8002 \
    ./run-app.sh
    App
    127.0.0.1:8001
    Exposes
    env HOST=127.0.0.1 \
    PORT=8001 \
    ./run-app.sh
    App
    127.0.0.1:8000
    Exposes
    env HOST=127.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

86. App
    10.0.0.4:8000
    Exposes
    env HOST=10.0.0.4 \
    PORT=8000 \
    ./run-app.sh
    10.0.0.3:8000
    Exposes
    env HOST=10.0.0.3 \
    PORT=8000 \
    ./run-app.sh
    App
    10.0.0.2:8000
    Exposes
    env HOST=10.0.0.2 \
    PORT=8000 \
    ./run-app.sh
    App
    10.0.0.1:8000
    Exposes
    env HOST=10.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

87. View Slide

88. View Slide

89. View Slide

90. View Slide

91. App
    10.0.0.4:8000
    Exposes
    env HOST=10.0.0.4 \
    PORT=8000 \
    ./run-app.sh
    10.0.0.3:8000
    Exposes
    env HOST=10.0.0.3 \
    PORT=8000 \
    ./run-app.sh
    App
    10.0.0.2:8000
    Exposes
    env HOST=10.0.0.2 \
    PORT=8000 \
    ./run-app.sh
    App
    10.0.0.1:8000
    Exposes
    env HOST=10.0.0.1 \
    PORT=8000 \
    ./run-app.sh
    App
    $PUBLIC_IP
    Load
    
    Balancer
    

    View Slide

92. View Slide

93. View Slide

94. Open Space: Docker
    Sunday
    
    Room 11 @ 11am
    

    View Slide

95. App
    

    View Slide

96. run-app.sh
    App
    

    View Slide

97. run-app.sh
    env HOST=0.0.0.0 PORT=8000 … App
    

    View Slide

98. run-app.sh
    SIGTERM
    env HOST=0.0.0.0 PORT=8000 … App
    

    View Slide

99. Secrets
    run-app.sh
    SIGTERM
    env HOST=0.0.0.0 PORT=8000 … App
    

    View Slide

100. Secrets
     run-app.sh
     SIGTERM
     External
     Resources
     Keeps data in
     env HOST=0.0.0.0 PORT=8000 … App
     

     View Slide

101. Secrets
     Exposes service
     0.0.0.0:8000
     run-app.sh
     SIGTERM
     External
     Resources
     Keeps data in
     env HOST=0.0.0.0 PORT=8000 … App
     

     View Slide

102. Secrets
     Exposes service
     0.0.0.0:8000
     run-app.sh
     Exposes state
     0.0.0.0:8000/-/*
     SIGTERM
     External
     Resources
     Keeps data in
     env HOST=0.0.0.0 PORT=8000 … App
     

     View Slide

103. Logs
     stdout
     Secrets
     Exposes service
     0.0.0.0:8000
     run-app.sh
     Exposes state
     0.0.0.0:8000/-/*
     SIGTERM
     External
     Resources
     Keeps data in
     env HOST=0.0.0.0 PORT=8000 … App
     

     View Slide

104. View Slide

105. •Loose Coupling
     

     View Slide

106. •Loose Coupling
     •Separate I/O & Logic
     

     View Slide

107. •Loose Coupling
     •Separate I/O & Logic
     •Avoid Global State
     

     View Slide

108. App Boundary
     
     =
     
     Just Another
     Boundary
     

     View Slide

109. Epilogue
     

     View Slide

110. Epilogue
     

     View Slide

111. ox.cx/df
     @hynek
     vrmd.de
     

     View Slide