Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Solid Python Application Deployments For Everybody EP Edition

Solid Python Application Deployments For Everybody EP Edition

An expanded version of my PyCon US 2013 talk held at EuroPython 2013 in Florence.

Hynek Schlawack

June 28, 2013
Tweet

More Decks by Hynek Schlawack

Other Decks in Technology

Transcript

  1. River Bar, 2013
    Solid Python
    Application
    Deployments For
    Everybody
    Hynek Schlawack

    View Slide

  2. View Slide

  3. @hynek
    http://hynek.me
    http://github.com/hynek
    http://www.variomedia.de
    H!

    View Slide

  4. ?

    View Slide

  5. AHEAD

    View Slide

  6. http://ox.cx/d
    Te Oe & Ol Ln

    View Slide

  7. OPINIONS
    AHEAD

    View Slide

  8. PaaS
    Schema Migrations

    View Slide

  9. View Slide

  10. Ky
    Cnet

    View Slide

  11. easy

    simple

    View Slide

  12. View Slide

  13. “Simplicity is
    prerequisite
    for reliability.”
    — Edsger W. Dijkstra

    View Slide

  14. “It is important to
    find simple solutions
    instead of stopping
    as soon as a first
    solution is found.”
    — Donald Knuth

    View Slide

  15. Put effort into
    making your
    deployments
    simple.

    View Slide

  16. View Slide

  17. Dvlpet

    View Slide

  18. Dvlpet

    View Slide

  19. View Slide

  20. No!

    View Slide

  21. View Slide

  22. “Python 2.4 is not
    supported. It came out
    8 years ago. That's older
    than Youtube. Upgrade.”
    — Kenneth Reitz

    View Slide

  23. Sal Pafr
    Key Infrastructure!

    View Slide

  24. Sal Pafr
    Application is tied to server OS version.
    Upgrading servers == updating your app.
    Some servers upgraded?

    View Slide

  25. Bt Hynek…
    My boss won’t
    let me!

    View Slide

  26. tests!
    Dvlpet

    View Slide

  27. View Slide

  28. אל

    View Slide

  29. spotty
    outdated
    loss of control
    Sse Pcae

    View Slide

  30. spotty
    outdated
    loss of control
    Sse Pcae

    View Slide

  31. spotty
    outdated
    loss of control
    Sse Pcae

    View Slide

  32. View Slide

  33. Ue vruln
    $ virtualenv venv; . venv/bin/activate
    $ pip install pyramid requests pytest
    $ py.test

    $ pip freeze >requirements.txt

    $ pip install -r requirements.txt

    View Slide

  34. Pn Dp Hr
    “Django == 1.4.3”
    Don’t rely on SemVer!
    update w/ pip-tools

    View Slide

  35. SECURITY!
    Bt Hynek…

    View Slide

  36. Scrt!?
    It’s your Job.

    View Slide

  37. Si I

    View Slide

  38. + git

    View Slide

  39. + git
    Ne!

    View Slide

  40. Fabric

    View Slide

  41. build tools
    repetitive
    downloads
    Wa’s Wog!?

    View Slide

  42. View Slide

  43. .rpm
    .deb
    .pkg.tgz

    View Slide

  44. introspection
    CM integration
    versatile
    Ntv Pcae !?

    View Slide

  45. 1. check out from VCS
    2. create virtualenv
    3. install dependencies
    4. do whatever you want
    5. package result
    6. push to your repo

    View Slide

  46. 1. check out from VCS
    2. create virtualenv
    3. install dependencies
    4. do whatever you want
    5. package result
    6. push to your repo

    View Slide

  47. Abuse the Pipeline
    run tests
    LESS/SASS/CoffeeScript
    compression
    cache busting

    View Slide

  48. Packaging is hard!
    Bt Hynek…

    View Slide

  49. fpm
    Np.

    View Slide

  50. fpm \
    -s dir \
    -t deb \

    View Slide

  51. repo server
    Bt Hynek…

    View Slide

  52. Rp Sre
    dpkg -i
    tar.bz2

    View Slide

  53. Atmt!
    e

    View Slide

  54. app_name: whois
    project: DOM
    build_deps:
    - libpq-dev
    run_deps:
    - libpq5
    - authbind

    View Slide

  55. Tee’s mr ta
    oe wy t d i…

    View Slide

  56. View Slide


  57. View Slide

  58. Cn grto
    Mngmn
    declarative
    describe the goal
    CM choses the path

    View Slide

  59. Sltos
    prise-oriented features to
    to compare the two
    pet Open
    ource
    Puppet
    Enterprise

    ✔ ✔

    View Slide

  60. prise-oriented features to
    to compare the two
    pet Open
    ource
    Puppet
    Enterprise

    ✔ ✔
    Not easy at all.
    Sltos

    View Slide

  61. Wy aya?
    safety/security
    reproducible
    “later”

    View Slide

  62. safety/security
    reproducible
    “later”
    Wy aya?

    View Slide

  63. safety/security
    reproducible
    “later”
    Wy aya?

    View Slide

  64. Ts I i Saig

    View Slide

  65. r t

    View Slide

  66. r t
    Nein!

    View Slide

  67. Js dn’t.

    View Slide

  68. Piiee Pr
    drop privileges
    authbind

    View Slide

  69. Need dat POWER!
    Bt Hynek…

    View Slide

  70. Snl Proe
    Wres
    celery
    rq
    zerorpc
    perspective broker/AMP

    View Slide

  71. B Prni
    /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban

    View Slide

  72. /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban
    B Prni

    View Slide

  73. /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban
    B Prni

    View Slide

  74. /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban
    B Prni

    View Slide

  75. /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban
    B Prni

    View Slide

  76. /bin/false
    iptables
    file
    sockets
    REVOKE
    ALL
    SSL
    fail2ban
    B Prni

    View Slide

  77. $ ./manage.py runserver ▌
    [0] 0:bash*

    View Slide

  78. View Slide

  79. $ ./manage.py runserver ▌
    [0] 0:bash*
    ᔒ༗!

    View Slide

  80. I’s Es!
    upstart
    systemd
    supervisord
    circus

    View Slide

  81. I’s Es!
    upstart
    systemd
    supervisord
    circus

    View Slide

  82. Eape: usat
    $ cat /etc/init/yourapp.conf
    start on static-network-up
    stop on deconfiguring-networking
    respawn
    chdir /path/to/yourapp
    setuid yourapp
    exec /path/to/gunicorn_django settings.py
    $ start yourapp

    View Slide

  83. Lg
    log to stderr
    redirect stderr syslog
    use OS tools

    View Slide

  84. Lg

    [uwsgi]
    log-syslog = your-app

    twistd --syslog --prefix your-app …

    View Slide

  85. Lg
    if $programname == 'you-app' \
    then /var/log/your-app.log
    & ~

    View Slide

  86. + mod_wsgi

    View Slide

  87. + mod_wsgi
    Нет!

    View Slide

  88. Dslie
    Using Apache is
    perfectly fine.

    View Slide

  89. Iff you decide
    consciously
    for it.
    Dslie

    View Slide

  90. mod_wsgi

    View Slide

  91. mod_wsgi ?
    ?

    View Slide

  92. +
    g
    or

    View Slide

  93. +
    g
    or
    Better separation
    of concerns.

    View Slide

  94. Es t St U: gncr
    $ gunicorn_django settings.py
    $ gunicorn_paster settings.ini

    View Slide

  95. $ cat settings.py

    INSTALLED_APPS = (

    "gunicorn",
    )

    $ manage.py run_gunicorn
    Es t St U: gncr

    View Slide

  96. location / {
    proxy_pass unix:///tmp/app.sock;
    }
    location /static/ {
    root /your/app/public/;
    }
    Es t St U: nix

    View Slide

  97. Fo Es t
    AEOE

    View Slide

  98. Text

    View Slide

  99. Sil Es: usi
    uwsgi --emperor production.ini

    [uwsgi]
    paste = config:%p
    uwsgi-socket = /tmp/app.sock
    processes = 2

    View Slide

  100. location / {
    include uwsgi_params;
    uwsgi_param UWSGI_SCHEME $scheme;
    uwsgi_pass unix:///tmp/app.sock;
    }
    Sil Es To: nix

    View Slide

  101. Dpo!

    View Slide

  102. Rlbc!

    View Slide

  103. Mntr

    View Slide

  104. Mntr

    View Slide

  105. View Slide

  106. View Slide

  107. Mntr

    View Slide

  108. Maue
    statsd
    graphite
    yunomi

    View Slide

  109. View Slide

  110. View Slide

  111. Maue
    statsd
    graphite
    yunomi

    View Slide

  112. gt 1

    View Slide

  113. http://ox.cx/d
    @hynek http://hynek.me
    http://vrmd.de

    View Slide