Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gunicorn, the thundering herd and other concurrency programming challenges

F04edc7cb2099745e5413c754d3d22f5?s=47 Benoit Chesneau
September 13, 2014

Gunicorn, the thundering herd and other concurrency programming challenges

Gunicorn design has not changed much since the project started. With the increasing number of cores and new concurrency solutions that appeared in python, it is about time to face to new challenges.

This talk will describe the design of Gunicorn, the different solutions we provide to handle more and more concurrent connections. How we fix the thundering herd problem, what solutions are possible and what we finally adopted. It will also show how we improved the support of Python 3 with the addition of the workers gaiohttp and gthreads and how you can use them. While we are here, the talk will recapitulate the different strategies you have to handle the connections coming to your application using the different workers depending on your concurrent programming framework or platform. It will finally highlight the recent changes in the support of different Python web frameworks.

Talk given at Pycon JP 2014

F04edc7cb2099745e5413c754d3d22f5?s=128

Benoit Chesneau

September 13, 2014
Tweet

Transcript

  1. the thundering herd and other concurrency programming challenges Pycon JP

    2014 - Tokyo By Benoit Chesneau
  2. GUNICORN DESIGN

  3. prefork-worker model share sockets between OS processes workers are monitored

    allows different kind of workers gunicorn design
  4. arbiter worker spawn & share sockets share socket

  5. each worker waits (accepts) for a connection the OS balance

    connections in the backlog sync worker accept only 1 connection at a time async workers accept many connections competition to accept
  6. arbiter worker notify monitor workers kill

  7. the arbiter workflow Signal check the 
 signal queue Put

    in signal queue handle the signal not empty empty murder workers sleep manage workers
  8. different concurrency strategy different platforms support don’t want to handle

    WSGI only more… why different workers?
  9. THE THUNDERING HERD
 PROBLEM

  10. All process can be awoken on accept for the same

    connection can trigger EAGAIN increase the CPU usage for nothing. not true with modern OSes for 1 socket the thundering herd problem
  11. ticket Gunicorn#792 Introduce signaling between workers and the arbiter using

    a Pipe Use our own internal messaging implementation the arbiter checks if the a socket can read Workers asks for more work or notify they are alive fixing thundering herd: how to solve it
  12. arbiter worker new signaling send message

  13. Messages Arbiter Worker ACCEPT:SOCKET ALIVE:TIMESTAMP ACCEPT

  14. the arbiter has a new job All workers need to

    be updated new signaling API to be handled by the worker: an helper module will be provided introduce complexity
  15. with one socket, let all workers accepting if wanted depending

    on the load, bypass the signaling possible optimisations
  16. ONE COMMAND TO RULE THEM ALL

  17. no more gunicorn_django, gunicorn_paster fix preloading issue offer the same

    experience for all WSGI applications new settings: —paste, —pythonpath, —env Only one command in 20.0 (deprecations are removed) gunicorn - the only command
  18. Enhance the user experience

  19. ticket Gunicorn#837 plugins everywhere: logging, workers more embeddable: the gunicorn

    engine A new organization: 
 http:/ /github.com/gunicorn WIP: Deadline 2014/12 Enhance the user experience.
  20. ? @benoitc Support me https:/ /gratipay.com/benoitc/