Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gunicorn, the thundering herd and other concurrency programming challenges

Benoit Chesneau
September 13, 2014

Gunicorn, the thundering herd and other concurrency programming challenges

Gunicorn design has not changed much since the project started. With the increasing number of cores and new concurrency solutions that appeared in python, it is about time to face to new challenges.

This talk will describe the design of Gunicorn, the different solutions we provide to handle more and more concurrent connections. How we fix the thundering herd problem, what solutions are possible and what we finally adopted. It will also show how we improved the support of Python 3 with the addition of the workers gaiohttp and gthreads and how you can use them. While we are here, the talk will recapitulate the different strategies you have to handle the connections coming to your application using the different workers depending on your concurrent programming framework or platform. It will finally highlight the recent changes in the support of different Python web frameworks.

Talk given at Pycon JP 2014

Benoit Chesneau

September 13, 2014

More Decks by Benoit Chesneau

Other Decks in Programming


  1. each worker waits (accepts) for a connection the OS balance

    connections in the backlog sync worker accept only 1 connection at a time async workers accept many connections competition to accept
  2. the arbiter workflow Signal check the 
 signal queue Put

    in signal queue handle the signal not empty empty murder workers sleep manage workers
  3. All process can be awoken on accept for the same

    connection can trigger EAGAIN increase the CPU usage for nothing. not true with modern OSes for 1 socket the thundering herd problem
  4. ticket Gunicorn#792 Introduce signaling between workers and the arbiter using

    a Pipe Use our own internal messaging implementation the arbiter checks if the a socket can read Workers asks for more work or notify they are alive fixing thundering herd: how to solve it
  5. the arbiter has a new job All workers need to

    be updated new signaling API to be handled by the worker: an helper module will be provided introduce complexity
  6. with one socket, let all workers accepting if wanted depending

    on the load, bypass the signaling possible optimisations
  7. no more gunicorn_django, gunicorn_paster fix preloading issue offer the same

    experience for all WSGI applications new settings: —paste, —pythonpath, —env Only one command in 20.0 (deprecations are removed) gunicorn - the only command
  8. ticket Gunicorn#837 plugins everywhere: logging, workers more embeddable: the gunicorn

    engine A new organization: 
 http:/ /github.com/gunicorn WIP: Deadline 2014/12 Enhance the user experience.