the thundering herd and other concurrency programming challenges Pycon JP 2014 - Tokyo By Benoit Chesneau

GUNICORN DESIGN

prefork-worker model share sockets between OS processes workers are monitored allows different kind of workers gunicorn design

arbiter worker spawn & share sockets share socket

each worker waits (accepts) for a connection the OS balance connections in the backlog sync worker accept only 1 connection at a time async workers accept many connections competition to accept

arbiter worker notify monitor workers kill

the arbiter workflow Signal check the 
 signal queue Put in signal queue handle the signal not empty empty murder workers sleep manage workers

different concurrency strategy different platforms support don’t want to handle WSGI only more… why different workers?

THE THUNDERING HERD
 PROBLEM

All process can be awoken on accept for the same connection can trigger EAGAIN increase the CPU usage for nothing. not true with modern OSes for 1 socket the thundering herd problem

ticket Gunicorn#792 Introduce signaling between workers and the arbiter using a Pipe Use our own internal messaging implementation the arbiter checks if the a socket can read Workers asks for more work or notify they are alive fixing thundering herd: how to solve it

arbiter worker new signaling send message

Messages Arbiter Worker ACCEPT:SOCKET ALIVE:TIMESTAMP ACCEPT

the arbiter has a new job All workers need to be updated new signaling API to be handled by the worker: an helper module will be provided introduce complexity

with one socket, let all workers accepting if wanted depending on the load, bypass the signaling possible optimisations

ONE COMMAND TO RULE THEM ALL

no more gunicorn_django, gunicorn_paster fix preloading issue offer the same experience for all WSGI applications new settings: —paste, —pythonpath, —env Only one command in 20.0 (deprecations are removed) gunicorn - the only command

Enhance the user experience

ticket Gunicorn#837 plugins everywhere: logging, workers more embeddable: the gunicorn engine A new organization: 
 http:/ /github.com/gunicorn WIP: Deadline 2014/12 Enhance the user experience.

? @benoitc Support me https:/ /gratipay.com/benoitc/