MORE THAN A WSGI SERVER Python FOSDEM 2016 

WHAT IS GUNICORN 

A WSGI SERVER ▸ PEP 3333 ▸ pre-fork worker model ▸ Python > 2.6x ▸ handle Django & Paste configuration ▸ support any concurrency framework 

ADDITIONS TO WSGI ▸ ALREADY_HANDLED: take control of the socket ▸ ‘gunicorn.socket’ environ ▸ ‘gunicorn.raw_uri’ environ ▸ `transfer-encoding: chunked` 

PREFORK WORKER MODEL ARBITER WORKER WORKER share socket WORKER listen handle signals supervise accept WSGI server handle signals 

SHARE NOTHING ▸ share only file descriptors ▸ Privilege separation ▸ Worker: simple process with a run function 

WORKER accept loop setup signals sockets configuration log parent pid tempfile class MyWorker(WorkerBase): def __init__(..): … def init_process(self): … def run(self): … 

LET IT CRASH ▸ run the application ▸ need to notify the arbiter ▸ tempfile updates ▸ handle application errors ▸ but should crash on errors most of the time 

set application set Config EMBED class MyApp(BaseApplication): def load_config(self): … def load(self): …
 return self.application 

DO MORE > 

DO MORE. BE SAFER. ▸ improve listening in the sync worker ▸ remove the tempfile ▸ prepare for the new protocols ▸ improve logging ▸ going further 

ACCEPT WHEN NEEDED ▸ place the listening sockets in an event loop ▸ on read, start to accept ▸ also have a timer 

IMPORT GUNICORN.IMSG 

GUNICORN.IMSG ▸ communicate between processes ▸ use UNIX socket ▸ sendmsg/recvmsg + buffer ▸ 2 methods: send, receive ▸ zero copy memory 

child parent IMSG EXAMPLE fds =socket.socketpair() pid = os.fork() if pid != 0: # parent socket.close(fds[1]) imsg = IMSG(fds[0] return main(imsg) # child socket.close(fds[0]) var imsg = IMSG(fds[1]) return main(imsg) 

wait for a message send a message IMSG EXAMPLE main(imsg): imsg.send(“sometype”, “somemsg”), dispatch_msg(imsg) dispatch_msg(imsg): while True: msg = imsg.recv() if msg is None: sleep(1) continue print(msg.data) 

REMOVING TEMPFILE ▸ a socket pair is opened for each workers ▸ an imsg object is created for each ▸ imsg object are placed in an eventloop to read asynchronously ▸ maybe extracted from the code as a simple lib 

DETACHED MODE ▸ rework ALREADY_HANDLED ▸ socket control is given to app ▸ but still supervised ▸ simple API 

simple proc abstraction return a proc instance EXAMPLE def app(environ, start_response): ..
 start_response(status, headers) proc = MyProc(environ) return proc class MyProc(gunicorn.Proc): def status(self): return STATUS_ALIVE def terminate(self, Reason): …. 

IMPROVE LOGGING ▸ logging will now be handled in its own process ▸ log infos passed via imsg and dispatched by the arbiter ▸ fix privilege separation issues 

GOING FURTHER ▸ HTTP 2 ▸ static file server? ▸ fastcgi? 

No content

HTTP://GUNICORN.ORG Gunicorn HTTP://ENKIM.EU Enki Multimedia