INTRODUCTION TO GEVENT a presentation by Isman Firmansyah 

$ WHOAMI on Twitter @iromli on GitHub iromli http://groovematic.com/ 

WHAT IS GEVENT gevent is a -based Python networking library that uses to provide a high-level synchronous API on top of the event loop. coroutine greenlet libevent 

GEVENT, UNDER THE HOOD http://gevent.org/ 

COROUTINE TL;DR http://en.wikipedia.org/wiki/Coroutine or in a nutshell: Well, a coroutine is a subroutine or function that can pause in the middle and return a value. When you call it again it picks up right where it was when you paused, environment intact, and keeps on going. http://is.gd/J2Zq0R 

GREENLET http://greenlet.readthedocs.org/ A lightweight micro-thread. 

LIBEVENT http://libevent.org/ A fast event loop; supports epoll, kqueue, select, etc. 

WHEN TO USE GEVENT THE CONCLUSION (SO FAR) Use gevent for concurrent execution!! 

CONCURRENCY DOES MATTER Concurrency is the composition of independently executing computations. It simulates the real-world. → Optimize performance and scalability. → http://is.gd/G0ewD8 http://is.gd/IsYrKF 

FOR WEB DEVELOPERS: TACKLE THE C10K PROBLEM http://www.kegel.com/c10k.html Handle ten thousand clients simultaneously!! 

CONCURRENCY IN PYTHON BATTERIES INCLUDED 

THREADING GIL problems!! Might requires locking mechanism CPU-bound programs run significantly worse 

MULTIPROCESSING Process-based "threading" Supports multi-cores Data passed between process must compatible with pickle 

CONCURRENT.FUTURES Mimic of Java's concurrent.futures Available in Python 3.2+ Backport for Python 2.5 - 3.1 → https://pypi.python.org/pypi/futures 

TULIP http://www.python.org/dev/peps/pep-3156/ Available in Python 3.4 ?? 

CONCURRENCY IN PYTHON 3RD-PARTY OPTIONS 

TWISTED http://twistedmatrix.com/ OH: "Twisted is heavy, I agree but not bloated." uses Twisted as its core Graphite 

TORNADO http://www.tornadoweb.org/ Battle-tested (originally developed at FriendFeed) Avoid libraries which doesn't compatible with Tornado's IOLoop 

GEVENT ... 

WHY NOT X - - - - - ... PyPy NodeJS Erlang Elixir Go Let's talk after this session ends :) 

SERIOUSLY, WHY GEVENT ... AND WHY YOU SHOULD TRY IT 

SYNCHRONOUS API Write program that looks blocking, but actually runs asynchronously. # blocking for x in xrange(1, 11): time.sleep() # non-blocking ops = [] for x in xrange(1, 11): ops.append(gevent.spawn(time.sleep)) gevent.joinall(ops) 

NO CALLBACK-WORRIES Forget the callback!! def _collection_handler(self, callback=None, on_error=None): def inner(fd, ev): try: self._db.read_query_result() resultset = self._db.use_result() rows = self._get_rows(resultset) if callback: callback(rows) except Exception, exc: self._throw_error(exc, on_error) finally: self._ioloop.remove_handler(self._db.fd) return inner 

MONKEY PATCHING UTILITY Patches stdlib and 3rd-party modules to become cooperative. import gevent.monkey gevent.monkey.patch_all() ops = [] for x in xrange(1, 11): ops.append(gevent.spawn(time.sleep)) gevent.joinall(ops) Caveats: cannot be used for C-extension 

FAST SERVERS Based on libevent-http. wsgi.WSGIServer pyswgi.WSGIServer: supports SSL server.StreamServer: generic TCP server 

HUGE COLLECTIONS OF COMPATIBLE LIBRARIES Thanks to monkey patching, we have abundant async libraries: MySQL => pymysql/ultramysql Redis => redis-py Beanstalk => beanstalkc Postgres => psycopg (with specific setup) ... and more 

TOOLS BUILT ON TOP/SUPPORTS GEVENT / : load testing : Python implementation of the Socket.IO : Requests + Gevent : WSGI server ... and more locust.io boom gevent-socketio grequests gunicorn 

GEVENT CRASH COURSE HOW TO ...?? 

SPAWNING GREENLET import gevent from gevent import Greenlet def foo(message, n): gevent.sleep(n) print message # Initialize a new Greenlet instance running the named function foo thread1 = Greenlet.spawn(foo, "Hello", 1) # Wrapper for Greenlet.spawn thread2 = gevent.spawn(foo, "I live!", 2) threads = [thread1, thread2] # Block until all threads complete. gevent.joinall(threads) 

MONITORING GREENLET Use Greenlet states: started, ready, successful(), value, and exception. Mostly, you'll only need value and exception. 

RETRIEVING THE VALUE FROM A GREENLET Use greenlet.value: import gevent def win(): return 'You win!' if __name__ == "__main__": winner = gevent.spawn(win) gevent.joinall([winner]) print "winner value:", winner.value # "You win!" 

HANDLING EXCEPTION Use greenlet.exception: import gevent def fail(): raise Exception('You fail at failing.') if __name__ == "__main__": loser = gevent.spawn(fail) # Exceptions raised in the Greenlet, stay inside the Greenlet. try: gevent.joinall([loser]) except Exception as exc: print "This will never be reached!" # It is possible though to raise the exception again outside #raise loser.exception # Or using #loser.get() # A stack trace will be printed to stdout but it # will not unwind the stack of the parent. print "loser exception:", loser.exception 

BONUS: THE HELLO WORLD OF CONCURRENT PROGRAMMING 

WITHOUT GEVENT import time def boring(): print "without gevent" for x in xrange(1, 11): print "boring!", x time.sleep(1) if __name__ == "__main__": boring() 

WITH GEVENT import gevent.monkey gevent.monkey.patch_all() import time def green(): print "with gevent" ops = [] for x in xrange(1, 11): print "boring!", x ops.append(gevent.spawn(time.sleep)) gevent.joinall(ops) if __name__ == "__main__": green() 

MORE TUTORIALS http://sdiehl.github.io/gevent-tutorial/ 

QUESTIONS ANYONE? 

THE END THANKS FOR LISTENING