Concurrency with Gevent

Concurrency with Gevent

Presented last October 2013 meetup of Pizzapy (http://pizzapy.ph/)

0f59545fe4b3d43c7e98710b0c17af9f?s=128

Marconi Moreto

October 29, 2013
Tweet

Transcript

  1. Concurrency with Gevent Marconi Moreto @marconimjr

  2. Concurrency From Wikipedia: “... concurrency is a property of systems

    in which several computations are executing simultaneously, and potentially interacting with each other” Gevent “gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop” From http://www.gevent.org
  3. Concurrency is not Parallelism “Concurrency is not parallelism, although it

    enables parallelism. If you have only one processor, your program can still be concurrent but it cannot be parallel.” - Rob Pike
  4. Lets see some code Stand back, I know concurrency

  5. import time import random def worker(multiplier): time.sleep(random.random()) print '*' *

    multiplier for i in range(1, 6): worker(i) Synchronous
  6. import time import random def worker(multiplier): time.sleep(random.random()) print '*' *

    multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output:
  7. import time import random def worker(multiplier): time.sleep(random.random()) print '*' *

    multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
  8. import time import random def worker(multiplier): time.sleep(random.random()) print '*' *

    multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop
  9. import time import random def worker(multiplier): time.sleep(random.random()) print '*' *

    multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop
  10. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers
  11. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output:
  12. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
  13. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1
  14. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2
  15. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2
  16. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4
  17. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5
  18. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout
  19. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output:
  20. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' *

    multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
  21. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*'

    * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
  22. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*'

    * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2
  23. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*'

    * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3
  24. 1 Timeout import gevent import random def worker(multiplier): gevent.sleep(random.random()) print

    '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3
  25. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result
  26. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output:
  27. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
  28. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1
  29. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2
  30. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2
  31. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4
  32. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' *

    multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5
  33. import random import gevent from gevent.queue import Queue tasks =

    Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2)
  34. import random import gevent from gevent.queue import Queue tasks =

    Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2) *** * ** **** ***** Output:
  35. worker(1) worker(2) worker(3) worker(4) worker(5) Producers and consumer (2/2) Queue

    Head
  36. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head
  37. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head
  38. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head ***
  39. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head ***
  40. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head Consumer ***
  41. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2)

    Queue Head Consumer ***
  42. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer

    (2/2) Queue Head Consumer ***
  43. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer

    (2/2) Queue Head Consumer ***
  44. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer

    (2/2) Queue Head Consumer *** *
  45. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and

    consumer (2/2) Queue Head Consumer *** *
  46. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and

    consumer (2/2) Queue Head Consumer *** *
  47. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and

    consumer (2/2) Queue Head Consumer *** * **
  48. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers

    and consumer (2/2) Queue Head Consumer *** * **
  49. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers

    and consumer (2/2) Queue Head Consumer *** * **
  50. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers

    and consumer (2/2) Queue Head Consumer *** * ** ****
  51. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1

    Producers and consumer (2/2) Queue Head Consumer *** * ** ****
  52. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1

    Producers and consumer (2/2) Queue Head Consumer *** * ** ****
  53. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1

    Producers and consumer (2/2) Queue Head Consumer *** * ** **** *****
  54. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1

    Producers and consumer (2/2) Queue Head Consumer
  55. import random import gevent from gevent.queue import Queue from gevent.pool

    import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2)
  56. import random import gevent from gevent.queue import Queue from gevent.pool

    import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2) ** * **** ***** *** Output:
  57. Worker pool (2/2) Queue Head

  58. worker(1) worker(2) Worker pool (2/2) Queue Head

  59. worker(1) worker(2) 1 Worker pool (2/2) Queue Head

  60. worker(1) worker(2) 1 Worker pool (2/2) Queue Head

  61. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head

  62. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head

  63. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer

  64. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer

  65. worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head

    Consumer
  66. worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head

    Consumer
  67. worker(1) worker(2) 2 1 Worker pool (2/2) ** * Queue

    Head Consumer
  68. worker(1) worker(2) worker(4) worker(5) 2 1 Worker pool (2/2) **

    * Queue Head Consumer
  69. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2)

    ** * Queue Head Consumer
  70. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2)

    ** * Queue Head Consumer
  71. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2)

    ** * **** Queue Head Consumer
  72. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool

    (2/2) ** * **** Queue Head Consumer
  73. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool

    (2/2) ** * **** Queue Head Consumer
  74. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool

    (2/2) ** * **** ***** Queue Head Consumer
  75. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Worker

    pool (2/2) ** * **** ***** Queue Head Consumer
  76. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5

    Worker pool (2/2) ** * **** ***** Queue Head Consumer
  77. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5

    Worker pool (2/2) ** * **** ***** Queue Head Consumer
  78. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5

    Worker pool (2/2) ** * **** ***** *** Queue Head Consumer
  79. But, but, what about real examples?

  80. Shorty: URL shortener - TCP - Message framing

  81. Shorty: URL shortener Client - TCP - Message framing

  82. Shorty: URL shortener Client Server - TCP - Message framing

  83. Shorty: URL shortener Client Server Long URL: Length + Payload

    - TCP - Message framing
  84. Shorty: URL shortener Client Server Long URL: Length + Payload

    - TCP - Message framing
  85. Shorty: URL shortener Client Server Long URL: Length + Payload

    Shortened URL: Length + Payload - TCP - Message framing
  86. Shorty: URL shortener Client Server Long URL: Length + Payload

    Shortened URL: Length + Payload - TCP - Message framing
  87. Shorty: URL shortener Client Server Long URL: Length + Payload

    Shortened URL: Length + Payload - TCP - Message framing Print short URL
  88. Shorty: URL shortener Client Server Long URL: Length + Payload

    Shortened URL: Length + Payload Close enough :P - TCP - Message framing Print short URL
  89. ... server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000))

    server.listen(500) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) time.sleep(1) # delay # read payload payload_len_buf = read_bytes(conn, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(conn, payload_len) # shorten url and send it back short_url = shorten(payload_buf) payload_len = struct.pack('<L', len(short_url)) conn.sendall(payload_len + short_url) conn.close() # accept and handle incoming client connections while True: conn, addr = server.accept() handle(conn, addr) Blocking server
  90. Client ... def run(): # connect to server client =

    socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('<L', len(payload)) client.sendall(payload_len + payload) # read payload payload_len_buf = read_bytes(client, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(client, payload_len) client.close() return payload_buf if __name__ == '__main__': print run()
  91. Client ... def run(): # connect to server client =

    socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('<L', len(payload)) client.sendall(payload_len + payload) # read payload payload_len_buf = read_bytes(client, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(client, payload_len) client.close() return payload_buf if __name__ == '__main__': print run() http://127.0.0.1:5000/867nv Output:
  92. So how does it perform?

  93. ... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__

    == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server
  94. ... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__

    == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server Output: ... took: 11.0153269768
  95. What did we discover? - Blocks on each request -

    Very slow - Poor performance - Isn’t web scale
  96. import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import

    Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server
  97. import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import

    Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server ... took: 2.00628781319 Output:
  98. How about now? - Doesn’t block on each request -

    Signi!cantly faster - Better performance
  99. Links Gevent home page: http://www.gevent.org/ Gevent for working python developer:

    http://sdiehl.github.io/gevent-tutorial/ Multi-part !le downloader using gevent: https://github.com/marconi/pullite/tree/experiment Slides and source !les: https://github.com/pizzapy/oct2013-meetup
  100. Thank you Marconi Moreto @marconimjr http://marconijr.com https://github.com/marconi