Concurrency with Gevent

Presented last October 2013 meetup of Pizzapy (http://pizzapy.ph/)

Transcript

1. Concurrency with Gevent Marconi Moreto @marconimjr
2. Concurrency From Wikipedia: “... concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other” Gevent “gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop” From http://www.gevent.org
3. Concurrency is not Parallelism “Concurrency is not parallelism, although it enables parallelism. If you have only one processor, your program can still be concurrent but it cannot be parallel.” - Rob Pike
4. Lets see some code Stand back, I know concurrency
5. import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous
6. import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output:
7. import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
8. import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop
9. import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop
10. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers
11. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output:
12. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
13. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1
14. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2
15. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2
16. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4
17. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5
18. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout
19. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output:
20. import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
21. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
22. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2
23. 1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3
24. 1 Timeout import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3
25. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result
26. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output:
27. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)
28. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1
29. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2
30. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2
31. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4
32. import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5
33. import random import gevent from gevent.queue import Queue tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2)
34. import random import gevent from gevent.queue import Queue tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2) *** * ** **** ***** Output:
35. worker(1) worker(2) worker(3) worker(4) worker(5) Producers and consumer (2/2) Queue Head
36. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head
37. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head
38. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head ***
39. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head ***
40. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head Consumer ***
41. worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head Consumer ***
42. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer ***
43. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer ***
44. worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer *** *
45. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** *
46. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** *
47. worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **
48. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **
49. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **
50. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****
51. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****
52. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****
53. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** **** *****
54. worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer
55. import random import gevent from gevent.queue import Queue from gevent.pool import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2)
56. import random import gevent from gevent.queue import Queue from gevent.pool import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2) ** * **** ***** *** Output:
57. Worker pool (2/2) Queue Head
58. worker(1) worker(2) Worker pool (2/2) Queue Head
59. worker(1) worker(2) 1 Worker pool (2/2) Queue Head
60. worker(1) worker(2) 1 Worker pool (2/2) Queue Head
61. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head
62. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head
63. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer
64. worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer
65. worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head Consumer
66. worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head Consumer
67. worker(1) worker(2) 2 1 Worker pool (2/2) ** * Queue Head Consumer
68. worker(1) worker(2) worker(4) worker(5) 2 1 Worker pool (2/2) ** * Queue Head Consumer
69. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * Queue Head Consumer
70. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * Queue Head Consumer
71. worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * **** Queue Head Consumer
72. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** Queue Head Consumer
73. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** Queue Head Consumer
74. worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** ***** Queue Head Consumer
75. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** ***** Queue Head Consumer
76. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** Queue Head Consumer
77. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** Queue Head Consumer
78. worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** *** Queue Head Consumer
79. But, but, what about real examples?
80. Shorty: URL shortener - TCP - Message framing
81. Shorty: URL shortener Client - TCP - Message framing
82. Shorty: URL shortener Client Server - TCP - Message framing
83. Shorty: URL shortener Client Server Long URL: Length + Payload - TCP - Message framing
84. Shorty: URL shortener Client Server Long URL: Length + Payload - TCP - Message framing
85. Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing
86. Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing
87. Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing Print short URL
88. Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload Close enough :P - TCP - Message framing Print short URL
89. ... server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(500) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) time.sleep(1) # delay # read payload payload_len_buf = read_bytes(conn, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(conn, payload_len) # shorten url and send it back short_url = shorten(payload_buf) payload_len = struct.pack('<L', len(short_url)) conn.sendall(payload_len + short_url) conn.close() # accept and handle incoming client connections while True: conn, addr = server.accept() handle(conn, addr) Blocking server
90. Client ... def run(): # connect to server client = socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('<L', len(payload)) client.sendall(payload_len + payload) # read payload payload_len_buf = read_bytes(client, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(client, payload_len) client.close() return payload_buf if __name__ == '__main__': print run()
91. Client ... def run(): # connect to server client = socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('<L', len(payload)) client.sendall(payload_len + payload) # read payload payload_len_buf = read_bytes(client, PLEN_BUF_SIZE) payload_len = struct.unpack('<L', payload_len_buf)[0] payload_buf = read_bytes(client, payload_len) client.close() return payload_buf if __name__ == '__main__': print run() http://127.0.0.1:5000/867nv Output:
92. So how does it perform?
93. ... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__ == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server
94. ... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__ == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server Output: ... took: 11.0153269768
95. What did we discover? - Blocks on each request - Very slow - Poor performance - Isn’t web scale
96. import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server
97. import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server ... took: 2.00628781319 Output:
98. How about now? - Doesn’t block on each request - Signi!cantly faster - Better performance
99. Links Gevent home page: http://www.gevent.org/ Gevent for working python developer: http://sdiehl.github.io/gevent-tutorial/ Multi-part !le downloader using gevent: https://github.com/marconi/pullite/tree/experiment Slides and source !les: https://github.com/pizzapy/oct2013-meetup
100. Thank you Marconi Moreto @marconimjr http://marconijr.com https://github.com/marconi