Tweet
Tweet

Slide 1

Slide 1 text

Concurrency with Gevent Marconi Moreto @marconimjr

Slide 2

Slide 2 text

Concurrency From Wikipedia: “... concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other” Gevent “gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop” From http://www.gevent.org

Slide 3

Slide 3 text

Concurrency is not Parallelism “Concurrency is not parallelism, although it enables parallelism. If you have only one processor, your program can still be concurrent but it cannot be parallel.” - Rob Pike

Slide 4

Slide 4 text

Lets see some code Stand back, I know concurrency

Slide 5

Slide 5 text

import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous

Slide 6

Slide 6 text

import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output:

Slide 7

Slide 7 text

import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)

Slide 8

Slide 8 text

import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop

Slide 9

Slide 9 text

import time import random def worker(multiplier): time.sleep(random.random()) print '*' * multiplier for i in range(1, 6): worker(i) Synchronous * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) Start Stop

Slide 10

Slide 10 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers

Slide 11

Slide 11 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output:

Slide 12

Slide 12 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)

Slide 13

Slide 13 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1

Slide 14

Slide 14 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2

Slide 15

Slide 15 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2

Slide 16

Slide 16 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4

Slide 17

Slide 17 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) Asynchronous printing workers ***** **** * ** *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5

Slide 18

Slide 18 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout

Slide 19

Slide 19 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output:

Slide 20

Slide 20 text

import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)

Slide 21

Slide 21 text

1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5)

Slide 22

Slide 22 text

1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2

Slide 23

Slide 23 text

1 import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3

Slide 24

Slide 24 text

1 Timeout import gevent import random def worker(multiplier): gevent.sleep(random.random()) print '*' * multiplier greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets, timeout=0.5) Workers with timeout **** * *** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 2 3

Slide 25

Slide 25 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result

Slide 26

Slide 26 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output:

Slide 27

Slide 27 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5)

Slide 28

Slide 28 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1

Slide 29

Slide 29 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 1 2

Slide 30

Slide 30 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2

Slide 31

Slide 31 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4

Slide 32

Slide 32 text

import random import gevent def worker(multiplier): gevent.sleep(random.random()) return '*' * multiplier def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) return [g.value for g in greenlets] print '\n'.join(producers()) Collecting workers result * ** *** **** ***** Output: worker(1) worker(2) worker(3) worker(4) worker(5) 3 1 2 4 5

Slide 33

Slide 33 text

import random import gevent from gevent.queue import Queue tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2)

Slide 34

Slide 34 text

import random import gevent from gevent.queue import Queue tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): greenlets = [gevent.spawn(worker, i) for i in range(1, 6)] gevent.joinall(greenlets) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Producers and consumer (1/2) *** * ** **** ***** Output:

Slide 35

Slide 35 text

worker(1) worker(2) worker(3) worker(4) worker(5) Producers and consumer (2/2) Queue Head

Slide 36

Slide 36 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head

Slide 37

Slide 37 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head

Slide 38

Slide 38 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head ***

Slide 39

Slide 39 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head ***

Slide 40

Slide 40 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head Consumer ***

Slide 41

Slide 41 text

worker(1) worker(2) worker(3) worker(4) worker(5) 1 Producers and consumer (2/2) Queue Head Consumer ***

Slide 42

Slide 42 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer ***

Slide 43

Slide 43 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer ***

Slide 44

Slide 44 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 1 Producers and consumer (2/2) Queue Head Consumer *** *

Slide 45

Slide 45 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** *

Slide 46

Slide 46 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** *

Slide 47

Slide 47 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **

Slide 48

Slide 48 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **

Slide 49

Slide 49 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * **

Slide 50

Slide 50 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****

Slide 51

Slide 51 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****

Slide 52

Slide 52 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** ****

Slide 53

Slide 53 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer *** * ** **** *****

Slide 54

Slide 54 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 5 4 3 1 Producers and consumer (2/2) Queue Head Consumer

Slide 55

Slide 55 text

import random import gevent from gevent.queue import Queue from gevent.pool import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2)

Slide 56

Slide 56 text

import random import gevent from gevent.queue import Queue from gevent.pool import Pool pool = Pool(2) tasks = Queue() def worker(multiplier): gevent.sleep(random.random()) tasks.put('*' * multiplier) def producers(): pool.map(worker, range(1, 6)) def consumer(): while True: print tasks.get() gevent.spawn(consumer) producers() Worker pool (1/2) ** * **** ***** *** Output:

Slide 57

Slide 57 text

Worker pool (2/2) Queue Head

Slide 58

Slide 58 text

worker(1) worker(2) Worker pool (2/2) Queue Head

Slide 59

Slide 59 text

worker(1) worker(2) 1 Worker pool (2/2) Queue Head

Slide 60

Slide 60 text

worker(1) worker(2) 1 Worker pool (2/2) Queue Head

Slide 61

Slide 61 text

worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head

Slide 62

Slide 62 text

worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head

Slide 63

Slide 63 text

worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer

Slide 64

Slide 64 text

worker(1) worker(2) 1 Worker pool (2/2) ** Queue Head Consumer

Slide 65

Slide 65 text

worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head Consumer

Slide 66

Slide 66 text

worker(1) worker(2) 2 1 Worker pool (2/2) ** Queue Head Consumer

Slide 67

Slide 67 text

worker(1) worker(2) 2 1 Worker pool (2/2) ** * Queue Head Consumer

Slide 68

Slide 68 text

worker(1) worker(2) worker(4) worker(5) 2 1 Worker pool (2/2) ** * Queue Head Consumer

Slide 69

Slide 69 text

worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * Queue Head Consumer

Slide 70

Slide 70 text

worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * Queue Head Consumer

Slide 71

Slide 71 text

worker(1) worker(2) worker(4) worker(5) 2 3 1 Worker pool (2/2) ** * **** Queue Head Consumer

Slide 72

Slide 72 text

worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** Queue Head Consumer

Slide 73

Slide 73 text

worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** Queue Head Consumer

Slide 74

Slide 74 text

worker(1) worker(2) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** ***** Queue Head Consumer

Slide 75

Slide 75 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 Worker pool (2/2) ** * **** ***** Queue Head Consumer

Slide 76

Slide 76 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** Queue Head Consumer

Slide 77

Slide 77 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** Queue Head Consumer

Slide 78

Slide 78 text

worker(1) worker(2) worker(3) worker(4) worker(5) 2 4 3 1 5 Worker pool (2/2) ** * **** ***** *** Queue Head Consumer

Slide 79

Slide 79 text

But, but, what about real examples?

Slide 80

Slide 80 text

Shorty: URL shortener - TCP - Message framing

Slide 81

Slide 81 text

Shorty: URL shortener Client - TCP - Message framing

Slide 82

Slide 82 text

Shorty: URL shortener Client Server - TCP - Message framing

Slide 83

Slide 83 text

Shorty: URL shortener Client Server Long URL: Length + Payload - TCP - Message framing

Slide 84

Slide 84 text

Shorty: URL shortener Client Server Long URL: Length + Payload - TCP - Message framing

Slide 85

Slide 85 text

Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing

Slide 86

Slide 86 text

Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing

Slide 87

Slide 87 text

Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload - TCP - Message framing Print short URL

Slide 88

Slide 88 text

Shorty: URL shortener Client Server Long URL: Length + Payload Shortened URL: Length + Payload Close enough :P - TCP - Message framing Print short URL

Slide 89

Slide 89 text

... server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(500) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) time.sleep(1) # delay # read payload payload_len_buf = read_bytes(conn, PLEN_BUF_SIZE) payload_len = struct.unpack('

Slide 90

Slide 90 text

Client ... def run(): # connect to server client = socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('

Slide 91

Slide 91 text

Client ... def run(): # connect to server client = socket() client.connect(('', 9000)) # send payload payload = 'http://127.0.0.1:5000/{0}'.format(uuid.uuid4()) payload_len = struct.pack('

Slide 92

Slide 92 text

So how does it perform?

Slide 93

Slide 93 text

... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__ == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server

Slide 94

Slide 94 text

... CONCURRENCY = 10 ... class Agent(Thread): ... if __name__ == '__main__': runner = Runner() start = time.time() for _ in range(CONCURRENCY): agent = Agent(runner) agent.setDaemon(True) agent.start() print 'spawned {0} agents'.format(runner.spawned_agents) while runner.running_agents > 0: time.sleep(1) print 'connections/second: {0}'.format(runner.conn_per_sec) runner.reset_conn_per_sec() end = time.time() elapsed = end - start print 'took: {0}'.format(elapsed) Benchmark against blocking server Output: ... took: 11.0153269768

Slide 95

Slide 95 text

What did we discover? - Blocks on each request - Very slow - Poor performance - Isn’t web scale

Slide 96

Slide 96 text

import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server

Slide 97

Slide 97 text

import gevent from gevent import monkey; monkey.patch_socket() from gevent.pool import Pool from gevent.socket import socket ... ... pool = Pool(100) server = socket(AF_INET, SOCK_STREAM) server.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) server.bind(('', 9000)) server.listen(50) def handle(conn, addr): logger.info('connected to {0}'.format(addr)) gevent.sleep(1) # delay ... # accept and handle incoming client connections while True: conn, addr = server.accept() pool.spawn(handle, conn, addr) Non-blocking server ... took: 2.00628781319 Output:

Slide 98

Slide 98 text

How about now? - Doesn’t block on each request - Signi!cantly faster - Better performance

Slide 99

Slide 99 text

Links Gevent home page: http://www.gevent.org/ Gevent for working python developer: http://sdiehl.github.io/gevent-tutorial/ Multi-part !le downloader using gevent: https://github.com/marconi/pullite/tree/experiment Slides and source !les: https://github.com/pizzapy/oct2013-meetup

Slide 100

Slide 100 text

Thank you Marconi Moreto @marconimjr http://marconijr.com https://github.com/marconi