Tweet
Tweet

Slide 1

Slide 1 text

Yield a

Slide 2

Slide 2 text

Yield from a

Slide 3

Slide 3 text

There Are Things • asyncio fundementals • asyncio.Future • Generators (yield) • PEP 380 (yield from) • Coroutines • asyncio examples

Slide 4

Slide 4 text

There’s No Time • asyncio fundementals • asyncio.Future • Generators (yield) • PEP 380 (yield from) • Coroutines • asyncio examples

Slide 5

Slide 5 text

Me • Call me TP • Follow @uranusjr • uranusjr.com

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

www. .com

Slide 8

Slide 8 text

Asynchrony • “Tell me when you’re done.” • Non-blocking API • Event loop

Slide 9

Slide 9 text

Event Loop?

Slide 10

Slide 10 text

• “Reactor” • In charge of dispatching events for tasks • Tasks give up control when they don’t need it Event Loop

Slide 11

Slide 11 text

import asyncio! ! def print_and_repeat(loop):! print('Hello World')! loop.call_later(! 2,! print_and_repeat,! loop! )! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_repeat(loop)! loop.run_forever() https://code.google.com/p/tulip/source/browse/examples/hello_callback.py

Slide 12

Slide 12 text

import asyncio! ! def print_and_repeat(loop):! print('Hello World')! loop.call_later(! 2,! print_and_repeat,! loop! )! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_repeat(loop)! loop.run_forever()

Slide 13

Slide 13 text

import asyncio! ! def print_and_repeat(loop):! print('Hello World')! loop.call_later(! 2,! print_and_repeat,! loop! )! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_repeat(loop)! loop.run_forever()

Slide 14

Slide 14 text

import asyncio! ! def print_and_repeat(loop):! print('Hello World')! loop.call_later(! 2,! print_and_repeat,! loop! )! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_repeat(loop)! loop.run_forever()

Slide 15

Slide 15 text

• Takes registration for callbacks • Calls them when it should • Hard to persist states Event Loop

Slide 16

Slide 16 text

Future

Slide 17

Slide 17 text

Future • Deferred? Promises/A+? • “Run this for me and tell me the result.” • Like concurrent.futures.Future

Slide 18

Slide 18 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 19

Slide 19 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 20

Slide 20 text

def sleep(secs):! f = asyncio.Future(loop=None)! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 21

Slide 21 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 22

Slide 22 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 23

Slide 23 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 24

Slide 24 text

class Future:! # …! def add_done_callback(self, fn):! # …! self._callbacks.append(fn)! ! def set_result(self, result):! # …! self._schedule_callbacks()! ! def _schedule_callbacks(self):! callbacks = self._callbacks[:]! # …! for cb in callbacks:! self._loop.call_soon(cb, self) http://hg.python.org/cpython/file/3.4/Lib/asyncio/futures.py

Slide 25

Slide 25 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule(None)! loop.run_forever()

Slide 26

Slide 26 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 27

Slide 27 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever() called by event loop (after 2 secs)

Slide 28

Slide 28 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(*args, **kwargs):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever() calls

Slide 29

Slide 29 text

future print_and_schedule event loop Initial invocation .__init__ .call_later .set_result .add_done_callback call as callback Starts running

Slide 30

Slide 30 text

Future • Encapsulates the asynchronous execution of a callable. • Callbacks are bundled with it • Event loop calls them for you

Slide 31

Slide 31 text

Coroutine

Slide 32

Slide 32 text

def producer():! yield 0! yield 1! ! g = producer()! print(type(g)) # ! ! for i in g:! print(i)! # 0! # 1 Generators

Slide 33

Slide 33 text

def producer():! yield 0! yield 1! ! g = producer()! print(type(g)) # ! ! for i in g:! print(i)! # 0! # 1 Generators

Slide 34

Slide 34 text

def producer():! yield 0! yield 1! ! g = producer()! print(type(g)) # ! ! for i in g:! print(i)! # 0! # 1 Generators

Slide 35

Slide 35 text

def producer():! yield 0! yield 1! ! g = producer()! print(type(g)) # ! ! for i in g:! print(i)! # 0! # 1 Generators

Slide 36

Slide 36 text

g = producer()! while True:! try:! i = next(g) # Python 3! print(i)! except StopIteration:! break! # 0! # 1

Slide 37

Slide 37 text

g = producer()! while True:! try:! i = g.__next__() # Python 3! print(i)! except StopIteration:! break! # 0! # 1

Slide 38

Slide 38 text

g = producer()! while True:! try:! i = g.send(None)! print(i)! except StopIteration:! break! # 0! # 1 http://www.python.org/dev/peps/pep-0342/

Slide 39

Slide 39 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None) # Starts g.! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 40

Slide 40 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 41

Slide 41 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 42

Slide 42 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 43

Slide 43 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 44

Slide 44 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 45

Slide 45 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 46

Slide 46 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 47

Slide 47 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 48

Slide 48 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 49

Slide 49 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs No more yielding.

Slide 50

Slide 50 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs No more yielding.

Slide 51

Slide 51 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Con: 0 ! Gen: 1! Con: 1! Gen: 2 Outputs

Slide 52

Slide 52 text

[Coroutines] allow multiple entry points for suspending and resuming execution at certain locations. http://en.wikipedia.org/wiki/Coroutine

Slide 53

Slide 53 text

def get_generator():! v = yield 0! print('Gen:', v)! v = yield v! print('Gen:', v)! ! g = get_generator()! v = g.send(None)! ! while True:! try:! print('Con:', v)! v = g.send(v + 1)! except StopIteration:! break Coroutines?

Slide 54

Slide 54 text

Generators • “Semi-coroutines” • Always yield to its caller • yield is for passing back a value to parent

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

Generators • “Semi-coroutines” • Always yield to its caller • yield is for passing back a value to parent

Slide 57

Slide 57 text

Generators • “Semi-coroutines” • Always yield to its caller • yield is for passing back a value to parent Event loop Future instance

Slide 58

Slide 58 text

yield from • PEP 380 and Python 3.3 • Sub-generator delegation • yield the results from another generator

Slide 59

Slide 59 text

I cannot possibly do this justice. — Guido, PyCon 2013

Slide 60

Slide 60 text

yield from • Exceptions bubbling • Return value without round-trip to scheduler • Cleaner/simpler API

Slide 61

Slide 61 text

def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f! ! def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 62

Slide 62 text

! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! return f

Slide 63

Slide 63 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f

Slide 64

Slide 64 text

def print_and_schedule(future):! print('Hello World')! f = sleep(2)! f.add_done_callback(print_and_schedule)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! print_and_schedule()! loop.run_forever()

Slide 65

Slide 65 text

@asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! f = sleep(2)! yield from f! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! loop.run_until_complete(! print_and_schedule! )

Slide 66

Slide 66 text

@asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! f = sleep(2)! yield from f! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! loop.run_until_complete(! print_and_schedule! )

Slide 67

Slide 67 text

@asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! f = sleep(2)! yield from f! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! loop.run_until_complete(! print_and_schedule! )

Slide 68

Slide 68 text

@asyncio.coroutine! def sleep(s):! f = asyncio.Future()! f._loop.call_later(s, f.set_result, None)! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2)! ! if __name__ == '__main__':! loop = asyncio.get_event_loop()! loop.run_until_complete(! print_and_schedule! )

Slide 69

Slide 69 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2)

Slide 70

Slide 70 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2) called by event loop

Slide 71

Slide 71 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2)

Slide 72

Slide 72 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2) yield to event loop

Slide 73

Slide 73 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2) Later, by event loop

Slide 74

Slide 74 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2) Resumes with g.send(None)

Slide 75

Slide 75 text

@asyncio.coroutine! def sleep(secs):! f = asyncio.Future()! f._loop.call_later(! secs, f.set_result, None! )! yield from f! ! @asyncio.coroutine! def print_and_schedule():! while True:! print('Hello World')! yield from sleep(2)

Slide 76

Slide 76 text

I got this.

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

import asyncio! from aiohttp import request! ! @asyncio.coroutine! def get_content(url):! resp = yield from request('GET', url)! content = yield from resp.read_and_close()! return content! ! @asyncio.roroutine! def get_them(url):! url = yield from get_content(url)! url = yield from get_content(url)! url = yield from get_content(url)! print(url)

Slide 79

Slide 79 text

Bottom Line • Asynchrony is cool • Promises are pluscool • Coroutines are doublepluscool

Slide 80

Slide 80 text

The Rest • How asyncio works • Return values from coroutines • Error-handling with yield from • …

Slide 81

Slide 81 text

Further Reading • PEPs 255, 342, 380, and 3148 • PyCon 2013 Keynote - Guido van Rossum • The difference between yield and yield-from • Deconstructing Deferred

Slide 82

Slide 82 text

No content