Иван Гончарук. Корутины для самых маленьких

Доклад будет посвящен механизму работы корутин в ЯП Python: какое отношение к ним имеют генераторы, как генераторы превратить в корутины и как реализовать простейший event-loop для асинхронного выполнения корутин.

Transcript

1. Корутины для самых маленьких Гончарук Иван Middle Python Developer Туристическая компания "DAN" Email: mc.ferden@hotmail.com Telegram: @mcferden
2. Asyncio 2 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!')
3. Asyncio 3 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!') >>> loop = asyncio.get_event_loop() >>> loop.create_task(tick()) >>> loop.create_task(tock()) >>> loop.run_forever()
4. Asyncio 4 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!') >>> loop = asyncio.get_event_loop() >>> loop.create_task(tick()) >>> loop.create_task(tock()) >>> loop.run_forever() Now awaiting! Tick!
5. Asyncio 5 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!') >>> loop = asyncio.get_event_loop() >>> loop.create_task(tick()) >>> loop.create_task(tock()) >>> loop.run_forever() Now awaiting! Tick! ...
6. Asyncio 6 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!') >>> loop = asyncio.get_event_loop() >>> loop.create_task(tick()) >>> loop.create_task(tock()) >>> loop.run_forever() Now awaiting! Tick! ... Now awaiting! Tick!
7. Asyncio 7 async def coroutine(): result = 0 for index in range(10): result += index print('Now awaiting!') await asyncio.sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): await asyncio.sleep(0) print('Tick!') >>> loop = asyncio.get_event_loop() >>> loop.create_task(tick()) >>> loop.create_task(tock()) >>> loop.run_forever() Now awaiting! Tick! ... Now awaiting! Tick! 45
8. Дисклеймер 8
9. Дисклеймер • Мало теории, много примеров, 99% кода; 9
10. Дисклеймер • Мало теории, много примеров, 99% кода; • Категорически не рекомендуется использовать увиденное в реальных боевых проектах. 10
11. Корутины (сопрограммы) 11
12. Корутины (сопрограммы) • Обычные подпрограммы; 12
13. Корутины (сопрограммы) • Обычные подпрограммы, но: 13
14. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; 14
15. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; 15
16. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; ◦ передает управление другой корутине; 16
17. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; ◦ передает управление другой корутине; ◦ может быть продолжена с точки приостановки. 17
18. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; ◦ передает управление другой корутине; ◦ может быть продолжена с точки приостановки. • Обобщенные подпрограммы; 18
19. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; ◦ передает управление другой корутине; ◦ может быть продолжена с точки приостановки. • Обобщенные подпрограммы; • Позволяют реализовать кооперативную многозадачность. 19
20. Параллелизм 20
21. Кооперативность 21
22. Корутины (сопрограммы) 22
23. Корутины (сопрограммы) • Не выполняются параллельно*; 23
24. Корутины (сопрограммы) • Не выполняются параллельно*; • Нет "магического" прироста производительности; 24
25. Корутины (сопрограммы) • Не выполняются параллельно*; • Нет "магического" прироста производительности; • В CPU-bound задачах не быстрее последовательного выполнения; 25
26. Корутины (сопрограммы) • Не выполняются параллельно*; • Нет "магического" прироста производительности; • В CPU-bound задачах не быстрее последовательного выполнения; ◦ ThreadPool или ProcessPool; 26
27. Корутины (сопрограммы) • Не выполняются параллельно*; • Нет "магического" прироста производительности; • В CPU-bound задачах не быстрее последовательного выполнения; ◦ ThreadPool или ProcessPool; • Большой прирост производительности в IO-bound задачах. 27
28. Корутины (сопрограммы) • Не выполняются параллельно*; • Нет "магического" прироста производительности; • В CPU-bound задачах не быстрее последовательного выполнения; ◦ ThreadPool или ProcessPool; • Большой прирост производительности в IO-bound задачах. 28 Возможно реализовать схему "Одна корутина — один поток".
29. Корутины (сопрограммы) • Обычные подпрограммы, но: ◦ корутина может быть приостановлена в произвольной точке; ◦ сохраняет свое состояние; ◦ передает управление другой корутине; ◦ может быть продолжена с точки приостановки. • Обобщенные подпрограммы; • Позволяют реализовать кооперативную многозадачность. 29
30. Генераторы def generator(): index = 0 while True: yield index index += 1 30 PEP 255: Simple Generators
31. Генераторы def generator(): index = 0 while True: yield index index += 1 31 >>> gen = generator() PEP 255: Simple Generators
32. Генераторы def generator(): index = 0 while True: yield index index += 1 32 >>> gen = generator() >>> next(gen) 0 PEP 255: Simple Generators
33. Генераторы def generator(): index = 0 while True: yield index index += 1 33 >>> gen = generator() >>> next(gen) 0 >>> next(gen) 1 PEP 255: Simple Generators
34. Генераторы def generator(): index = 0 while True: yield index index += 1 34 >>> gen = generator() >>> next(gen) 0 >>> next(gen) 1 >>> next(gen) 2 PEP 255: Simple Generators
35. Генераторы def generator(): index = 0 while True: yield index index += 1 35 >>> gen = generator() >>> next(gen) 0 >>> next(gen) 1 >>> next(gen) 2 Функция-генератор PEP 255: Simple Generators
36. Генераторы def generator(): index = 0 while True: yield index index += 1 36 >>> gen = generator() >>> next(gen) 0 >>> next(gen) 1 >>> next(gen) 2 Функция-генератор Объект-генератор PEP 255: Simple Generators
37. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 37 PEP 342: Coroutines via Enhanced Generators
38. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 38 >>> gen = generator() PEP 342: Coroutines via Enhanced Generators
39. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 39 >>> gen = generator() >>> next(gen) 0 PEP 342: Coroutines via Enhanced Generators
40. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 40 >>> gen = generator() >>> next(gen) 0 >>> gen.send(10) 10 PEP 342: Coroutines via Enhanced Generators
41. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 41 >>> gen = generator() >>> next(gen) 0 >>> gen.send(10) 10 10 PEP 342: Coroutines via Enhanced Generators
42. Генераторы def generator(): index = 0 while True: step = yield index index += step or 1 42 >>> gen = generator() >>> next(gen) 0 >>> gen.send(10) 10 >>> next(gen) 11 10 PEP 342: Coroutines via Enhanced Generators
43. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except ValueError: print('Oh!') 43 >>> gen = generator() >>> next(gen) 0 >>> gen.throw(ValueError) Oh! 0 PEP 342: Coroutines via Enhanced Generators
44. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except ValueError: print('Oh!') 44 >>> gen = generator() >>> next(gen) 0 >>> gen.throw(ValueError) Oh! 0 ValueError PEP 342: Coroutines via Enhanced Generators
45. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except ValueError: print('Oh!') 45 >>> gen = generator() >>> next(gen) 0 >>> gen.throw(ValueError) Oh! 0 >>> next(gen) 1 ValueError PEP 342: Coroutines via Enhanced Generators
46. Генераторы def generator(): index = 0 while True: yield index index += 1 46 >>> gen = generator() >>> next(gen) 0 >>> gen.close() PEP 342: Coroutines via Enhanced Generators
47. Генераторы def generator(): index = 0 while True: yield index index += 1 47 >>> gen = generator() >>> next(gen) 0 >>> gen.close() GeneratorExit PEP 342: Coroutines via Enhanced Generators
48. Генераторы def generator(): index = 0 while True: yield index index += 1 48 >>> gen = generator() >>> next(gen) 0 >>> gen.close() >>> next(gen) StopIteration GeneratorExit PEP 342: Coroutines via Enhanced Generators
49. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except GeneratorExit: print('Oh!') 49 PEP 342: Coroutines via Enhanced Generators
50. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except GeneratorExit: print('Oh!') 50 >>> gen = generator() >>> next(gen) 0 >>> gen.close() Oh! RuntimeError: generator ignored GeneratorExit PEP 342: Coroutines via Enhanced Generators
51. Генераторы def generator(): index = 0 while True: try: yield index index += 1 except GeneratorExit: print('Oh!') break 51 >>> gen = generator() >>> next(gen) 0 >>> gen.close() Oh! >>> next(gen) StopIteration PEP 342: Coroutines via Enhanced Generators
52. Генераторы def generator(): try: index = 0 while True: yield index index += 1 except GeneratorExit: print('Oh!') 52 >>> gen = generator() >>> next(gen) 0 >>> gen.close() Oh! >>> next(gen) StopIteration PEP 342: Coroutines via Enhanced Generators
53. Генераторы def generator(): try: index = 0 while True: yield index index += 1 except GeneratorExit: print('Oh!') 53 >>> gen = generator() >>> next(gen) 0 >>> del gen Oh! PEP 342: Coroutines via Enhanced Generators
54. Генераторы def recscan(path): for entry in os.scandir(path): if entry.is_dir(): yield entry for child in recscan(entry.path): yield child else: yield entry 54 PEP 380: Syntax for Delegating to a Subgenerator
55. Генераторы def recscan(path): for entry in os.scandir(path): if entry.is_dir(): yield entry for child in recscan(entry.path): yield child else: yield entry 55 >>> for entry in recscan('/some/path'): ... print(entry.name) ... PEP 380: Syntax for Delegating to a Subgenerator
56. Генераторы def recscan(path): for entry in os.scandir(path): if entry.is_dir(): yield entry for child in recscan(entry.path): yield child else: yield entry 56 >>> for entry in recscan('/some/path'): ... print(entry.name) ... foo/ foo/bar/ foo/bar/baz foo/qux PEP 380: Syntax for Delegating to a Subgenerator
57. Генераторы def recscan(path): for entry in os.scandir(path): if entry.is_dir(): yield entry yield from recscan(entry.path) else: yield entry 57 >>> for entry in recscan('/some/path'): ... print(entry.name) ... foo/ foo/bar/ foo/bar/baz foo/qux PEP 380: Syntax for Delegating to a Subgenerator
58. Генераторы def recscan(path): for entry in os.scandir(path): if entry.is_dir(): yield entry yield from recscan(entry.path) else: yield entry 58 >>> for entry in recscan('/some/path'): ... print(entry.name) ... foo/ foo/bar/ foo/bar/baz foo/qux PEP 380: Syntax for Delegating to a Subgenerator Автоматические next, send, throw и close
59. Генераторы def recscan(path): count = 0 for entry in os.scandir(path): count += 1 if entry.is_dir(): yield entry count += yield from recscan(entry.path) else: yield entry return count 59 PEP 380: Syntax for Delegating to a Subgenerator
60. Генераторы def recscan(path): count = 0 for entry in os.scandir(path): count += 1 if entry.is_dir(): yield entry count += yield from recscan(entry.path) else: yield entry return count 60 PEP 380: Syntax for Delegating to a Subgenerator Значение return или StopIteration
61. Генераторы >>> gen = recscan('/some/path') 61 PEP 380: Syntax for Delegating to a Subgenerator
62. Генераторы >>> gen = recscan('/some/path') >>> while True: ... try: ... entry = next(gen) ... print(entry.name) ... except StopIteration as e: ... print(e.value) ... break ... 62 PEP 380: Syntax for Delegating to a Subgenerator
63. Генераторы >>> gen = recscan('/some/path') >>> while True: ... try: ... entry = next(gen) ... print(entry.name) ... except StopIteration as e: ... print(e.value) ... break ... foo/ foo/bar/ foo/bar/baz foo/qux 4 63 PEP 380: Syntax for Delegating to a Subgenerator
64. Генераторы >>> gen = recscan('/some/path') >>> while True: ... try: ... entry = next(gen) ... print(entry.name) ... except StopIteration as e: ... print(e.value) ... break ... foo/ foo/bar/ foo/bar/baz foo/qux 4 64 PEP 380: Syntax for Delegating to a Subgenerator
65. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 65 PEP 255: Simple Generators
66. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 66 >>> tick_coro = tick() >>> tock_coro = tock() PEP 255: Simple Generators
67. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 67 >>> tick_coro = tick() >>> tock_coro = tock() >>> next(tick_coro) Tick! PEP 255: Simple Generators
68. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 68 >>> tick_coro = tick() >>> tock_coro = tock() >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! PEP 255: Simple Generators
69. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 69 >>> tick_coro = tick() >>> tock_coro = tock() >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! >>> next(tick_coro) Tick! PEP 255: Simple Generators
70. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 70 >>> tick_coro = tick() >>> tock_coro = tock() >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! PEP 255: Simple Generators
71. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 71 >>> tick_coro = tick() >>> tock_coro = tock() >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! >>> next(tick_coro) Tick! >>> next(tock_coro) Tock! ... PEP 255: Simple Generators
72. Корутины class Scheduler: def __init__(self): self.queue = deque() def add(self, callback): self.queue.append(callback) def run(self): while self.queue: callback = self.queue.popleft() callback() 72 PEP 255: Simple Generators
73. Корутины class Scheduler: def __init__(self): self.queue = deque() def add(self, callback): self.queue.append(callback) def run(self): while self.queue: callback = self.queue.popleft() callback() 73 PEP 255: Simple Generators
74. Корутины class Scheduler: def __init__(self): self.queue = deque() def add(self, callback): self.queue.append(callback) def run(self): while self.queue: callback = self.queue.popleft() callback() 74 PEP 255: Simple Generators
75. Корутины class Scheduler: def __init__(self): self.queue = deque() def add(self, callback): self.queue.append(callback) def run(self): while self.queue: callback = self.queue.popleft() callback() 75 PEP 255: Simple Generators
76. Корутины class Task: def __init__(self, coroutine, scheduler): self.coroutine = coroutine self.scheduler = scheduler self.schedule() def step(self): try: next(self.coroutine) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 76 PEP 255: Simple Generators
77. Корутины class Task: def __init__(self, coroutine, scheduler): self.coroutine = coroutine self.scheduler = scheduler self.schedule() def step(self): try: next(self.coroutine) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 77 PEP 255: Simple Generators
78. Корутины class Task: def __init__(self, coroutine, scheduler): self.coroutine = coroutine self.scheduler = scheduler self.schedule() def step(self): try: next(self.coroutine) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 78 PEP 255: Simple Generators
79. Корутины class Task: def __init__(self, coroutine, scheduler): self.coroutine = coroutine self.scheduler = scheduler self.schedule() def step(self): try: next(self.coroutine) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 79 PEP 255: Simple Generators
80. Корутины class Scheduler: def __init__(self): self.queue = deque() def add(self, callback): self.queue.append(callback) def run(self): while self.queue: callback = self.queue.popleft() callback() def create_task(self, coroutine): task = Task(coroutine, self) return task 80 PEP 255: Simple Generators
81. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 81 PEP 255: Simple Generators
82. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 82 >>> scheduler = Scheduler() PEP 255: Simple Generators
83. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 83 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) PEP 255: Simple Generators
84. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 84 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() PEP 255: Simple Generators
85. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 85 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! PEP 255: Simple Generators
86. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 86 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! Tock! PEP 255: Simple Generators
87. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 87 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! Tock! Tick! PEP 255: Simple Generators
88. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 88 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! Tock! Tick! Tock! PEP 255: Simple Generators
89. Корутины def tick(): for _ in range(10): print('Tick!') yield def tock(): for _ in range(10): print('Tock!') yield 89 >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! Tock! Tick! Tock! ... PEP 255: Simple Generators
90. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 90 PEP 255: Simple Generators
91. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 91 PEP 255: Simple Generators
92. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 92 PEP 255: Simple Generators
93. Корутины class Task: def __init__(self, coroutine, scheduler): self.coroutine = coroutine self.stack = [] self.scheduler = scheduler self.schedule() ... 93 PEP 255: Simple Generators
94. Корутины ... def step(self): try: yielded = next(self.coroutine) except StopIteration: if self.stack: self.coroutine = self.stack.pop() self.schedule() else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule() ... 94 PEP 255: Simple Generators
95. Корутины ... def step(self): try: yielded = next(self.coroutine) except StopIteration: if self.stack: self.coroutine = self.stack.pop() self.schedule() else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule() ... 95 PEP 255: Simple Generators
96. Корутины ... def step(self): try: yielded = next(self.coroutine) except StopIteration: if self.stack: self.coroutine = self.stack.pop() self.schedule() else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule() ... 96 PEP 255: Simple Generators
97. Корутины ... def step(self): try: yielded = next(self.coroutine) except StopIteration: if self.stack: self.coroutine = self.stack.pop() self.schedule() else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule() ... 97 PEP 255: Simple Generators
98. Корутины ... def step(self): try: yielded = next(self.coroutine) except StopIteration: if self.stack: self.coroutine = self.stack.pop() self.schedule() else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule() ... 98 PEP 255: Simple Generators
99. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 99 PEP 255: Simple Generators
100. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 100 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run()
101. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 101 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick!
102. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 102 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек>
103. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 103 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock!
104. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 104 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock! <пауза в 1 сек>
105. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 105 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock! <пауза в 1 сек> Tick!
106. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 106 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock! <пауза в 1 сек> Tick! <пауза в 1 сек>
107. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 107 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock! <пауза в 1 сек> Tick! <пауза в 1 сек> Tock!
108. Корутины def sleep(delay=0): d = time() + delay while True: yield if d <= time(): break def tick(): for _ in range(10): print('Tick!') yield sleep(2) def tock(): yield sleep(1) for _ in range(10): print('Tock!') yield sleep(2) 108 PEP 255: Simple Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Tick! <пауза в 1 сек> Tock! <пауза в 1 сек> Tick! <пауза в 1 сек> Tock! ...
109. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 109 PEP 342: Coroutines via Enhanced Generators
110. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 110 PEP 342: Coroutines via Enhanced Generators
111. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 111 PEP 342: Coroutines via Enhanced Generators
112. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 112 PEP 342: Coroutines via Enhanced Generators
113. Корутины def step(self, value=None, error=None): try: if error: yielded = self.coroutine.throw(error) else: yielded = self.coroutine.send(value) except StopIteration as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(value=e.value) except Exception as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(error=e) else: raise else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule(value=yielded) 113 PEP 342: Coroutines via Enhanced Generators
114. Корутины def step(self, value=None, error=None): ... 114 PEP 342: Coroutines via Enhanced Generators
115. Корутины ... try: if error: yielded = self.coroutine.throw(error) else: yielded = self.coroutine.send(value) ... 115 PEP 342: Coroutines via Enhanced Generators
116. Корутины ... except StopIteration as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(value=e.value) ... 116 PEP 342: Coroutines via Enhanced Generators
117. Корутины ... except Exception as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(error=e) else: raise ... 117 PEP 342: Coroutines via Enhanced Generators
118. Корутины ... else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule(value=yielded) 118 PEP 342: Coroutines via Enhanced Generators
119. Корутины def schedule(self, value=None, error=None): self.scheduler.add( functools.partial( self.step, value=value, error=error, ) ) 119 PEP 342: Coroutines via Enhanced Generators
120. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 120 PEP 342: Coroutines via Enhanced Generators
121. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 121 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run()
122. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 122 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick!
123. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 123 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ...
124. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 124 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ... Now yielding! Tick!
125. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 125 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ... Now yielding! Tick! 45
126. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = yield coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 126 PEP 342: Coroutines via Enhanced Generators >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ... Now yielding! Tick! 45 StopIteration: 45
127. Корутины def step(self, value=None, error=None): try: if error: yielded = self.coroutine.throw(error) else: yielded = self.coroutine.send(value) except StopIteration as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(value=e.value) except Exception as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(error=e) else: raise else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule(value=yielded) 127 PEP 380: Syntax for Delegating to a Subgenerator
128. Корутины def step(self, value=None, error=None): try: if error: yielded = self.coroutine.throw(error) else: yielded = self.coroutine.send(value) except StopIteration as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(value=e.value) except Exception as e: if self.stack: self.coroutine = self.stack.pop() self.schedule(error=e) else: raise else: if inspect.isgenerator(yielded): self.stack.append(self.coroutine) self.coroutine = yielded self.schedule() else: self.schedule(value=yielded) 128 PEP 380: Syntax for Delegating to a Subgenerator
129. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 129 PEP 380: Syntax for Delegating to a Subgenerator
130. Корутины def step(self): try: self.coroutine.send(None) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 130 PEP 380: Syntax for Delegating to a Subgenerator
131. Корутины def step(self): try: self.coroutine.send(None) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 131 PEP 380: Syntax for Delegating to a Subgenerator
132. Корутины def step(self): try: self.coroutine.send(None) except StopIteration: pass else: self.schedule() def schedule(self): self.scheduler.add(self.step) 132 PEP 380: Syntax for Delegating to a Subgenerator
133. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 133 PEP 380: Syntax for Delegating to a Subgenerator
134. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 134 PEP 380: Syntax for Delegating to a Subgenerator >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run()
135. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 135 PEP 380: Syntax for Delegating to a Subgenerator >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick!
136. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 136 PEP 380: Syntax for Delegating to a Subgenerator >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ...
137. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 137 PEP 380: Syntax for Delegating to a Subgenerator >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ... Now yielding! Tick!
138. Корутины def coroutine(): result = 0 for number in range(10): result += number print('Now yielding!') yield return result def tick(): result = \ yield from coroutine() print(result) def tock(): for _ in range(10): print('Tick!') yield 138 PEP 380: Syntax for Delegating to a Subgenerator >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now yielding! Tick! ... Now yielding! Tick! 45
139. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 139 PEP 492: Coroutines with async and await syntax
140. Awaitable 140 PEP 492: Coroutines with async and await syntax
141. Awaitable • Нативная корутина (async def); 141 PEP 492: Coroutines with async and await syntax
142. Awaitable • Нативная корутина (async def); • Корутина-генератор, помеченная декоратором types.coroutine; 142 PEP 492: Coroutines with async and await syntax
143. Awaitable • Нативная корутина (async def); • Корутина-генератор, помеченная декоратором types.coroutine; • Объект с "магическим" методом __await__, возвращающим генератор. 143 PEP 492: Coroutines with async and await syntax
144. Awaitable • Нативная корутина (async def); • Корутина-генератор, помеченная декоратором types.coroutine; • Объект с "магическим" методом __await__, возвращающим генератор. !!! Цепочка await / yield from всегда заканчивается простым yield. 144 PEP 492: Coroutines with async and await syntax
145. Async/await @types.coroutine def sleep(delay=0): d = time.time() + delay while True: yield if d <= time.time(): break 145 PEP 492: Coroutines with async and await syntax
146. Async/await class sleep: def __init__(self, delay=0): self.delay = delay def __await__(self): delay = self.delay d = time.time() + delay while True: yield if d <= time.time(): break 146 PEP 492: Coroutines with async and await syntax
147. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 147 PEP 492: Coroutines with async and await syntax
148. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 148 PEP 492: Coroutines with async and await syntax >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run()
149. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 149 PEP 492: Coroutines with async and await syntax >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now awaiting! Tick!
150. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 150 PEP 492: Coroutines with async and await syntax >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now awaiting! Tick! ...
151. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 151 PEP 492: Coroutines with async and await syntax >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now awaiting! Tick! ... Now awaiting! Tick!
152. Async/await async def coroutine(): result = 0 for number in range(10): result += number print('Now awaiting!') await sleep(0) return result async def tick(): result = await coroutine() print(result) async def tock(): for _ in range(10): print('Tick!') await sleep(0) 152 PEP 492: Coroutines with async and await syntax >>> scheduler = Scheduler() >>> scheduler.create_task(tick()) >>> scheduler.create_task(tock()) >>> scheduler.run() Now awaiting! Tick! ... Now awaiting! Tick! 45
153. Выводы 153
154. Выводы • Корутины = генераторы; 154
155. Выводы • Корутины = генераторы; • Корутины ≠ магия; 155
156. Выводы • Корутины = генераторы; • Корутины ≠ магия; • Корутины ≠ серебряная пуля; 156
157. Выводы • Корутины = генераторы; • Корутины ≠ магия; • Корутины ≠ серебряная пуля; • asyncio проще, чем может показаться. 157
158. Спасибо за внимание Вопросы?