Tweet
Tweet

Slide 1

Slide 1 text

New in Python 3.11: asyncio.TaskGroup and “Hello-ish World” of Asyncio Junya Fukuda PyCon JP 2022

Slide 2

Slide 2 text

Who are you? •Junya Fukudaʢ@JunyaFffʣ • Japan belongs to Japan System Laboratories, Inc.(JSL) Company in Nagano Prefecture •Web Engineer •GEEKLAB.NAGANO - Community space management •Valuebooks, inc •Aiming to be a service loved by book lovers (used book purchase and sales) Here is today's code

Slide 3

Slide 3 text

Python Practical Recipes •Launched in January 2022 Gijutsu-Hyoron Co., Ltd •Python 3 Engineer Certi fi cation Practice Exam •Use it like a dictionary! • I am responsible for the following chapters • Chapter2 coding rule • Chapter5 Type hints • Chapter19 Concurrency, Parallel Processing

Slide 4

Slide 4 text

asyncio's new features and New "Hello-ish world"

Slide 5

Slide 5 text

asyncio, are you using it? ✋

Slide 6

Slide 6 text

Thank you very much.

Slide 7

Slide 7 text

Right? Why not use it?

Slide 8

Slide 8 text

Review - asyncio basics •Of fi cial library for writing asynchronous processing code •Evolved with each version High level API - very easy to write •I gave such a talk in my book and at PyCon JP 2021. Please do! "4(*ΞϓϦέʔγϣϯೖ໳͜Θ͘ͳ͍BTZODJPجૅͱඇಉظ*0'BTU"1*Λྫʹ 1Z$PO+1 •Handle multiple tasks concurrent •Effective for I/O-bound processing (e.g., DB access/HTTP requests)

Slide 9

Slide 9 text

🤔

Slide 10

Slide 10 text

asyncio I've been trying to think of reasons not to use it. •The synchronous process I'm used to is suf fi cient. •No IO (mostly machine learning and data analysis) (lots of CPU hard work) •Asynchronous processing seems to have unpredictable behavior. •Write in other languages (Go, JS(nodejs/deno...), Rust, C#, Swift, etc.)

Slide 11

Slide 11 text

"What happens when an exception occurs in this process?"

Slide 12

Slide 12 text

Do you have anything to say or be said in a review? "What happens when an exception occurs in this process?"

Slide 13

Slide 13 text

Are you using asyncio? •Asynchronous processing seems to have unpredictable behavior. •that's true •Likely to move unpredictably •I don't know what happens when an exception happens. •What happens if I EXCEPT this parallel running task?" •When an exception occurs, what are the other tasks... •A new feature that solves this

Slide 14

Slide 14 text

asyncio.TaskGroup

Slide 15

Slide 15 text

asyncio.TaskGroup Exceptions and Cancellations

Slide 16

Slide 16 text

Today's Goal •Exception handling and cancellation in using asyncio •Previous (Python 3.10 and earlier) and future (Python 3.11), respectively •Exceptionally easy to write! •This is how cancellations work. •How to write "predictable" and "safe" asyncio •“Hello-ish world” •TaskGroup Let's give it a try.

Slide 17

Slide 17 text

Motivation (why are we talking about this?) •Two big ones. •Python Core Developer Yury Selivanov tweetʢasync/await Authorʣ •When Taskgroup was MERGE, they were posting a series of excited tweets. •I don't understand it after reading the documentation. •I don't know what's so great about it 🤔

Slide 18

Slide 18 text

Motivation (why are we talking about this?) •Using Asyncio in Python, a book that has been a great help to us. •There was a code I didn't understand. •Two big ones. •Quickstart - Example 3-2.

Slide 19

Slide 19 text

•Quickstart - Example 3-2. import asyncio import time async def main(): print(f"{time.ctime()} Hello!") await asyncio.sleep(1.0) print(f"{time.ctime()} GoodBye!") loop = asyncio.get_event_loop() task = loop.create_task(main()) loop.run_until_complete(task) pending = asyncio.all_tasks(loop=loop) for task in pending: task.cancel() group = asyncio.gather(*pending, return_exceptions=True) loop.run_until_complete(group) loop.close()

Slide 20

Slide 20 text

Motivation (why are we talking about this?) •Using Asyncio in Python, a book that has been a great help to us. •There was a code I didn't understand. •Two big ones. •Quickstart - Example 3-2. The “Hello-ish World” of Asyncio •I would like to revamp this code with asyncio.TaskGroup

Slide 21

Slide 21 text

What to say/not to say •Talk. •Exception handling in asyncio (also new except* syntax) •Cancel Task •asyncio's New Hello World •Don't talk. •asyncio basics no more •Speci fi c examples in the application •What's happening at ASGI IUUQTHJUIVCDPNKSGLUBML

Slide 22

Slide 22 text

Attention! •Python 3.11 has not been released yet! •It was scheduled for release on October 3, but is now scheduled for October 24. •Operation is con fi rmed with Python 3.11.0rc2 IUUQTQFQTQZUIPOPSHQFQ •I don't think so, but there is no possibility that asyncio.TaskGroup is a dream function... •This talk is designed to disappear from your mind.

Slide 23

Slide 23 text

asyncio.TaskGroup

Slide 24

Slide 24 text

•Let's start with the of fi cial documentation •Features of asyncio.TaskGroup •Handle multiple tasks at once •It's in Python 3.10 and earlier. •asyncio.gather() and asyncio.wait() asycio.TaskGroup

Slide 25

Slide 25 text

import asyncio async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(some_coro(...)) task2 = tg.create_task(another_coro(...)) print("Both tasks have completed now.") asycio.TaskGroup

Slide 26

Slide 26 text

import asyncio async def main(): await asyncio.gather( some_coro(...), another_coro(...) ) print("Both tasks have completed now.") asycio.TaskGroup

Slide 27

Slide 27 text

•Features of asyncio.TaskGroup •Handle multiple tasks at once •It's in Python 3.10 and earlier. •asyncio.gather() and asyncio.wait() •There are two major features that distinguish it from them asycio.TaskGroup •Catching Concurrent Exceptions •Cancel task when exception occurs

Slide 28

Slide 28 text

Features of asyncio.TaskGroup •Catching Concurrent Exceptions •Cancel task when exception occurs

Slide 29

Slide 29 text

Features of asyncio.TaskGroup •Catching Concurrent Exceptions •Cancel task when exception occurs

Slide 30

Slide 30 text

•Handle multiple tasks at once Catching Concurrent Exceptions •Naturally, exceptions may occur at the same time •Let's see how to capture it in Python 3.10 or earlier •Using the high-level API asyncio.gather() added in Python 3.7 •Options for exceptions to asyncio.gather()ʮreturn_exceptionsʯ •return_exceptions False •return_exceptions True •Simultaneous execution of speci fi ed tasks (asynchronous functions)

Slide 31

Slide 31 text

•Prepare asynchronous functions to run concurrently - 3 coroutines Advance preparation - no change import asyncio async def coro_success(): return "੒ޭ" async def coro_value_err(): raise ValueError async def coro_type_err(): raise TypeError

Slide 32

Slide 32 text

•asyncio.gather - return_exceptions False •asyncio.gather - return_exceptions True

Slide 33

Slide 33 text

•asyncio.gather - return_exceptions False •asyncio.gather - return_exceptions True

Slide 34

Slide 34 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 35

Slide 35 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 36

Slide 36 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 37

Slide 37 text

$ python310 1_asyncio_gather_except.py async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 38

Slide 38 text

$ python310 1_asyncio_gather_except.py err=ValueError() $ python310 async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 39

Slide 39 text

$ python310 1_asyncio_gather_except.py err=ValueError() $ python310 async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions ValueError only TypeError cannot be caught

Slide 40

Slide 40 text

$ python310 1_asyncio_gather_except.py err=ValueError() $ python310 ValueError only TypeError cannot be caught Only the fi rst exception is captured The rest are suppressed. async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}") asyncio.run(main()) asyncio.gather - return_exceptions False Catching Concurrent Exceptions

Slide 41

Slide 41 text

•asyncio.gather - return_exceptions False •asyncio.gather - return_exceptions True return_exceptions If speci fi ed Get a list of results after all tasks have been executed Need to check the return list.

Slide 42

Slide 42 text

async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 43

Slide 43 text

async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 44

Slide 44 text

async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 45

Slide 45 text

$ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 46

Slide 46 text

$ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") results=['੒ޭ', ValueError(), TypeError()] $ python310 asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 47

Slide 47 text

$ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") results=['੒ޭ', ValueError(), TypeError()] $ python310 Get all results in a list Need to see what exceptions were raised asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 48

Slide 48 text

for result in results: match result: case ValueError(): print("ValueError") case TypeError(): print("TypeError") async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 49

Slide 49 text

for result in results: match result: case ValueError(): print("ValueError") case TypeError(): print("TypeError") async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 50

Slide 50 text

$ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") for result in results: match result: case ValueError(): print("ValueError") case TypeError(): print("TypeError") asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 51

Slide 51 text

$ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") results=['੒ޭ', ValueError(), TypeError()] ValueError() for result in results: match result: case ValueError(): print("ValueError") case TypeError(): print("TypeError") TypeError() asyncio.gather - return_exceptions True Catching Concurrent Exceptions

Slide 52

Slide 52 text

asyncio.gather - return_exceptions True $ python310 2_asyncio_gather_except_true.py async def main(): """return_exceptions=True͋Γ""" results = await asyncio.gather( coro_success(), coro_value_err(), coro_type_err(), return_exceptions=True) print(f"{results=}") results=['੒ޭ', ValueError(), TypeError()] ValueError() for result in results: match result: case ValueError(): print("ValueError") case TypeError(): print("TypeError") TypeError() Need to check results All tasks need to be performed Catching Concurrent Exceptions

Slide 53

Slide 53 text

•For example, a process that makes 10 HTTP requests at the same time •Exception occurred in the fi rst task •We'll have to wait until the remaining nine that might fail are complete. (even if all 10 result in the same error) Catching Concurrent Exceptions Python 3.10 and earlier issues •Unable to capture except for the fi rst exception. •Cannot log all exceptions that occur •Need to check the return list. •All tasks must be performed •return_exceptions False •return_exceptions True

Slide 54

Slide 54 text

Catching Concurrent Exceptions Python 3.10 and earlier issues •Unable to capture except for the fi rst exception. •Cannot log all exceptions that occur •Need to check the return list. •All tasks must be performed •return_exceptions False •return_exceptions True •The solution is asyncio.TaskGroup and the new "except*" syntax

Slide 55

Slide 55 text

•Prepare asynchronous functions to run concurrently - 3 coroutines Advance preparation - no change import asyncio async def coro_success(): return "੒ޭ" async def coro_value_err(): raise ValueError async def coro_type_err(): raise TypeError

Slide 56

Slide 56 text

async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] Catching Exceptions in asycio.TaskGroup

Slide 57

Slide 57 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] Catching Exceptions in asycio.TaskGroup

Slide 58

Slide 58 text

async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") try: Catching Exceptions in asycio.TaskGroup

Slide 59

Slide 59 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Catching Exceptions in asycio.TaskGroup

Slide 60

Slide 60 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Can be captured by except as a synchronous process Catching Exceptions in asycio.TaskGroup

Slide 61

Slide 61 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Catching Exceptions in asycio.TaskGroup

Slide 62

Slide 62 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") $ python311 3_asyncio_taskgroup_except.py Catching Exceptions in asycio.TaskGroup

Slide 63

Slide 63 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") $ python311 3_asyncio_taskgroup_except.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) err=ExceptionGroup('unhandled errors in a TaskGroup', [TypeError()]) $ python311 Catching Exceptions in asycio.TaskGroup

Slide 64

Slide 64 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") $ python311 3_asyncio_taskgroup_except.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) err=ExceptionGroup('unhandled errors in a TaskGroup', [TypeError()]) $ python311 Two Exception are captured. Catching Exceptions in asycio.TaskGroup

Slide 65

Slide 65 text

asyncio.TaskGroup new syntaxʮexcept*ʯ •Can catch concurrent exceptions •Easy to read, written in the same way as synchronous processing This is the fi rst feature of asyncio.TaskGroup

Slide 66

Slide 66 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Catching Exceptions in asycio.TaskGroup

Slide 67

Slide 67 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Catching Exceptions in asycio.TaskGroup

Slide 68

Slide 68 text

try: async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(coro_success()) task2 = tg.create_task(coro_value_err()) task3 = tg.create_task(coro_type_err()) results = [task1.result(), task2.result(), task3.result()] except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") Catching Exceptions in asycio.TaskGroup

Slide 69

Slide 69 text

except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") except* except* Catching Exceptions in asycio.TaskGroup

Slide 70

Slide 70 text

Catching Exceptions in asycio.TaskGroup except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f"{err=}") except* except* err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) err=ExceptionGroup('unhandled errors in a TaskGroup', [TypeError()])

Slide 71

Slide 71 text

except* except* •PEP 654: Exception Groups and except*. •Python 3.11new syntax •Use with ExceptionGroup (added 3.11) for built-in exception types •ExceptionGroup •Use when you need to raise multiple unrelated exceptions •try except •try except* •ExceptionGroup(“message”, [ValueError(1), TypeError(2)]) new syntax

Slide 72

Slide 72 text

asycio.TaskGroup IUUQTHJUIVCDPNQZUIPODQZUIPOCMPCNBJO-JCBTZODJPUBTLHSPVQTQZ class TaskGroup: ... async def __aexit__(self, et, exc, tb): ... if self._errors: # Exceptions are heavy objects that can have object # cycles (bad for GC); let's not keep a reference to # a bunch of them. errors = self._errors self._errors = None me = BaseExceptionGroup('unhandled errors in a TaskGroup', errors) raise me from None ExceptionGroup

Slide 73

Slide 73 text

class TaskGroup: ... async def __aexit__(self, et, exc, tb): ... if self._errors: # Exceptions are heavy objects that can have object # cycles (bad for GC); let's not keep a reference to # a bunch of them. errors = self._errors self._errors = None me = BaseExceptionGroup('unhandled errors in a TaskGroup', errors) raise me from None IUUQTHJUIVCDPNQZUIPODQZUIPOCMPCNBJO-JCBTZODJPUBTLHSPVQTQZ asycio.TaskGroup ExceptionGroup

Slide 74

Slide 74 text

IUUQTHJUIVCDPNQZUIPODQZUIPOCMPCNBJO-JCBTZODJPUBTLHSPVQTQZ class TaskGroup: ... async def __aexit__(self, et, exc, tb): ... if self._errors: # Exceptions are heavy objects that can have object # cycles (bad for GC); let's not keep a reference to # a bunch of them. errors = self._errors self._errors = None me = BaseExceptionGroup('unhandled errors in a TaskGroup', errors) raise me from None asycio.TaskGroup ExceptionGroup

Slide 75

Slide 75 text

new syntax except* except* •PEP 654: Exception Groups and except*. •PEP654 Motivation 4 cases •Concurrent errors Born for what? •There is no good way to handle exceptions sent by multiple tasks. •asyncio.gather()

Slide 76

Slide 76 text

•except* syntax is implemented for asynchronous processing ? •Python 3.11ͷ৽ػೳ(ͦͷ4ʣPEP 654 ྫ֎άϧʔϓͱTaskGroup •https://www.python.jp/news/wnpython311/except-grp.html •The 2021 Python Language Summit: PEP 654 — Exception Groups and except* •https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-pep-654.html • How Exception Groups Will Improve Error Handling in AsyncIO - Łukasz Langa | Power IT Conference •https://www.youtube.com/watch?v=Lfe2zsGS0Js new syntax except* Born for what?

Slide 77

Slide 77 text

•Catching Concurrent Exceptions •Cancel task when exception occurs •How do you cancel a task? •Python 3.10 or earlier - asyncio.gather() •Python 3.11 - asyncio.TaskGroup() Features of asyncio.TaskGroup

Slide 78

Slide 78 text

Cancel task when exception occurs •If an exception occurs, what happens to the remaining tasks that were running at the same time? •In some cases, the remaining tasks are executed as they are. •Let's take asyncio.gather() as an example of a case that remains

Slide 79

Slide 79 text

import asyncio async def coro_success(): return "੒ޭ" async def coro_value_err(): raise ValueError async def coro_type_err(): raise TypeError มߋͳ͠ มߋͳ͠ •Add one asynchronous function - wait a bit Preparation - add one asynchronous function

Slide 80

Slide 80 text

•Add one asynchronous function - wait a bit Preparation - add one asynchronous function import asyncio async def coro_success(): return "੒ޭ" async def coro_value_err(): raise ValueError async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" มߋͳ͠ มߋͳ͠ ௥Ճ

Slide 81

Slide 81 text

Cases where tasks remain - asyncio.gather - return_exceptions false async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main())

Slide 82

Slide 82 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 83

Slide 83 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 84

Slide 84 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 85

Slide 85 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 86

Slide 86 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 87

Slide 87 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 88

Slide 88 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ If an exception occurs Other tasks remain running Hard to predict behavior Cases where tasks remain - asyncio.gather - return_exceptions false

Slide 89

Slide 89 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ I want to stop. I want to roll back Cases where tasks remain - asyncio.gather - return_exceptions false If an exception occurs Other tasks remain running Hard to predict behavior

Slide 90

Slide 90 text

λεΫ͕࢒ͬͯ͠·͏έʔε - asyncio.gather - return_exceptions ࢦఆͳ͠ async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ ɾࢭΊ͍ͨ ɾϩʔϧόοΫ͍ͨ͠ ྫ֎͕ൃੜͨ͠৔߹ ଞͷλεΫ͸ಈ͍ͨ·· 
 ಈ࡞Λ༧ଌͮ͠Β͍ I want to stop. I want to roll back If an exception occurs Other tasks remain running Hard to predict behavior

Slide 91

Slide 91 text

Cancel task when exception occurs •If an exception occurs, what happens to the remaining tasks that were running at the same time? •In some cases, the remaining tasks are executed as they are. •Cancel a task (Cancellations were not added on 3.11) •Cancellation Basics •Cancel when using asyncio.gather() before 3.10 •Let's fi x the process that leaves the task

Slide 92

Slide 92 text

Cancel task when exception occurs •Let's fi x the process that leaves the task •Asynchronous function (coro_long()) to be canceled •Instructions to check and cancel remaining tasks •Wait for post-cancellation processing to complete. Added post-processing for cancellations •There are three perspectives for modi fi cation

Slide 93

Slide 93 text

•task.cancel() •Cancel request to a task •CancelledError exception is sent to the cancelled task •receive asyncio.CancelledError in an asynchronous function •Resend asyncio.CancelledError •I'm a little confused. Let's try to implement it. Added post-processing for cancellations

Slide 94

Slide 94 text

import asyncio async def coro_success(): return "੒ޭ" async def coro_value_err(): raise ValueError async def coro_long(): มߋͳ͠ มߋͳ͠ มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 95

Slide 95 text

async def coro_long(): มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 96

Slide 96 text

async def coro_long(): มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" try: •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 97

Slide 97 text

async def coro_long(): มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 98

Slide 98 text

async def coro_long(): มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 99

Slide 99 text

Cancel task when exception occurs •Let's fi x the process that leaves the task •Asynchronous function (coro_long()) to be canceled •Instructions to check and cancel remaining tasks •Wait for post-cancellation processing to complete. Added post-processing for cancellations •There are three perspectives for modi fi cation

Slide 100

Slide 100 text

Cancel task when exception occurs •Let's fi x the process that leaves the task •Asynchronous function (coro_long()) to be canceled •Instructions to check and cancel remaining tasks •Wait for post-cancellation processing to complete. Added post-processing for cancellations •There are three perspectives for modi fi cation

Slide 101

Slide 101 text

•Request a task to cancel - Task.cancel() Instructions to check and cancel remaining tasks •Determine completion of a task - Task.done() •Wait for post-cancel process to complete - asyncio.sleep() •Let's modify the code.

Slide 102

Slide 102 text

async def main(): """return_exceptionsͳ͠""" try: print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) asyncio.run(main()) print("ऴྃ") Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 103

Slide 103 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 104

Slide 104 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) λεΫͷঢ়ଶΛ ֬ೝ͠΍͘͢͢Δ ॲཧͱͯ͠͸ಉ͡ print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 105

Slide 105 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") for task in [task1, task2, task3]: if task.done() is False: Task.done()Ͱ ະ׬ྃͷλεΫΛಛఆ Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 106

Slide 106 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ ະ׬ྃͷλεΫʹର͠ Task.cancel()Λ࣮ߦ print("ऴྃ") Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 107

Slide 107 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ ׬ྃ଴ͪͷ asyncio.sleep() print("ऴྃ") await asyncio.sleep(1) Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 108

Slide 108 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ શମ͜Μͳײ͡ print("ऴྃ") await asyncio.sleep(1) Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 109

Slide 109 text

Review a bit of the output in case a task remains

Slide 110

Slide 110 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ Cases where tasks remain - asyncio.gather - return_exceptions false Review Execution before modi fi cation

Slide 111

Slide 111 text

async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ Cases where tasks remain - asyncio.gather - return_exceptions false Review Execution before modi fi cation Output from remaining tasks

Slide 112

Slide 112 text

Cases where tasks remain - asyncio.gather - return_exceptions false async def main(): """return_exceptionsͳ͠""" try: results = await asyncio.gather( coro_success(), coro_value_err(), coro_long() ) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) print("ऴྃ") asyncio.run(main()) $ python310 4_remaining_tasks.py err=ValueError() ऴྃ ׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢ Review Execution before modi fi cation Output from remaining tasks

Slide 113

Slide 113 text

async def coro_long(): มߋ await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError •Therefore, it is necessary to receive asyncio.CancelledError in an asynchronous function •In addition, resend asyncio.CancelledError in an asynchronous function Advance preparation - add post-processing in case of cancellation

Slide 114

Slide 114 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) ࣮ߦͯ͠Έ·͠ΐ͏ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 115

Slide 115 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) $ python310 5_canceling_task_gather.py async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 116

Slide 116 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) $ python310 5_canceling_task_gather.py err=ValueError() async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 117

Slide 117 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) $ python310 5_canceling_task_gather.py err=ValueError() Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 118

Slide 118 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) $ python310 5_canceling_task_gather.py err=ValueError() Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError ऴྃ Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 119

Slide 119 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) Ωϟϯηϧ͞ΕΔͱ ඇಉظؔ਺ଆͰ ΩϟϯηϧΛัଊ $ python310 5_canceling_task_gather.py err=ValueError() Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError ऴྃ Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 120

Slide 120 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) If canceled On the asynchronous function side Catch Cancel $ python310 5_canceling_task_gather.py err=ValueError() Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError ऴྃ basic principle of cancellation Cases where tasks remain → Add instructions to con fi rm and cancel tasks Python 3.10 or earlier - asyncio.gather()

Slide 121

Slide 121 text

Cases where tasks remain → Add instructions to con fi rm and cancel tasks async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) asyncio.run(main()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) Python 3.10 or earlier - asyncio.gather() $ python310 5_canceling_task_gather.py err=ValueError() Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError ऴྃ Checking task status Waiting for cancellation process

Slide 122

Slide 122 text

Cancel task when exception occurs •Let's fi x the process that leaves the task •Asynchronous function (coro_long()) to be canceled •Instructions to check and cancel remaining tasks •Wait for post-cancellation processing to complete. Added post-processing for cancellations •There are three perspectives for modi fi cation

Slide 123

Slide 123 text

•At the time of exiting the context manager, •Remaining tasks are canceled. •Wait for cancellation process to complete. •That's all! •In short, nothing special needs to be done. It will be canceled. Let's refactor the code we just wrote. Cancellation process in asycio.TaskGroup

Slide 124

Slide 124 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) asyncio.run(main()) print(f"{results=}") for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) Cancellation process in asycio.TaskGroup Let's refactor it. results = await asyncio.gather(*[task1, task2, task3]) except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”)

Slide 125

Slide 125 text

async def main(): """return_exceptionsͳ͠""" try: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) results = await asyncio.gather(*[task1, task2, task3]) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 126

Slide 126 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) results = await asyncio.gather(*[task1, task2, task3]) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 127

Slide 127 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: task1 = asyncio.create_task(coro_success()) task2 = asyncio.create_task(coro_value_err()) task3 = asyncio.create_task(coro_long()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 128

Slide 128 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 129

Slide 129 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 130

Slide 130 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: print(f"{results=}") except ValueError as err: print(f"{err=}") except TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 131

Slide 131 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 132

Slide 132 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) for task in [task1, task2, task3]: if task.done() is False: task.cancel() # ະ׬ྃͷλεΫΛΩϟϯηϧ print("ऴྃ") await asyncio.sleep(1) async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 133

Slide 133 text

async def main(): """return_exceptionsͳ͠""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 134

Slide 134 text

async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup Let's refactor it.

Slide 135

Slide 135 text

Let's run it. async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) $ python311 6_cancelled_task_taskgroup.py Cancellation process in asycio.TaskGroup

Slide 136

Slide 136 text

Let's run it. async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) $ python311 6_cancelled_task_taskgroup.py Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ Cancellation process in asycio.TaskGroup

Slide 137

Slide 137 text

Let's run it. async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) $ python311 6_cancelled_task_taskgroup.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ Cancellation process in asycio.TaskGroup

Slide 138

Slide 138 text

Let's run it. async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) $ python311 6_cancelled_task_taskgroup.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ ऴྃ Cancellation process in asycio.TaskGroup

Slide 139

Slide 139 text

Catching Exceptions and Processing after canceling The order of catching async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) $ python311 6_cancelled_task_taskgroup.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ ऴྃ Cancellation process in asycio.TaskGroup

Slide 140

Slide 140 text

In TaskGroup complete Predictable behavior async def coro_long(): await asyncio.sleep(1) print("׬͍ྃͯ͠ͳ͍λεΫ͕ग़ྗ͍ͯ͠·͢") return "੒ޭʁ" except asyncio.CancelledError as err: try: print("Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢") raise asyncio.CancelledError async def main(): """TaskGroupͰͷΩϟϯηϧ""" try: asyncio.run(main()) print("ऴྃ") async with asyncio.TaskGroup() as g: print(f"{results=}") except* ValueError as err: print(f"{err=}") except* TypeError as err: print(f”{err=}”) results = [task1.result(), task2.result(), task3.result()] task1 = g.create_task(coro_success()) task2 = g.create_task(coro_value_err()) task3 = g.create_task(coro_long()) Cancellation process in asycio.TaskGroup $ python311 6_cancelled_task_taskgroup.py err=ExceptionGroup('unhandled errors in a TaskGroup', [ValueError()]) Ωϟϯηϧ͞ΕͨλεΫ͕ग़ྗ͍ͯ͠·͢ ऴྃ

Slide 141

Slide 141 text

asycio.TaskGroup Features •Catching Concurrent Exceptions •Cancel task when exception occurs •Simply catch exceptions in a conventional way of writing •Cancel without indication •Except* will not spare any concurrent exceptions. •No longer need to be aware of tasks when executing them •How to write "predictable," "safe," and readable asyncio •New “Hello-ish world”

Slide 142

Slide 142 text

import asyncio async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(some_coro(...)) task2 = tg.create_task(another_coro(...)) New “Hello-ish world”

Slide 143

Slide 143 text

import asyncio async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(some_coro(...)) task2 = tg.create_task(another_coro(...)) try: except* BaseException as e: print(f"{e=}") asyncio.run(main()) New “Hello-ish world”

Slide 144

Slide 144 text

New “Hello-ish world” import asyncio async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task(some_coro(...)) task2 = tg.create_task(another_coro(...)) try: except* BaseException as e: print(f"{e=}") asyncio.run(main()) Python 3.11

Slide 145

Slide 145 text

•Quickstart - Example 3-2. The “Hello-ish World” of Asyncio import asyncio import time async def main(): print(f"{time.ctime()} Hello!") await asyncio.sleep(1.0) print(f"{time.ctime()} GoodBye!") loop = asyncio.get_event_loop() task = loop.create_task(main()) loop.run_until_complete(task) pending = asyncio.all_tasks(loop=loop) for task in pending: task.cancel() group = asyncio.gather(*pending, return_exceptions=True) loop.run_until_complete(group) loop.close()

Slide 146

Slide 146 text

•Quickstart - Example 3-2. import asyncio import time async def main(): print(f"{time.ctime()} Hello!") await asyncio.sleep(1.0) print(f"{time.ctime()} GoodBye!") The “Hello-ish World” of Asyncio Checking task status Waiting for cancellation process loop = asyncio.get_event_loop() task = loop.create_task(main()) loop.run_until_complete(task) pending = asyncio.all_tasks(loop=loop) for task in pending: task.cancel() group = asyncio.gather(*pending, return_exceptions=True) loop.run_until_complete(group) loop.close()

Slide 147

Slide 147 text

•Quickstart - Example 3-2. import asyncio import time async def coro(): print(f"{time.ctime()} Hello!") await asyncio.sleep(1.0) print(f"{time.ctime()} GoodBye!") The “Hello-ish World” of Asyncio Checking task status Waiting for cancellation process async def main(): async with asyncio.TaskGroup() as tg: task = tg.create_task(coro(...)) try: except* BaseException as e: print(f"{e=}") asyncio.run(main())

Slide 148

Slide 148 text

•Catching and canceling exceptions in asyncio.TaskGroup •asyncio for external I/O •Because of external dependence, exceptions are likely to occur. •Easy to write and read •Let's touch TaskGroup. •Exception/cancellation triggers a review of the existing code. •A more comfortable asyncio life

Slide 149

Slide 149 text

•Task Status Appendix: Task Status •PENDING •FINISHED •CANCELLED •You can check with task._state •The sample code also outputs the state. Please try to run it. Here is the code IUUQTHJUIVCDPNKSGLUBML

Slide 150

Slide 150 text

Appendix: HTTPX with TaskGroup Here is the code IUUQTHJUIVCDPNKSGLUBML import asyncio from time import time import httpx URL = "https://pokeapi.co/api/v2/pokemon/" async def access_url_poke(client, num: int) -> str: r = await client.get(f"{URL}{num}") pokemon = r.json() # JSONύʔε return pokemon["name"] # ϙέ໊Λൈ͘ async def main_poke(): """httpxͰϙέϞϯ151ඖҾͬ͜ൈ͘""" start = time() async with httpx.AsyncClient() as client: try: async with asyncio.TaskGroup() as tg: tasks = [ tg.create_task(access_url_poke(client, number)) for number in range(1, 151) ] except* BaseException as e: print(f"{e=}") results = [task.result() for task in tasks] print(results) print("time: ", time() - start) asyncio.run(main_poke())

Slide 151

Slide 151 text

Appendix: New features that are still available! Here is the code IUUQTHJUIVCDPNKSGLUBML import asyncio async def main(): try: async with asyncio.TaskGroup() as tg: task1 = tg.create_task(some_coro(...)) task2 = tg.create_task(another_coro(...)) except* BaseException as e: print(f"{e=}") with asyncio.Runner() as runner: runner.run(main()) 3.11 What's New What is asyncio.Runner! For interactive sessions? run multiple times.

Slide 152

Slide 152 text

Appendix: it was a scheduled draft 🙏. •asyncio.timeout() •Sneakily Added Python 3.11 New Features •Born of a wait_for bugs? •Implement asyncio.timeout() context manager #90927 IUUQTHJUIVCDPNQZUIPODQZUIPOJTTVFT …

Slide 153

Slide 153 text

Appendix: it was a scheduled draft 🙏. •asyncio evolution history •Asynchronous function asyncore, which has actually been around since Python 1.5 •Third-party asynchronous implementation •curio, trio , AnyIO, quattro, etc. … …

Slide 154

Slide 154 text

thanks •Using Asyncio in Python - Oreilly & Associates Inc •Python 3.11 Preview: Task and Exception Groups - realpython.com •https://realpython.com/python311-exception-groups/ •Python 3.11ͷ৽ػೳ(ͦͷ4ʣPEP 654 ྫ֎άϧʔϓͱTaskGroup •https://www.python.jp/news/wnpython311/except-grp.html •The 2021 Python Language Summit: PEP 654 — Exception Groups and except* •https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-pep-654.html • How Exception Groups Will Improve Error Handling in AsyncIO - Łukasz Langa | Power IT Conference •https://www.youtube.com/watch?v=Lfe2zsGS0Js

Slide 155

Slide 155 text

Thank you for your attention 👋