如何⾃幹⼀個 Python Coroutine kaohsiung.py 20180613 

No content

故事是這樣的 ಭᑤ PyConTW 牫 Ꮈ绗 Channels 牫牫 覍ݶྍ 牫牫牫 Coroutine 牫牫牫牫 樄ত፡ԧӞञ୽粙牧簁盅磪讨睞眖牦 

–David Beazley 「如果 Python 的書籍是⼀種指南，協同程序是最沒⼈寫 到、晦澀，且顯然無⽤的 Python 功能。」 

Asynchronous • ᴥलୗग़ᤈ纷 (ྯ㮆藶穩樄Ӟ㮆 process, ֕᩼㬵᩼ग़ cpu context-switch 疰᩼ṛ) • ᴥलୗԪग़ᤈ纷ग़䁆ᤈ姼 (樄ग़㮆ᤈ纷膏䁆ᤈ姼牧蝨౮ dead lock, race condition, 疪ٌ蒂ቘوݶ䩚ᥜ碻蝡犚㺔氂疰䨝ๅ瑥᯿牧犖犋অ䌃) • 覍ᴥलୗԪկ詴㵕 (㻌Ӟ蝅瑹䲒礚磪篷翕᪠ IO Ԫկ咳ኞ牧፜݄ԧ context-switch, process copy 缛牧砰䌃蕦褾牧ܻࢩฎӞկԪ眐ᤩ藉咳牧磪Ԫ眐䷱螭蒂ቘ疰ض懿 袅牧ӥ稞藉咳ٚ㬵礬硁ض獮制眲㬵蒂ቘ) ex: Twisted • 覍ᴥलୗ Coroutine (猟ฎᴥलୗӞ䰬ፗ薪牧䟖㻌Ӟ蝅瑹䲒礚Ԫկ牧֕ےӤ Coroutine) ex: gevent, asyncio 

No content

– Unyielding Glyph, 2014 Problem:「Threads Are Bad」 

Asynchronous • ᴥलୗग़ᤈ纷 (ྯ㮆藶穩樄Ӟ㮆 process, ֕᩼㬵᩼ग़ cpu context-switch 疰᩼ṛ) • ᴥलୗԪग़ᤈ纷ग़䁆ᤈ姼 (樄ग़㮆ᤈ纷膏䁆ᤈ姼牧蝨౮ dead lock, race condition, 疪ٌ蒂ቘوݶ䩚ᥜ碻蝡犚㺔氂疰䨝ๅ瑥᯿牧犖犋অ䌃) • 覍ᴥलୗԪկ詴㵕 (㻌Ӟ蝅瑹䲒礚磪篷翕᪠ IO Ԫկ咳ኞ牧፜݄ԧ context-switch, process copy 缛牧砰䌃蕦褾牧ܻࢩฎӞկԪ眐ᤩ藉咳牧磪Ԫ眐䷱螭蒂ቘ疰ض懿 袅牧ӥ稞藉咳ٚ㬵礬硁ض獮制眲㬵蒂ቘ) ex: Twisted • 覍ᴥलୗ Coroutine (猟ฎᴥलୗӞ䰬ፗ薪牧䟖㻌Ӟ蝅瑹䲒礚Ԫկ牧֕ےӤ Coroutine) ex: gevent, asyncio 

No content

⽼闆，來點例⼦說明吧 • https://github.com/chairco/kaohsiung.py_talk/tree/master/ 20180613_python_coroutine_work • https://blog.chairco.me/posts/2018/06/how-python- coroutines-work.html 

# sync def get(path): s = socket.socket() s.connect(('localhost', 5000)) request = 'GET %s HTTP/1.0\r\n\r\n' % path s.send(request.encode()) chunks = [] while True: chunk = s.recv(1000) if chunk: chunks.append(chunk) else: body = (b''.join(chunks)).decode() print(body.split('\n')[0]) return 

Multithreading with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor: future_to_url = {executor.submit(get, url): url for url in URLS} for future in concurrent.futures.as_completed(future_to_url): url = future_to_url[future] try: data = future.result() except Exception as exc: print('%r generated an exception: %s' % (url, exc)) print('multithreading took %.1f sec' % (time.time() - start)) 

Non-blocking • Event-driven • Socket Multiplexing • UNIX: select, poll, epoll. Windows, Solaris: IOCP. BSD/OSX: kqueue 

from selectors import DefaultSelector, EVENT_WRITE, EVENT_READ selector = DefaultSelector() def get_non_blocking(path): s = socket.socket() s.setblocking(False) try: s.connect(('localhost', 5000)) except BlockingIOError: pass Non-blocking 

No content

Callback Future/Promise yield, yield from async/await 

def get_callback(path): ... callback = lambda: connected(s, path) ... callback() def connected(s, path): ... callback = lambda: readable(s, chunks) ... callback() def readable(s, chunks): ... if chunk: ... callback = lambda: readable(s, chunks) ... callback() else: ... return 

Callback Hell, Pyramid of Doom 

Event-loop Javascript’s Event-loop sample ֦狶অ盅ݞ౯ (by TP) A 咳ኞ, 䁆ᤈ B https://pjchender.blogspot.com/2017/08/javascript-learn-event-loop-stack-queue.html 

n_jobs = 0 def get_eventloop(path): global n_jobs n_jobs += 1 ... callback = lambda: connected_event(s, path) # closure selector.register(s.fileno(), EVENT_WRITE, data=callback) def connected_event(s, path): ... callback = lambda: readable_event(s, chunks) def readable_event(s, chunks): global n_jobs ... if chunk: ... else: n_jobs -= 1 

get_eventloop('/foo') get_eventloop('/bar') while n_jobs: events = selector.select() for key, mask in events: cb = key.data cb() 

No content

Coroutine single thread ӥر战纷ୗ㬵究ਧ纷ୗ䁆ᤈጱ殼ଧ牧ᘒ磪硳螈౮覍ݶྍ I/O ጱӞ圵ොဩ Future, Generator, Task 

selector = DefaultSelector() c_n_jobs = 0 class Future: def __init__(self): self.callbacks = None def resolve(self): self.callbacks() def __await__(self): yield self 

class Task: def __init__(self, coro): self.coro = coro self.step() def step(self): try: f = self.coro.send(None) except StopIteration: return f.callbacks = self.step 

async def get_coroutines(path): global n_jobs n_jobs += 1 ... f = Future() selector.register(s.fileno(), EVENT_WRITE, data=f) await f selector.unregister(s.fileno()) request = 'GET %s HTTP/1.0\r\n\r\n' % path s.send(request.encode()) chunks = [] while True: f = Future() selector.register(s.fileno(), EVENT_READ, data=f) await f selector.unregister(s.fileno()) chunk = s.recv(1000) if chunk: chunks.append(chunk) else: break body = (b''.join(chunks)).decode() print(body.split('\n')[0]) n_jobs -= 1 

Task(get_coroutines('/foo')) Task(get_coroutines('/bar')) while n_jobs: events = selector.select() for key, mask in events: fut = key.data fut.resolve() 

No content

No content

No content