Python and Asynchrony - Speaker Deck

Slide 1

Slide 1 text

Python and Asynchrony

Slide 2

Slide 2 text

http://bit.ly/asyncio-examples

Slide 3

Slide 3 text

Quick Questions • Concurrency with Python • Threads and multi-processing • Single-thread asynchrony • asyncio, Twisted, Tornado, etc.

Slide 4

Slide 4 text

Me • Call me TP • Follow @uranusjr • https://uranusjr.com

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

http://macdown.uranusjr.com

Slide 8

Slide 8 text

www. .com

Slide 9

Slide 9 text

10–11 June 2017 (Ծ) 7

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Agenda • Introduction to asynchrony • Async the asyncio way • The road ahead

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Waiting (wasted) Synchronous

Slide 14

Slide 14 text

Threads

Slide 15

Slide 15 text

Threads • More than you need • Race conditions • Dead-locking • Resource starvation

Slide 16

Slide 16 text

To share, or not to share: that is the question.

Slide 17

Slide 17 text

LET’S TALK ABOUT SOMETHING “DIFFERENT”

Slide 18

Slide 18 text

Context Switching • First as a CPU thing • Interrupts • Software interrupts

Slide 19

Slide 19 text

Multitasking • A common OS problem • More tasks than threads • Cooperative • Pre-emptive

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Same problem?

Slide 22

Slide 22 text

Threads

Slide 23

Slide 23 text

Scheduling

Slide 24

Slide 24 text

Asynchronous

Slide 25

Slide 25 text

Sync vs. Async

Slide 26

Slide 26 text

ASYNCIO

Slide 27

Slide 27 text

Async How? • Callbacks • Promises • Coroutines

Slide 28

Slide 28 text

Event Loop

Slide 29

Slide 29 text

Launch Initialise Running Cleanup Terminate Register callbacks Receive events

Slide 30

Slide 30 text

import asyncio # Init code goes here. loop = asyncio.get_event_loop() loop.run_forever()

Slide 31

Slide 31 text

import asyncio # Init code goes here. def print_and_exit(): print('Hello asyncio!') asyncio.get_event_loop().stop() loop = asyncio.get_event_loop() loop.call_soon(print_and_exit) loop.run_forever()

Slide 32

Slide 32 text

CALLBACK When the going gets rough.

Slide 33

Slide 33 text

Pyramid of Doom $(function () { $("button").click(function (e) { $.get(URL, function (data) { $(".list").each(function () { $(this).click(function () { setTimeout(function () { alert("Top of the world"); }, 1000); }); }); }); }); });

Slide 34

Slide 34 text

WELCOME TO CALLBACK HELL

Slide 35

Slide 35 text

def send_repos_request(): ... task = ... task.add_done_callback(...) def read_repos(future): ... = future.result() ... task = ... task.add_done_callback(...) def send_forked_repo_requests(future): ... = future.result() ... task = ... task.add_done_callback(...) def read_forked_repos(future): ... = future.result() ... task = ... task.add_done_callback(...)

Slide 36

Slide 36 text

Promise

Slide 37

Slide 37 text

def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 38

Slide 38 text

def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 39

Slide 39 text

def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 40

Slide 40 text

def main(): (send_repos_request() .then(read_repos) .then(send_forked_repo_requests, cleanup) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 41

Slide 41 text

def main(): (send_repos_request() .then(read_repos) .catch(cleanup) .then(send_forked_repo_requests) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 42

Slide 42 text

def main(): (send_repos_request() .then(read_repos) .catch(cleanup) .then(send_forked_repo_requests) .then(read_forked_repo_requests) .then(print_source_names) .done(cleanup))

Slide 43

Slide 43 text

???????

Slide 44

Slide 44 text

Coroutine

Slide 45

Slide 45 text

async def run(session): response = await send_repos_request(session) try: data = await read_repos(response) except APIError: return responses = await send_forked_repo_requests(session, data) datasets = await read_forked_repo_requests(responses) print_source_names(datasets)

Slide 46

Slide 46 text

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

Everything Has a Counterpart • async def / await • async with • async for (Python 3.6)

Slide 49

Slide 49 text

class AsyncContextManager: async def __aenter__(self): await log('entering context') async def __aexit__(self, exc_type, exc, tb): await log('exiting context')

Slide 50

Slide 50 text

class AsyncIterable: def __aiter__(self): return self async def __anext__(self): data = await ... if data: return data else: raise StopAsyncIteration

Slide 51

Slide 51 text

Python asyncio.

Slide 52

Slide 52 text

Wrapping Up • What is asynchrony • How asynchrony works • Asynchrony and asyncio

Slide 53

Slide 53 text

Further Watching • A gentle introduction to asyncio • What in the World is Asyncio? • Topics of Interest (Python Asyncio) • You Might Not Want Async • and more…!