Share
Embed
Asynchronous I/O in Python 3
Asynchronous I/O in Python 3
http://www.slideshare.net/megafeihong/tulip-24190096
Zoom.Quiet
July 22, 2013
Transcript
44 slides
-
Asychronous I/O in Python 3 Feihong Hsu Chicago Python User
Group July 11, 2013 -
What is PEP 3156? • "Asynchronous IO Support Rebooted" •
Describes a new event loop API • Aims to be compatible with current popular event loops (Twisted, Tornado, etc.) • Should land in the standard library in 3.4 (early 2014) -
The big take-away We're going to be able to write
asynchronous code without threads or callbacks! -
Threads vs event loop • I'm not going to talk
about this • Twisted being around for more than a decade kinda validates the event loop model • For a detailed discussion see An Intro to Asynchronous Programming and Twisted -
Twisted vs Tulip Ugh, how to explain this? I could
just give you a boring list of differences... Or I could use an analogy... -
Mystical Ninja Warrior • Can defeat demons and demigods by
deploying an army of shadow clones • Mental gymnastics required to coordinate shadow clones requires years of brutal training • Ninja magic comes with a terrible price: the more clones you make, the more insane you become -
Playing Plants Vs Zombies • Can defeat a zombie horde
by deploying and managing a large number of horticultural specimens • Not particularly mentally taxing • Somehow end up playing for hours without eating or sleeping -
Alrighty With the analogies out of the way, let's look
at some examples. -
Control flow diagram I was going to draw one but
then I got sleepy. -
Problems with callback- based asynch code • Can be hard
to understand control flow • Error messages are not friendly • Debugging is harder -
Fetching a web page in Twisted w/ inline callbacks https://github.com/feihong/tulip-talk/blob/
master/examples/4-twisted-download-inline- callbacks.py -
The Tulip API • Basics are documented in PEP 3156
• Not available in Python Package Index • Includes sockets, file I/O, etc. • Includes concurrency data structures like locks, queues, semaphores • Includes SSL and HTTP (with support for websockets!) -
Coroutines vs tasks When I first started, the most confusing
thing about Tulip. Coroutines are executed by "yield from". Tasks are not executed by "yield from". -
Coroutine • Basically a function that contains at least one
"yield from" statement. • Not the same thing as a generator function, which is a function that contains a yield expression. • Tulip will barf if you try to make it execute a generator. -
Coroutine @tulip.coroutine def download(url): response = yield from tulip.http.request('GET', url)
return yield from response.read() Calling "download()" returns a generator object, but otherwise does nothing! You need to do "yield from download()" to run the body of a coroutine. -
Task • An object that manages a coroutine • Roughly
equivalent to the Deferred object in Twisted -
Task @tulip.task def download(url): response = yield from tulip.http.request('GET', url)
return yield from response.read() Calling "download()" actually does run the body of the function. The "yield from" part is done implicitly for you. -
Task In Tulip, there are two ways of creating tasks:
tulip.task is a decorator that produces a task-wrapping function tulip.Task is a constructor that accepts a coroutine -
Why bother to use tasks? • To interoperate with callback-based
frameworks like Twisted • To cancel an already-running coroutine • To start a new coroutine from within another coroutine -
Wait, so what's a Future? PEP 3156 makes frequent mention
of Futures. But talking about Futures is a little confusing, since there are two Future classes: tulip.Future and concurrent.futures.Future. -
Future is the superclass of Task • Future don't necessarily
manage a coroutine • In practice you never create Future objects, only Task objects • Futures are acceptable to yield from expressions -
Methods on Future • cancel(), cancelled() • running(), done() •
result(), set_result() • add_done_callback(), remove_done_callback() • ... -
Web development with Tulip • All the classes you need
are in the tulip.http module • Make subclass of tulip.http.ServerHttpProtocol • Override the handle_request() method -
Some observations The HTTP API is fairly simple, but a
bit low level for everyday web programming. Expect a thousand microframeworks to bloom in the near future. Speaking of microframeworks... -
Introducing viol • Tiny web framework based on Tulip •
After initial page load, messages between client and server are exchanged via websockets • Makes code demos a bit more visual -
Tulip Links • PEP 3156 • Guido's PyCon 2013 keynote
• Tulip project page on GoogleCode -
Other links • An Introduction to Asynchronous Programming and Twisted
• Code for this talk on Github • Viol project page on Github