like Twisted, Tornado, or even gevent to either adapt the default event loop implementation to their needs using a lightweight adapter or proxy, or to replace the default event loop implementation with an adaptation of their own event loop implementation.” “Interoperability - asyncio” https://www.python.org/dev/peps/pep-3156/
adaptation in third party frameworks is to keep the default event loop and adapt it to the framework's API. Ideally all third party frameworks would give up their own event loop implementation in favor of the standard implementation.” “Interoperability - asyncio” https://www.python.org/dev/peps/pep-3156/
I/O thing. Therefore they are the same kind of thing. I only need one kind of thing in each category of thing. Therefore I only need one of them, and the “standard” one is probably the better one to depend on. So I guess nobody will need Twisted any more!” https://glyph.twistedmatrix.com/2014/05/the-report-of-our-death.html
function is complicated by the need to prevent unbounded recursion # arising from repeatedly yielding immediately ready deferreds. This while # loop and the waiting variable solve that by manually unfolding the # recursion. waiting = [True, # waiting for result? None] # result while 1: try: # Send the last result back as the result of the yield expression. isFailure = isinstance(result, failure.Failure) if isFailure: result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: # fell off the end, or "return" statement deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: # returnValue() was called; time to give a result to the original # Deferred. First though, let's try to identify the potentially # confusing situation which results when returnValue() is # accidentally invoked from a different function, one that wasn't # decorated with @inlineCallbacks. # The traceback starts in this frame (the one for # _inlineCallbacks); the next one down should be the application # code. appCodeTrace = exc_info()[2].tb_next if isFailure: # If we invoked this generator frame by throwing an exception # into it, then throwExceptionIntoGenerator will consume an # additional stack frame itself, so we need to skip that too. appCodeTrace = appCodeTrace.tb_next # Now that we've identified the frame being exited by the # exception, let's figure out if returnValue was called from it # directly. returnValue itself consumes a stack frame, so the # application code will have a tb_next, but it will *not* have a # second tb_next. if appCodeTrace.tb_next.tb_next: # If returnValue was invoked non-local to the frame which it is # exiting, identify the frame that ultimately invoked # returnValue so that we can warn the user, as this behavior is # confusing. ultimateTrace = appCodeTrace while ultimateTrace.tb_next.tb_next: ultimateTrace = ultimateTrace.tb_next filename = ultimateTrace.tb_frame.f_code.co_filename lineno = ultimateTrace.tb_lineno warnings.warn_explicit( "returnValue() in %r causing %r to exit: " "returnValue should only be invoked by functions decorated " "with inlineCallbacks" % ( ultimateTrace.tb_frame.f_code.co_name, appCodeTrace.tb_frame.f_code.co_name), DeprecationWarning, filename, lineno) deferred.callback(e.value) return deferred except: deferred.errback() return deferred if isinstance(result, Deferred): # a deferred was yielded, get the result. def gotResult(r): if waiting[0]: waiting[0] = False waiting[1] = r else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: # Haven't called back yet, set flag so that we get reinvoked # and return from the loop waiting[0] = False return deferred result = waiting[1] # Reset waiting to initial values for next loop. gotResult uses # waiting, but this isn't a problem because gotResult is only # executed once, and if it hasn't been executed yet, the return # branch above would have been taken. waiting[0] = True waiting[1] = None def inlineCallbacks(f): """ inlineCallbacks helps you write L{Deferred}-using code that looks like a regular sequential function. For example:: @inlineCallbacks def thingummy(): thing = yield makeSomeRequestResultingInDeferred() print(thing) # the result! hoorj! When you call anything that results in a L{Deferred}, you can simply yield it; your generator will automatically be resumed when the Deferred's result is available. The generator will be sent the result of the L{Deferred} with the 'send' method on generators, or if the result was a failure, 'throw'. Things that are not L{Deferred}s may also be yielded, and your generator will be resumed with the same object sent back. This means C{yield} performs an operation roughly equivalent to L{maybeDeferred}. Your inlineCallbacks-enabled generator will return a L{Deferred} object, which will result in the return value of the generator (or will fail with a failure object if your generator raises an unhandled exception). Note that you can't use C{return result} to return a value; use C{returnValue(result)} instead. Falling off the end of the generator, or simply using C{return} will cause the L{Deferred} to have a result of C{None}. Be aware that L{returnValue} will not accept a L{Deferred} as a parameter. If you believe the thing you'd like to return could be a L{Deferred}, do this:: result = yield result returnValue(result) The L{Deferred} returned from your deferred generator may errback if your generator raised an exception:: @inlineCallbacks def thingummy(): thing = yield makeSomeRequestResultingInDeferred() if thing == 'I love Twisted': # will become the result of the Deferred returnValue('TWISTED IS GREAT!') else: # will trigger an errback raise Exception('DESTROY ALL LIFE') If you are using Python 3.3 or later, it is possible to use the C{return} statement instead of L{returnValue}:: @inlineCallbacks def loadData(url): response = yield makeRequest(url) return json.loads(response) """ @wraps(f) def unwindGenerator(*args, **kwargs): try: gen = f(*args, **kwargs) except _DefGen_Return: raise TypeError( "inlineCallbacks requires %r to produce a generator; instead" "caught returnValue being used in a non-generator" % (f,)) if not isinstance(gen, types.GeneratorType): raise TypeError( "inlineCallbacks requires %r to produce a generator; " "instead got %r" % (f, gen)) return _inlineCallbacks(None, gen, Deferred()) return unwindGenerator
return _inlineCallbacks(None, gen, Deferred()) return unwindGenerator 2 _inlineCallbacks is called with an initial result of None, the generator, and a new Deferred.
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred We send result (None) into the generator, which spins it off and gets to the first Deferred 3
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] We tell the returned Deferred to call gotResult when it fires. 4
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] If we’re waiting for a result (which because this is the first run we are), return the Deferred from the original inlineCallbacks call, and say that we’re not waiting for a result. 5
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] When the Deferred returned by the generator fires, it checks to see if it is waiting for a result. If it is not, it calls _inlineCallbacks again with the new result. 6
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred Now result is a value we have, so it is sent in to the generator. 7
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred If this is the final result, we will get either StopIteration, _DefGen_Return, or an exception. 8
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred StopIteration is when it runs off the “end”, or in Python 3, when you use return in a generator. 9a
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred On Python 2, you can’t return from a generator. Twisted uses returnValue, which raises this special exception which holds a value. 9b
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred Sometimes, the generator will raise an exception before it yields its next value. 9c
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred In all three cases, the Deferred created in the initial inlineCallbacks call is fired with a result (or a failure). 9c 9b 9a
set its result. If the future is already done when this method is called, raises InvalidStateError. """ if self._state != _PENDING: raise InvalidStateError('{}: {!r}'.format(self._state, self)) self._result = result self._state = _FINISHED self._schedule_callbacks() def _schedule_callbacks(self): """Internal: Ask the event loop to call all callbacks. The callbacks are scheduled to be called as soon as possible. Also clears the callback list. """ callbacks = self._callbacks[:] if not callbacks: return self._callbacks[:] = [] for callback in callbacks: self._loop.call_soon(callback, self)
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] If the result Deferred already has a result, addBoth will be called synchronously 10
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] We ARE waiting for a result, and we have it, so we set it to say that we don’t, and put the result beside it. 11
def gotResult(r): if waiting[0]: waiting = [False, r] else: _inlineCallbacks(r, g, deferred) result.addBoth(gotResult) if waiting[0]: waiting[0] = False return deferred result = waiting[1] waiting = [True, None] Because we’re not waiting for a result, this code is ran, and we say that we want a new value. 12
for result and the result while 1: try: if isinstance(result, failure.Failure): result = result.throwExceptionIntoGenerator(g) else: result = g.send(result) except StopIteration as e: deferred.callback(getattr(e, "value", None)) return deferred except _DefGen_Return as e: deferred.callback(e.value) return deferred except: deferred.errback() return deferred We loop back up, and send the synchronously-got result back into the generator. And so on, so forth. 13
_MakeDeferredAwaitable(object): """ A subgenerator which is created by L{Deferred.__await__}. """ result = _NO_RESULT def __init__(self, d): self.d = d self.d.addBoth(self._setResult) def _setResult(self, value): self.result = value def __next__(self): if self.result is not _NO_RESULT: raise StopIteration(self.result) return self.d def __await__(self): return self def _awaitTick(result, coro, deferred): try: result = coro.send(result) except StopIteration as e: deferred.callback(e.value) except: deferred.errback() if isinstance(result, Deferred): result.addBoth(_awaitTick, coro=coro, deferred=deferred) return deferred def deferredCoroutine(f): """ A decorator for supporting coroutine-style programming using L{Deferred}s. It implements the awaitable protocol from PEP-0492. When using this decorator, the wrapped function may use the C{await} keyword to suspend execution until the awaited L{Deferred} has fired. For example:: import treq from twisted.internet.defer import deferredCoroutine from twisted.internet.task import react def main(reactor): pages = [ "https://google.com/", "https://twistedmatrix.com", ] d = crawl(pages) d.addCallback(print) return d @deferredCoroutine async def crawl(pages): results = {} for page in pages: results[page] = await treq.content(await treq.get(page)) return results react(main) In the above example, L{treq.get} and L{treq.content} return L{Deferreds}, and the decorated function returns a L{Deferred} itself. """ def wrapped(*args, **kwargs): coro = f(*args, **kwargs) return _awaitTick(None, coro, Deferred()) return wrapped
hawkowl
ufried
masahirokanatani
evilsmile
shoheiyamamoto
yukiarrr
nobuakikikuchi
smzksts
clustervr
sumi
yfutamura
picardparis
adarapata
smashingmag
rmw
yeseniaperezcruz
destraynor
colly
jeffersonlam
bryan
roundedbygravity
jasonvnalue
moore
tanoku
malarkey
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_91.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_95.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_96.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_97.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_98.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_99.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_100.jpg){kind=link}
![def _inlineCallbacks(result, g, deferred): waiting = [True, None] # waiting](https://files.speakerdeck.com/presentations/3ccf026ceed341019df093bc87f5ffae/slide_108.jpg){kind=link}
