Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
130
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
住手啊,這樣用 Python 寫程式沒有人會獲得幸福的
leew
0
45
Unlocking the Future of Data Pipeline
leew
0
170
Hold on! You have a data team in PyCon Taiwan!
leew
0
15
朝聖之路
leew
0
76
20250328 Airflow 3.0 The First Glance
leew
0
230
踏入開源的第一步
leew
0
180
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
85
Unlocking Python's Core Magic
leew
1
270
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
180
Other Decks in Technology
See All in Technology
スマートファクトリーの第一歩 〜AWSマネージドサービスで 実現する予知保全と生成AI活用まで
ganota
2
310
slog.Handlerのよくある実装ミス
sakiengineer
4
460
Codeful Serverless / 一人運用でもやり抜く力
_kensh
7
450
はじめてのOSS開発からみえたGo言語の強み
shibukazu
3
970
「何となくテストする」を卒業するためにプロダクトが動く仕組みを理解しよう
kawabeaver
0
430
テストを軸にした生き残り術
kworkdev
PRO
0
210
IoT x エッジAI - リアルタイ ムAI活用のPoCを今すぐ始め る方法 -
niizawat
0
110
OCI Oracle Database Services新機能アップデート(2025/06-2025/08)
oracle4engineer
PRO
0
180
今日から始めるAWSセキュリティ対策 3ステップでわかる実践ガイド
yoshidatakeshi1994
0
110
要件定義・デザインフェーズでもAIを活用して、コミュニケーションの密度を高める
kazukihayase
0
120
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
10
75k
まずはマネコンでちゃちゃっと作ってから、それをCDKにしてみよか。
yamada_r
2
120
Featured
See All Featured
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
6.1k
Building an army of robots
kneath
306
46k
Bash Introduction
62gerente
615
210k
How to Ace a Technical Interview
jacobian
279
23k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Designing for humans not robots
tammielis
253
25k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
188
55k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.6k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.