Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
130
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
住手啊,這樣用 Python 寫程式沒有人會獲得幸福的
leew
0
20
Unlocking the Future of Data Pipeline
leew
0
72
Hold on! You have a data team in PyCon Taiwan!
leew
0
14
朝聖之路
leew
0
74
20250328 Airflow 3.0 The First Glance
leew
0
220
踏入開源的第一步
leew
0
180
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
84
Unlocking Python's Core Magic
leew
1
270
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
180
Other Decks in Technology
See All in Technology
スプリントレトロスペクティブはチーム観察の宝庫? 〜チームの衝突レベルに合わせたアプローチ仮説!〜
electricsatie
1
150
Function Body Macros で、SwiftUI の View に Accessibility Identifier を自動付与する/Function Body Macros: Autogenerate accessibility identifiers for SwiftUI Views
miichan
2
150
「魔法少女まどか☆マギカ Magia Exedra」のグローバル展開を支える、開発チームと翻訳チームの「意識しない協創」を実現するローカライズシステム
gree_tech
PRO
0
430
ヘブンバーンズレッドにおける、世界観を活かしたミニゲーム企画の作り方
gree_tech
PRO
0
430
DuckDB-Wasmを使って ブラウザ上でRDBMSを動かす
hacusk
1
140
生成AI時代に必要な価値ある意思決定を育てる「開発プロセス定義」を用いた中期戦略
kakehashi
PRO
1
240
ヘブンバーンズレッドのレンダリングパイプライン刷新
gree_tech
PRO
0
440
ヒューリスティック評価を用いたゲームQA実践事例
gree_tech
PRO
0
420
Skrub: machine-learning with dataframes
gaelvaroquaux
0
110
実運用で考える PGO
kworkdev
PRO
0
130
新規案件の立ち上げ専門チームから見たAI駆動開発の始め方
shuyakinjo
0
640
Flutterでキャッチしないエラーはどこに行く
taiju59
0
210
Featured
See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
It's Worth the Effort
3n
187
28k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
510
BBQ
matthewcrist
89
9.8k
Site-Speed That Sticks
csswizardry
10
800
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Navigating Team Friction
lara
189
15k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Building Applications with DynamoDB
mza
96
6.6k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.