Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
120
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
朝聖之路
leew
0
58
20250328 Airflow 3.0 The First Glance
leew
0
170
踏入開源的第一步
leew
0
160
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
69
Unlocking Python's Core Magic
leew
1
260
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
160
Intro to Airflow - From Zero to Hero
leew
0
970
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
120
Back to the last time we were here
leew
0
36
Other Decks in Technology
See All in Technology
AIのAIによるAIのための出力評価と改善
chocoyama
2
550
Welcome to the LLM Club
koic
0
170
Prox Industries株式会社 会社紹介資料
proxindustries
0
290
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
340
Node-REDのFunctionノードでMCPサーバーの実装を試してみた / Node-RED × MCP 勉強会 vol.1
you
PRO
0
110
Fabric + Databricks 2025.6 の最新情報ピックアップ
ryomaru0825
1
140
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
26k
なぜ私はいま、ここにいるのか? #もがく中堅デザイナー #プロダクトデザイナー
bengo4com
0
440
HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representation
spatial_ai_network
0
110
“社内”だけで完結していた私が、AWS Community Builder になるまで
nagisa53
1
390
AIの最新技術&テーマをつまんで紹介&フリートークするシリーズ #1 量子機械学習の入門
tkhresk
0
140
Lambda Web Adapterについて自分なりに理解してみた
smt7174
3
110
Featured
See All Featured
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Making Projects Easy
brettharned
116
6.3k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.3k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
Gamification - CAS2011
davidbonilla
81
5.3k
Scaling GitHub
holman
459
140k
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
490
Large-scale JavaScript Application Architecture
addyosmani
512
110k
A Tale of Four Properties
chriscoyier
160
23k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
60k
Unsuck your backbone
ammeep
671
58k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.