Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
130
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
Hold on! You have a data team in PyCon Taiwan!
leew
0
10
朝聖之路
leew
0
64
20250328 Airflow 3.0 The First Glance
leew
0
190
踏入開源的第一步
leew
0
170
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
75
Unlocking Python's Core Magic
leew
1
260
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
160
Intro to Airflow - From Zero to Hero
leew
0
980
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
120
Other Decks in Technology
See All in Technology
地図と生成AI
nakasho
0
700
エンジニアリングマネージャー“お悩み相談”パネルセッション
ar_tama
1
650
ゼロから始めるSREの事業貢献 - 生成AI時代のSRE成長戦略と実践 / Starting SRE from Day One
shinyorke
PRO
0
230
AI駆動開発 with MixLeap Study【大阪支部 #3】
lycorptech_jp
PRO
0
200
ecspressoの設計思想に至る道 / sekkeinight2025
fujiwara3
9
1.4k
RapidPen: AIエージェントによる高度なペネトレーションテスト自動化の研究開発
laysakura
1
390
会社もクラウドも違うけど 通じたコスト削減テクニック/Cost optimization strategies effective regardless of company or cloud provider
aeonpeople
2
160
AI エンジニアの立場からみた、AI コーディング時代の開発の品質向上の取り組みと妄想
soh9834
6
260
スプリントレビューを効果的にするために
miholovesq
9
1.6k
スプリントゴール未達症候群に送る処方箋
kakehashi
PRO
1
190
公開初日に個人環境で試した Gemini CLI 体験記など / Gemini CLI実験レポート
you
PRO
3
320
Talk to Someone At Delta Airlines™️ USA Contact Numbers
travelcarecenter
0
170
Featured
See All Featured
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Mobile First: as difficult as doing things right
swwweet
223
9.7k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Docker and Python
trallard
45
3.5k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Side Projects
sachag
455
43k
Scaling GitHub
holman
461
140k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.