Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
83
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
30
Unlocking Python's Core Magic
leew
1
220
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
130
Intro to Airflow - From Zero to Hero
leew
0
870
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
95
Back to the last time we were here
leew
0
23
Atomic Commits - An Easy & Proven Way to Manage & Automate Release Process
leew
0
430
Python Table Manners @ Taichung.py
leew
0
340
Python Table Manners- Cut the Cookie Gracefully @ Hualien py
leew
0
200
Other Decks in Technology
See All in Technology
Amazon Aurora のバージョンアップ手法について
smt7174
2
190
DeepSeekとは?何がいいの? - Databricksと学ぶDeepSeek! 〜これからのLLMに備えよ!〜
taka_aki
1
180
Snowflake ML モデルを dbt データパイプラインに組み込む
estie
0
120
ディスプレイ広告(Yahoo!広告・LINE広告)におけるバックエンド開発
lycorptech_jp
PRO
0
590
ABWG2024採択者が語るエンジニアとしての自分自身の見つけ方〜発信して、つながって、世界を広げていく〜
maimyyym
1
220
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
2
1.1k
事業を差別化する技術を生み出す技術
pyama86
2
520
2025/3/1 公共交通オープンデータデイ2025
morohoshi
0
110
どちらかだけじゃもったいないかも? ECSとEKSを適材適所で併用するメリット、運用課題とそれらの対応について
tk3fftk
2
280
ExaDB-XSで利用されているExadata Exascaleについて
oracle4engineer
PRO
3
300
事業モメンタムを生み出すプロダクト開発
macchiitaka
0
110
EDRの検知の仕組みと検知回避について
chayakonanaika
12
5.3k
Featured
See All Featured
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Typedesign – Prime Four
hannesfritz
41
2.5k
Rails Girls Zürich Keynote
gr2m
94
13k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.4k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
For a Future-Friendly Web
brad_frost
176
9.6k
Six Lessons from altMBA
skipperchong
27
3.6k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.