Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the...
Search
Lee Wei
May 08, 2024
Technology
0
140
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
住手啊,這樣用 Python 寫程式沒有人會獲得幸福的
leew
0
55
Unlocking the Future of Data Pipeline
leew
0
230
Hold on! You have a data team in PyCon Taiwan!
leew
0
21
朝聖之路
leew
0
85
20250328 Airflow 3.0 The First Glance
leew
0
260
踏入開源的第一步
leew
0
210
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
0
93
Unlocking Python's Core Magic
leew
1
280
Unleash the Chaos - Developing a Linter for Un-Pythonic Code!
leew
1
190
Other Decks in Technology
See All in Technology
AWS re:Invent 2025事前勉強会資料 / AWS re:Invent 2025 pre study meetup
kinunori
0
340
GraphRAG グラフDBを使ったLLM生成(自作漫画DBを用いた具体例を用いて)
seaturt1e
1
150
What's new in OpenShift 4.20
redhatlivestreaming
0
280
20251027_マルチエージェントとは
almondo_event
1
440
IoTLT@ストラタシスジャパン_20251021
norioikedo
0
140
現場の壁を乗り越えて、 「計装注入」が拓く オブザーバビリティ / Beyond the Field Barriers: Instrumentation Injection and the Future of Observability
aoto
PRO
1
610
AIエージェントによる業務効率化への飽くなき挑戦-AWS上の実開発事例から学んだ効果、現実そしてギャップ-
nasuvitz
5
1.2k
AIプロダクトのプロンプト実践テクニック / Practical Techniques for AI Product Prompts
saka2jp
0
110
「最速」で Gemini CLI を使いこなそう! 〜Cloud Shell/Cloud Run の活用〜 / The Fastest Way to Master the Gemini CLI — with Cloud Shell and Cloud Run
aoto
PRO
1
180
Biz職でもDifyでできる! 「触らないAIワークフロー」を実現する方法
igarashikana
7
3.5k
SQLAlchemy の select(User).where(User.id =="123") を理解してみる/sqlalchemy deep dive
3l4l5
3
450
AI-Readyを目指した非構造化データのメダリオンアーキテクチャ
r_miura
1
330
Featured
See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
508
140k
KATA
mclloyd
PRO
32
15k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Done Done
chrislema
185
16k
How to train your dragon (web standard)
notwaldorf
97
6.3k
The Cost Of JavaScript in 2023
addyosmani
55
9.1k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.5k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
116
20k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
The Cult of Friendly URLs
andyhume
79
6.6k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.