Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the triggerer
Search
Lee Wei
May 08, 2024
Technology
0
19
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
Intro to Airflow - From Zero to Hero
leew
0
730
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
56
Back to the last time we were here
leew
0
8
Atomic Commits - An Easy & Proven Way to Manage & Automate Release Process
leew
0
360
Python Table Manners @ Taichung.py
leew
0
300
Python Table Manners- Cut the Cookie Gracefully @ Hualien py
leew
0
160
小林的程式碼會不會遇到 SQL Injection
leew
1
93
Python Table Manners: Cut the Cookie Gracefully @ Kaohsiung.py
leew
0
370
commitizen-tools: What can we gain from crafting a git message convention? @ Tainan.py June
leew
0
83
Other Decks in Technology
See All in Technology
データベース研修 分析向けSQL入門【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
110
ここがすごいよ! AWS Systems Manager!
saichan11
0
1.8k
Azure Pipelinesを使用したCICDベースラインアーキテクチャ実践
yuriemori
0
190
RAGのサービスをリリースして1年3ヶ月が経ちました
segavvy
4
930
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
160
How to Think Like a Performance Engineer
csswizardry
4
590
「単なる OAuth 2.0 を認証に使うと、車が通れるほどのどでかいセキュリティー・ホールができる」のか検証してみた
terara
0
380
20240725 LLMによるDXのビジョンと、今何からやるべきか @Azure OpenAI Service Dev Day
nrryuya
3
1.2k
AIエージェントを現場に導入する目線とは
masahiro_nishimi
1
1.5k
【基調講演】変える、今ここから ― IoTとAIで紡ぐ未来
soracom
PRO
0
320
ABEMAにおけるLLMを用いたコンテンツベース推薦システム導入と効果検証
cyberagentdevelopers
PRO
1
740
[I/O Extended Android 2024] What`s new in Android 2024
kyeongwan
0
220
Featured
See All Featured
Raft: Consensus for Rubyists
vanstee
134
6.5k
5 minutes of I Can Smell Your CMS
philhawksworth
200
19k
Art, The Web, and Tiny UX
lynnandtonic
291
20k
Gamification - CAS2011
davidbonilla
78
4.9k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
19k
The Straight Up "How To Draw Better" Workshop
denniskardys
229
130k
Speed Design
sergeychernyshev
9
270
Writing Fast Ruby
sferik
623
60k
How to Think Like a Performance Engineer
csswizardry
4
590
Embracing the Ebb and Flow
colly
81
4.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
24
1.8k
Done Done
chrislema
179
15k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.