Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Starts Airflow task execution directly from the triggerer
Search
Lee Wei
May 08, 2024
Technology
0
16
Starts Airflow task execution directly from the triggerer
Lee Wei
May 08, 2024
Tweet
Share
More Decks by Lee Wei
See All by Lee Wei
Intro to Airflow - From Zero to Hero
leew
0
730
Does Kobayashi-San write code vulnerable to SQL injection?
leew
0
50
Back to the last time we were here
leew
0
8
Atomic Commits - An Easy & Proven Way to Manage & Automate Release Process
leew
0
350
Python Table Manners @ Taichung.py
leew
0
300
Python Table Manners- Cut the Cookie Gracefully @ Hualien py
leew
0
160
小林的程式碼會不會遇到 SQL Injection
leew
1
92
Python Table Manners: Cut the Cookie Gracefully @ Kaohsiung.py
leew
0
370
commitizen-tools: What can we gain from crafting a git message convention? @ Tainan.py June
leew
0
77
Other Decks in Technology
See All in Technology
240622_WACATE2024夏_BPPセッション_虚実皮膜のWACATEハイ
mkwrd
PRO
1
270
Next.js の fetch 拡張とキャッシュ機構の違いを理解する
ryo_manba
3
540
フロントエンドの Monorepo をやめてリポジトリ分割したワケ / Why did we stop using Monorepo on the frontend and split the repository?
kaminashi
6
2.8k
EC2の脆弱性対応で何が使える? Inspector や SSM あたりを整理する #nakanoshima_dev
masahirokawahara
1
160
LeanとDevOpsのためにE2Eテストができること
magicpod
2
530
AWSの生成AIサービス Amazon Bedrock入門!(2024/6月版)
minorun365
PRO
5
1.3k
半年かけてPHP5.6からPHP7.4までバージョンアップした苦労と工夫 PHPカンファレンス福岡2024
kechiiin
1
170
フィーチャー開発から ホールプロダクト開発へ ~ 顧客価値へ向き合い続ける挑戦 ~ @itohiro73 #開発生産性con_findy
itohiro73
5
950
Bring your app’s core features to users with App Intents とか App Intents 関連の要約
ryomm
1
140
10社以上のCTO/技術顧問を経験してみえた 技術組織に起こる課題と対策
otani_yuji
3
1.2k
はてなのチーム開発一巡り / Hatena Engineer Seminar 30
daiksy
0
220
超アナログ中心な印刷会社で「エンジニアリング」を見直す
logica0419
4
150
Featured
See All Featured
From Idea to $5000 a Month in 5 Months
shpigford
377
46k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
73
15k
Gamification - CAS2011
davidbonilla
77
4.8k
Scaling GitHub
holman
457
140k
YesSQL, Process and Tooling at Scale
rocio
165
14k
A Modern Web Designer's Workflow
chriscoyier
689
190k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
20
1.7k
Optimising Largest Contentful Paint
csswizardry
14
2.6k
The Pragmatic Product Professional
lauravandoore
28
6k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
The Brand Is Dead. Long Live the Brand.
mthomps
51
36k
Six Lessons from altMBA
skipperchong
22
3.2k
Transcript
Starts Air fl ow task execution directly from the triggerer
$ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__
= [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
File "weilee.py", line 1 __name__ = 李唯 / Wei Lee
^^^ SyntaxError: invalid syntax $ python weilee.py
None
How did deferrable operators work before this change?
What has been changed?
What has been changed?
What it could be in the future
But why?
"execute" methods are not always needed This is the only
logic when deferrable is set to true and does not need to be executed in the worker.
How does it affect DAG authors? More ef fi cient
operators to come
How does it affect operator authors? A new way to
implement operators in an async manner
The Original Way
The New Way
How does it work under the hook?
How does it work under the hook?
How does it work under the hook?
Limitations • It is necessary to go back to workers
to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.