Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Airflow: Data Workflow Management
Search
Gilson Filho
October 20, 2017
Programming
0
470
Airflow: Data Workflow Management
Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte
Gilson Filho
October 20, 2017
Tweet
Share
More Decks by Gilson Filho
See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
gilsondev
0
2.1k
Airflow - Automatizando seu fluxo de trabalho
gilsondev
1
1.3k
Protocolo HTTP
gilsondev
0
550
Workshop Docker
gilsondev
0
430
Django: framework web para perfeccionistas com prazos
gilsondev
0
420
Django - Framework para perfeccionistas em prazos
gilsondev
0
47
Vim - Um editor onipresente e flexível
gilsondev
2
470
Vim - Um editor onipresente e flexível
gilsondev
2
130
Desenvolvendo aplicações web com python e web2py
gilsondev
0
600
Other Decks in Programming
See All in Programming
tool ディレクティブを導入してみた感想
sgash708
1
160
兎に角、コードレビュー
mitohato14
0
160
CSC305 Summer Lecture 12
javiergs
PRO
0
130
250830 IaCの選定~AWS SAMのLambdaをECSに乗り換えたときの備忘録~
east_takumi
0
350
レガシープロジェクトで最大限AIの恩恵を受けられるようClaude Codeを利用する
tk1351
4
1.5k
AIエージェント開発、DevOps and LLMOps
ymd65536
1
370
OSS開発者という働き方
andpad
5
1.6k
Namespace and Its Future
tagomoris
6
660
フロントエンドのmonorepo化と責務分離のリアーキテクト
kajitack
2
150
ECS初心者の仲間 – TUIツール「e1s」の紹介
keidarcy
0
140
為你自己學 Python - 冷知識篇
eddie
1
310
モバイルアプリからWebへの横展開を加速した話_Claude_Code_実践術.pdf
kazuyasakamoto
0
290
Featured
See All Featured
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
185
54k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
A designer walks into a library…
pauljervisheath
207
24k
Optimizing for Happiness
mojombo
379
70k
Designing for Performance
lara
610
69k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Become a Pro
speakerdeck
PRO
29
5.5k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.6k
Transcript
Airflow Data Workflow Management
O que é? Execução de workflow de dados Escritos em
Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify
DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio
Código enxuto e fácil de entender quando criado
DAG Airflow - Backfill
DAG Airflow - Visualização de Grafos
DAG Airflow - Detalhes
Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data
quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb
meucandidato.me
Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf
Email:
[email protected]
Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que
posso ajudar?