Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Avatar for Gilson Filho Gilson Filho
October 20, 2017
Programming
0
490

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Avatar for Gilson Filho

Gilson Filho

October 20, 2017
Tweet

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
Avatar for Gilson Filho gilsondev
0
2.1k
Airflow - Automatizando seu fluxo de trabalho
Avatar for Gilson Filho gilsondev
1
1.3k
Protocolo HTTP
Avatar for Gilson Filho gilsondev
0
570
Workshop Docker
Avatar for Gilson Filho gilsondev
0
460
Django: framework web para perfeccionistas com prazos
Avatar for Gilson Filho gilsondev
0
450
Django - Framework para perfeccionistas em prazos
Avatar for Gilson Filho gilsondev
0
52
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
480
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
140
Desenvolvendo aplicações web com python e web2py
Avatar for Gilson Filho gilsondev
0
610

Other Decks in Programming

See All in Programming
The Past, Present, and Future of Enterprise Java
Avatar for ivargrimstad ivargrimstad
0
1k
Feature Toggle は捨てやすく使おう
Avatar for gennei gennei
0
340
Goの型安全性で実現する複数プロダクトの権限管理
Avatar for ishikawa-pro ishikawa_pro
2
1.4k
エンジニアの「手元の自動化」を加速するn8n 2026.02.27
Avatar for 西尾義英 symy2co
0
180
安いハードウェアでVulkan
Avatar for Fadis fadis
1
790
コードレビューをしない選択 #でぃーぷらすトウキョウ
Avatar for Takuma Kajikawa kajitack
3
1.1k
Symfony + NelmioApiDocBundle を使った スキーマ駆動開発 / Schema Driven Development with NelmioApiDocBundle
Avatar for Shohei Okada okashoi
0
230
Reactive ❤️ Loom: A Forbidden Love Story
Avatar for Francesco Nigro franz1981
2
160
野球解説AI Agentを開発してみた - 2026/02/27 LayerX社内LT会資料
Avatar for Shinichi Nakagawa shinyorke
PRO
0
370
PHP でエミュレータを自作して Ubuntu を動かそう
Avatar for memory m3m0r7
PRO
2
140
Codex CLIのSubagentsによる並列API実装 / Parallel API Implementation with Codex CLI Subagents
Avatar for 冨澤宝斗 takatty
2
430
Codex の「自走力」を高める
Avatar for yorifuji yorifuji
0
1.3k

Featured

See All Featured
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
Avatar for Ines Montani inesmontani
PRO
0
280
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
エンジニアに許された特別な時間の終わり
Avatar for watany watany
106
240k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
120
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
55k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
93
Kristin Tynski - Automating Marketing Tasks With AI
Avatar for Tech SEO Connect techseoconnect
PRO
0
200
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
130
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1032
470k
Ruling the World: When Life Gets Gamed
Avatar for Sebastian Deterding codingconduct
0
180
Building Better People: How to give real-time feedback that sticks.
Avatar for Will Jessup wjessup
370
20k
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
62
53k

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?