Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Avatar for Gilson Filho Gilson Filho
October 20, 2017
Programming
500
0

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Avatar for Gilson Filho

Gilson Filho

October 20, 2017

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
Avatar for Gilson Filho gilsondev
0
2.2k
Airflow - Automatizando seu fluxo de trabalho
Avatar for Gilson Filho gilsondev
1
1.3k
Protocolo HTTP
Avatar for Gilson Filho gilsondev
0
570
Workshop Docker
Avatar for Gilson Filho gilsondev
0
480
Django: framework web para perfeccionistas com prazos
Avatar for Gilson Filho gilsondev
0
460
Django - Framework para perfeccionistas em prazos
Avatar for Gilson Filho gilsondev
0
57
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
490
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
150
Desenvolvendo aplicações web com python e web2py
Avatar for Gilson Filho gilsondev
0
610

Other Decks in Programming

See All in Programming
Hunting Vulnerabilities in Symfony with LLMs
Avatar for Vincent Amstoutz vinceamstoutz
0
550
Make SRE Operations Easier with Azure SRE Agent
Avatar for KAMEGAWA Kazushi kkamegawa
0
7.3k
New "Type" system on PicoRuby
Avatar for pocke pocke
1
990
Language Server 使ってる？ 〜VSCode と Zed の場合〜 / Are you using a Language Server? ~For VS Code and Zed~
Avatar for NAGATA Hiroaki handlename
0
800
生成AI時代にこそ効くGo | Why Go Works in the Age of Generative AI
Avatar for mom0tomo mom0tomo
8
3.3k
代数的データ型って何が嬉しいの？ #frontend_phpcon_do
Avatar for Takuma Kajikawa kajitack
8
3.7k
正しくソフトウェアを作る、前提を疑うための認知の視点 / doubt-premise
Avatar for MinoDriven minodriven
21
6.9k
依存関係から依存物へ―Dependencyという言葉の歴史をひも解く
Avatar for j_lee_inst j_lee
0
130
Spec Driven Development | AI Summit Lisbon
Avatar for Daniel Sogl danielsogl
PRO
0
200
TSKaigi Night Talks 2026_TypeScriptでサプライチェーンの整合性を型に閉じ込める
Avatar for ギークプラス ソフトウェア事業部 geekplus_tech
0
400
Developing with AI Agents — Codex, Claude Code & Cowork Practical Guide
Avatar for Daisuke Masuda x5gtrn
PRO
0
1.3k
軽量Java基盤の設計 DIコンテナに頼らない、長期保守と1秒起動の実現 JJUG CCC 2026 Spring
Avatar for Masaaki Takahashi macha64
0
550

Featured

See All Featured
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
22k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
6.9k
The SEO Collaboration Effect
Avatar for Kristina Bergwall kristinabergwall1
1
490
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
230
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4.3k
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
360
Paper Plane
Avatar for Katie Cordina Lindsey katiecoart
PRO
1
52k
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
270
14k
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
5.1k
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.1k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
672
58k
Art, The Web, and Tiny UX
Avatar for Lynn Fisher lynnandtonic
304
22k

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?