Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Avatar for Gilson Filho Gilson Filho
October 20, 2017
Programming
490
0

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Avatar for Gilson Filho

Gilson Filho

October 20, 2017

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
Avatar for Gilson Filho gilsondev
0
2.2k
Airflow - Automatizando seu fluxo de trabalho
Avatar for Gilson Filho gilsondev
1
1.3k
Protocolo HTTP
Avatar for Gilson Filho gilsondev
0
570
Workshop Docker
Avatar for Gilson Filho gilsondev
0
460
Django: framework web para perfeccionistas com prazos
Avatar for Gilson Filho gilsondev
0
450
Django - Framework para perfeccionistas em prazos
Avatar for Gilson Filho gilsondev
0
53
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
480
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
140
Desenvolvendo aplicações web com python e web2py
Avatar for Gilson Filho gilsondev
0
610

Other Decks in Programming

See All in Programming
PHP でエミュレータを自作して Ubuntu を動かそう
Avatar for memory m3m0r7
PRO
2
170
Vibe NLP for Applied NLP
Avatar for Ines Montani inesmontani
PRO
0
210
iOS機能開発のAI環境と起きた変化
Avatar for Ryu-nakayama ryunakayama
0
170
仕様漏れ実装漏れをなくすトレーサビリティAI基盤のご紹介
Avatar for Kuniwak orgachem
PRO
9
5.3k
PHP で mp3 プレイヤーを実装しよう
Avatar for memory m3m0r7
PRO
0
210
アーキテクチャモダナイゼーションとは何か
Avatar for nwiizo nwiizo
17
4.7k
Redox OS でのネームスペース管理と chroot の実現
Avatar for isan isanethen
0
560
SkillがSkillを生む：QA観点出しを自動化した
Avatar for sontixyou sontixyou
6
3.2k
実践ハーネスエンジニアリング #MOSHTech
Avatar for Takuma Kajikawa kajitack
7
6.1k
10年分の技術的負債、完済へ ― Claude Code主導のAI駆動開発でスポーツブルを丸ごとリプレイスした話
Avatar for nero15 takuya_houshima
0
2.2k
テレメトリーシグナルが導くパフォーマンス最適化 / Performance Optimization Driven by Telemetry Signals
Avatar for shiro seike seike460
PRO
2
220
AWS re：Invent 2025の少し振り返り ＋ DevOps AgentとBacklogを連携させてみた
Avatar for Satoshi Kaneyasu satoshi256kbyte
2
150

Featured

See All Featured
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
55k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
39
3.1k
Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
Avatar for Ines Montani inesmontani
PRO
3
2.1k
Test your architecture with Archunit
Avatar for Yoan thirion
1
2.2k
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
Avatar for Mine Cetinkaya-Rundel minecr
1
220
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
Avatar for Sara Fernández Carmona sarafernandez
2
1.4k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
274
21k
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
62
53k
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
430
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
5
35k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
408
66k
Paper Plane (Part 1)
Avatar for Katie Cordina Lindsey katiecoart
PRO
0
6.5k

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?