Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Gilson Filho Gilson Filho
October 20, 2017
Programming
490
0

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Avatar for Gilson Filho

Gilson Filho

October 20, 2017

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
Avatar for Gilson Filho gilsondev
0
2.2k
Airflow - Automatizando seu fluxo de trabalho
Avatar for Gilson Filho gilsondev
1
1.3k
Protocolo HTTP
Avatar for Gilson Filho gilsondev
0
570
Workshop Docker
Avatar for Gilson Filho gilsondev
0
470
Django: framework web para perfeccionistas com prazos
Avatar for Gilson Filho gilsondev
0
450
Django - Framework para perfeccionistas em prazos
Avatar for Gilson Filho gilsondev
0
53
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
480
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
140
Desenvolvendo aplicações web com python e web2py
Avatar for Gilson Filho gilsondev
0
610

Other Decks in Programming

See All in Programming
From Formal Specification to Property Based Test
Avatar for Masato Ohba ohbarye
0
2.4k
Symfony AI in Action - SymfonyLive Berlin 2026
Avatar for Christopher Hertel chr_hertel
1
140
UaaL×Androidアプリのメモリ計測 — Memory Profilerの先へ
Avatar for Ryosuke Shimizu (RIO) rio432
0
150
Making the RBS Parser Faster
Avatar for Soutaro Matsumoto soutaro
0
710
PHPでローカル環境用のSSL/TLS証明書を発行することはできるのか？ #phpconkagawa
Avatar for akase244 akase244
0
360
t *testing.T は どこからやってくるの？
Avatar for Kotaro Otaka otakakot
1
930
KMP × Kotlin 2.3 - How Android Got Slower While iOS Builds Improved by 47%
Avatar for Ryosuke Shimizu (RIO) rio432
0
170
サプライチェーン攻撃対策「層を重ねて落ちない壁」を10日間で組み上げた話 #TechLeadConf2026
Avatar for Kashun Yoshida kashewnuts
1
250
ついに来た！本格的なマルチクラウド時代の Google Cloud
Avatar for maroon1st maroon1st
0
410
SkillsをS3 Filesに置く時のあれこれ
Avatar for watany watany
3
1.5k
Road to RubyKaigi: Play Hard(ware)
Avatar for makicamel makicamel
1
570
Explore CoroutineScope
Avatar for tomoEng11 tomoeng11
0
190

Featured

See All Featured
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
659
62k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
55k
Docker and Python
Avatar for Tania Allard trallard
47
3.8k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
698
190k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
200
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
130
Bioeconomy Workshop: Dr. Julius Ecuru, Opportunities for a Bioeconomy in West Africa
Avatar for AKADEMIYA2063 akademiya2063
PRO
1
110
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.4k
More Than Pixels: Becoming A User Experience Designer
Avatar for Michelle Schulp Hunt marktimemedia
3
400
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
3k
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
77
5.3k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.5k

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?