Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Gilson Filho Gilson Filho
October 20, 2017
Programming
500
0

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Avatar for Gilson Filho

Gilson Filho

October 20, 2017

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
Avatar for Gilson Filho gilsondev
0
2.2k
Airflow - Automatizando seu fluxo de trabalho
Avatar for Gilson Filho gilsondev
1
1.3k
Protocolo HTTP
Avatar for Gilson Filho gilsondev
0
570
Workshop Docker
Avatar for Gilson Filho gilsondev
0
490
Django: framework web para perfeccionistas com prazos
Avatar for Gilson Filho gilsondev
0
460
Django - Framework para perfeccionistas em prazos
Avatar for Gilson Filho gilsondev
0
57
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
490
Vim - Um editor onipresente e flexível
Avatar for Gilson Filho gilsondev
2
150
Desenvolvendo aplicações web com python e web2py
Avatar for Gilson Filho gilsondev
0
610

Other Decks in Programming

See All in Programming
Building a Meta Ray-Ban display app
Avatar for Akio Itaya akkeylab
0
140
仕様駆動開発へのトライを機に チームに適合する手法を模索し続けている話
Avatar for freee freee
PRO
0
500
Foundation Models frameworkで画像分析
Avatar for リョウ ryodeveloper
1
610
Claude CodeとAgentCore Gatewayを繋ぐ際の認証認可 / Authentication and authorization when connecting Claude Code with AgentCore Gateway
Avatar for Har1101 har1101
2
270
torikago - Ruby::Boxで照らすモジュラモノリスの実行境界
Avatar for nori se4weed
1
370
yield再入門 #phpcon
Avatar for hideki kinjyo o0h
PRO
0
1k
メールのエイリアス機能を履き違えない
Avatar for 船長（せんちょう） isshinfunada
0
240
Terraform標準の組織で AWS CDKをどう使うか
Avatar for mu7889yoon / Yuta Nakamura mu7889yoon
1
510
【やさしく解説 設計編・中級 #4】ルールの寿命と、システムの年輪
Avatar for HideyukiKitao panda728
PRO
2
180
Jindong: Introducing Declarative Haptics in Compose Multiplatform
Avatar for HyunWoo Lee l2hyunwoo
0
110
komatsuna「分散システムにおけるバグ分析手法」
Avatar for こまつな komatsunaqa
0
260
Claude Code全社展開のためにやったことn選～プラグイン302個・コミッター271人を支えるために～
Avatar for Kenichi Takahashi kenchan
5
1.5k

Featured

See All Featured
Measuring & Analyzing Core Web Vitals
Avatar for Philip Tellis bluesmoon
9
950
How Fast Is Fast Enough? [PerfNow 2025]
Avatar for Tammy Everts tammyeverts
3
760
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
6.2k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
370
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
300
What’s in a name? Adding method to the madness
Avatar for The Alliance productmarketing
PRO
24
4.1k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.9k
What does AI have to do with Human Rights?
Avatar for Per Axbom axbom
PRO
1
2.3k
16th Malabo Montpellier Forum Presentation
Avatar for AKADEMIYA2063 akademiya2063
PRO
0
350
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon Słowik szymonslowik
1
380
We Analyzed 250 Million AI Search Results: Here's What I Found
Avatar for Josh Blyskal joshbly
1
1.8k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?