Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Airflow: Data Workflow Management

Gilson Filho
October 20, 2017
Programming
0
460

Airflow: Data Workflow Management

Apresentação do Airflow como Lightning Talk na Python Brasil[13] em Belo Horizonte

Gilson Filho

October 20, 2017
Tweet

More Decks by Gilson Filho

See All by Gilson Filho
Como monitorar os erros da sua aplicação com o Sentry
gilsondev
0
2.1k
Airflow - Automatizando seu fluxo de trabalho
gilsondev
1
1.2k
Protocolo HTTP
gilsondev
0
530
Workshop Docker
gilsondev
0
410
Django: framework web para perfeccionistas com prazos
gilsondev
0
410
Django - Framework para perfeccionistas em prazos
gilsondev
0
46
Vim - Um editor onipresente e flexível
gilsondev
2
460
Vim - Um editor onipresente e flexível
gilsondev
2
120
Desenvolvendo aplicações web com python e web2py
gilsondev
0
590

Other Decks in Programming

See All in Programming
GitHub Actions × RAGでコードレビューの検証の結果
sho_000
0
280
なぜイベント駆動が必要なのか - CQRS/ESで解く複雑系システムの課題 -
j5ik2o
13
4.5k
dbt Pythonモデルで実現するSnowflake活用術
trsnium
0
230
お前もAI鬼にならないか？👹Bolt & Cursor & Supabase & Vercelで人間をやめるぞ、ジョジョー！👺
taishiyade
7
4.1k
Djangoアプリケーション 運用のリアル 〜問題発生から可視化、最適化への道〜 #pyconshizu
kashewnuts
1
250
Serverless Rust: Your Low-Risk Entry Point to Rust in Production (and the benefits are huge)
lmammino
1
140
負債になりにくいCSSをデザイナとつくるには？
fsubal
10
2.5k
AIプログラミング雑キャッチアップ
yuheinakasaka
9
1.5k
パスキーのすべて ── 導入・UX設計・実装の紹介 / 20250213 パスキー開発者の集い
kuralab
3
860
GoとPHPのインターフェイスの違い
shimabox
2
200
Visual StudioのGitHub Copilotでいろいろやってみる
tomokusaba
1
190
データの整合性を保つ非同期処理アーキテクチャパターン / Async Architecture Patterns
mokuo
53
18k

Featured

See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.2k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
The Cult of Friendly URLs
andyhume
78
6.2k
Java REST API Framework Comparison - PWX 2021
mraible
29
8.4k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
Typedesign – Prime Four
hannesfritz
40
2.5k
4 Signs Your Business is Dying
shpigford
182
22k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.8k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
6
570

Transcript

  1. Airflow Data Workflow Management

  2. O que é? Execução de workflow de dados Escritos em

    Python Forma de DAGs Criado pelo Airbnb Open Source (Apache License) Desenvolvido em Python Flask, SQLAlchemy, Jinja, etc Alternativa ao Luigi do Spotify

  3. DAG Directed Acyclic Graphs Rede de jobs Possui agendamento próprio

    Código enxuto e fácil de entender quando criado

  4. DAG Airflow - Backfill

  5. DAG Airflow - Visualização de Grafos

  6. DAG Airflow - Detalhes

  7. Alguns processos feitos com Airflow Data warehousing: cleanse, organize, data

    quality check Growth analytics Experimentation: compute our A/B testing Email targeting Sessionization: compute clickstream Search: compute search ranking related metrics Data infrastructure maintenance: database scrapes, folder cleanup, applying data retention policies, … feitos pelo Airbnb

  8. meucandidato.me

  9. Blog: Telegram: Meetup: df.python.org.br @grupydf meetup.com/pt-BR/pythondf

  10. Email: [email protected] Blog: Telegram/Github: @gilsondev Twitter: gilsondev.in @gilsonfilho Em que

    posso ajudar?