Meetup 2 2019 - Gustavo Arjones - Arquitecturas de Big Data

Transcript

1. PROYECTOS CORPORATIVOS DE DATOS DESAFÍOS PARA ARMAR UNA PLATAFORMA DE BIG DATA GUSTAVO ARJONES | GUSTAVO@ARJONES.NET | @ARJONES | LINKEDIN/ARJONES
2. GUSTAVO ARJONES gustavo@arjones.net | @arjones
3. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación gustavo@arjones.net | @arjones
4. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal gustavo@arjones.net | @arjones
5. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal > Ex-CTO/co-founder de Socialmetrix gustavo@arjones.net | @arjones
6. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal > Ex-CTO/co-founder de Socialmetrix gustavo@arjones.net | @arjones
7. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal > Ex-CTO/co-founder de Socialmetrix > Data Geek gustavo@arjones.net | @arjones
8. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal > Ex-CTO/co-founder de Socialmetrix > Data Geek > Armado/implementación de arquitecturas de Big Data gustavo@arjones.net | @arjones
9. GUSTAVO ARJONES > MBA Marketing; Licenciado Ciencias de la Computación > CTO de Extendeal > Ex-CTO/co-founder de Socialmetrix > Data Geek > Armado/implementación de arquitecturas de Big Data > Machine Learning, DevOps, etc gustavo@arjones.net | @arjones
10. AL PRINCIPIO, HABÍA OLTP gustavo@arjones.net | @arjones
11. DATABASE REPLICA gustavo@arjones.net | @arjones
12. ANALYTICS? OLAP + ETL gustavo@arjones.net | @arjones
13. ETL COMPLEJOS + DATOS NO-ESTRUCTURADOS gustavo@arjones.net | @arjones
14. HADOOP gustavo@arjones.net | @arjones
15. LENTO Y DIFÍCIL OPERAR PERO "ANDA" gustavo@arjones.net | @arjones
16. SE PONE MÁS COMPLEJO gustavo@arjones.net | @arjones
17. ADEMÁS SEGUÍMOS EN BATCH Y NECESITAMOS DE SOLUCIONES REALTIME gustavo@arjones.net | @arjones
18. LAMBDA ARCHITECTURE BATCH (HADOOP) STREAM (STORM) gustavo@arjones.net | @arjones
20. TURNING THE DATABASE INSIDE-OUT gustavo@arjones.net | @arjones
21. TURNING THE DATABASE INSIDE-OUT > Turning the database inside-out with Apache Samza gustavo@arjones.net | @arjones
22. TURNING THE DATABASE INSIDE-OUT > Turning the database inside-out with Apache Samza > Apache Kafka, Samza, and the Unix Philosophy of Distributed Data gustavo@arjones.net | @arjones
23. TURNING THE DATABASE INSIDE-OUT > Turning the database inside-out with Apache Samza > Apache Kafka, Samza, and the Unix Philosophy of Distributed Data > The Log: What every software engineer should know about real- time data's unifying abstraction gustavo@arjones.net | @arjones
24. KAPPA ARCHITECTURE http://www.kappa-architecture.com
25. DESIGNING DATA- INTENSIVE APPLICATIONS https://www.safaribooksonline.com/library/view/designing-data-intensive- applications/9781491903063/
27. EVENT SOURCING https://martinfowler.com/eaaDev/EventSourcing.html
28. TRANSACTIONS? ROLLBACK / COMMIT gustavo@arjones.net | @arjones
29. WRITE-AHEAD LOG gustavo@arjones.net | @arjones
30. SOURCE OF TRUTH + ML TRAINING gustavo@arjones.net | @arjones
32. FOSS FRAMEWORKS & TOOLING gustavo@arjones.net | @arjones
33. FOSS FRAMEWORKS & TOOLING > Apache Beam gustavo@arjones.net | @arjones
34. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink gustavo@arjones.net | @arjones
35. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink > Apache Kafka | Apache Pulsar gustavo@arjones.net | @arjones
36. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink > Apache Kafka | Apache Pulsar > Streamsets | Apache NiFi gustavo@arjones.net | @arjones
37. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink > Apache Kafka | Apache Pulsar > Streamsets | Apache NiFi > Debezium gustavo@arjones.net | @arjones
38. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink > Apache Kafka | Apache Pulsar > Streamsets | Apache NiFi > Debezium > Presto gustavo@arjones.net | @arjones
39. FOSS FRAMEWORKS & TOOLING > Apache Beam > Apache Spark | Apache Flink > Apache Kafka | Apache Pulsar > Streamsets | Apache NiFi > Debezium > Presto > Apache Airflow gustavo@arjones.net | @arjones
40. ON-PREM VS. CLOUD APACHE KAFKA, PULSAR VS. AWS KINESIS, AZURE EVENTHUB, ETC gustavo@arjones.net | @arjones
41. STREAMING SYSTEMS THE WHAT, WHERE, WHEN, AND HOW OF LARGE-SCALE DATA PROCESSING https://www.safaribooksonline.com/library/view/streaming-systems/ 9781491983867/
42. DO YOU REALLY NEED BIG DATA? POSTGRES + POSTGIS + TIMESCALE EXTENSION = ❤ Reading suggestions: Reasons to Fall in Love for Postgres | Postgres: Not Your Grandfather’s RDBMS
43. POSTGRES + SOURCE OF TRUTH + ML TRAINING gustavo@arjones.net | @arjones
44. HAPPINESS gustavo@arjones.net | @arjones
45. GRACIAS | OBRIGADO GUSTAVO ARJONES | GUSTAVO@ARJONES.NET | @ARJONES | LINKEDIN/ARJONES gustavo@arjones.net | @arjones