Architecting Data Lake/Mart with GCP

Presented at Google Cloud Next Extended KL April 2019

Transcript

1. Architecting Data Lake/Data Mart using Google Cloud Platform (GCP) Hafeez Nazri Head of Data Engineering, Media Prima Digital 27th April 2019
2. 01 Introduction to Media Prima Digital & Achievements 02 Deep Dive into GCP Components & Big Query 03 Use Case – Architecting Data Lakes/Mart Using GCP 04 Q &A Agenda
3. 01 Introduction to Media Prima Digital & Achievements #nextxkl #next19extended
4. Quick demographic check! •  How many have already embark on Data Science, Big Data or Machine Learning project? •  How many have utilize GCP on production? Or other Cloud Platform – AWS, Azure, Ali Cloud? •  How many knows about Big Query or any other GCP managed services? •  How many didn’t know what I’m talking about? J
5. Media Prima by the numbers •  4 TV Network •  4 Radio Sites •  3 News Sites •  >15 Digital Portal •  >15 Digital Apps Average 1P byte data processed every month!
7. Media Prima journey to become Data-driven Company
12. 02 Deep Dive into GCP Components & Big Query #nextxkl #next19extended
13. 163 Zetabytes of Data by 2025 (1 trillion gigabytes) (IDC)
17. For questions, contact us at: next18extended@gmail.com | #next18extended
23. #nextxkl #next18extended GCP Storage Portfolio
24. What is Big Query? •  GCP’s Enterprise Data Warehouse for Analytics •  Petabyte scale storage and analysis •  Real-time analytics on streaming data •  Fully-managed and serverless •  Support standard SQL, JDBC & ODBC •  Power of Big Query: having petabyte scale of data that is fully managed and serverless
25. Why our team choose to use Big Query? •  Easy ingestion – from other 3rd party systems into Data Warehouse •  You can verify and query your data right away using familiar SQL statement & super fast! •  Inexpensive pricing •  Work well with our ETL tools – data ingestion
28. 03 Use Case – Architecting Data Lakes/Mart using GCP #nextxkl #next19extended
29. •  Large storage repository for vast amounts of data •  Advanced concept of Data Warehousing •  Key benefits – data could be normalized and enriched and allow metadata extraction, format conversion & metadata stitching Enterprise Data Lake (EDL) Concept
30. Data Lake High Level Architecture with GCP
32. DEMO!
33. Query Editor Project ID Datasets Query Results
36. 04 Q&A #nextxkl #next19extended
37. #nextxkl #next19extended Thanks! We’re hiring! hafeez.nazri@mediaprima.com.my