https://speakerdeck.com/amadeusmagrabi 2020-04-18 06:54:31 -0400 Talk at the Open Data Science Conference East 2020. Abstract: Database indexes can make or break the performance of a database. Efficient indexes need to be tailored to the specific queries that are sent to a database, but since query patterns can vary a lot and change over time, it is often a painful process to manually manage indexes. In this session, I will talk about our data-driven approach to automatically estimate optimal indexes from log data of our MongoDB databases. You will learn how we use Google Cloud Functions to stream log data from Stackdriver to Google BigQuery, how we use BigQuery to scale our data analysis, and how we use Python’s Jupyter Notebooks to visualize and monitor our results. Talk at the Open Data Science Conference East 2020. Abstract: Database indexes can make or break the performance of a database. Efficient indexes need to be tailored to the specific queries that are sent to a database, but since query patterns can vary a lot and change over time, it is often a painful process to manually manage indexes. In this session, I will talk about our data-driven approach to automatically estimate optimal indexes from log data of our MongoDB databases. You will learn how we use Google Cloud Functions to stream log data from Stackdriver to Google BigQuery, how we use BigQuery to scale our data analysis, and how we use Python’s Jupyter Notebooks to visualize and monitor our results. Fri, 17 Apr 2020 00:00:00 -0400 https://speakerdeck.com/amadeusmagrabi/managing-database-indexes-a-data-driven-approach-amadeus-magrabi https://speakerdeck.com/amadeusmagrabi/managing-database-indexes-a-data-driven-approach-amadeus-magrabi