Will the Era of Specialized Databases be Over?

AWS has always been a strong proponent of “the right tool for the right job”, specially for databases, as the then-CEO Jeff Bezos published in 2018 in his annual letter to shareholders.

As a result of this philosophy, AWS has as of today 11 types of database types (some of them covering multiple technologies/variants).

However, latest trends may be pointing to a weakening of this strategy. Consolidation on a smaller number of database technologies seems to be starting. With a clear protagonist in the crime scene.

AWS Data Hero Álvaro Hernández will try to be the Oracle (no pun intended) that disentangles the future of AWS Databases.

This talk was delivered at the AWS Community Day Spain 2024 (https://2024.awscommunity.es/en/speakers/alvaro-hernandez).

Transcript

1. Will the era of specialized databases be over? Will the Era of Specialized Databases be Over? Álvaro Hernández @ahachete
2. Will the era of specialized databases be over? @ahachete ` whoami ` Alvaro Hernandez <aht@ongres.com> aht.es • Founder & CEO, OnGres • 20+ years Postgres user and DBA • Mostly doing R&D to create new, innovative software on Postgres • More than 140 tech talks • AWS Data Hero • Founder and President of the NPO Fundación PostgreSQL
3. Will the era of specialized databases be over? @ahachete How it all started
4. Will the era of specialized databases be over? @ahachete
5. Will the era of specialized databases be over? @ahachete Jeff Bezos – 2018 Letter to Shareholders We’re also optimistic about specialized databases for specialized workloads. Over the past 20 to 30 years, companies ran most of their workloads using relational databases. [...] data set sizes were often small enough [...] many applications are storing very large amounts of data – terabytes and petabytes. And the requirements for apps have changed. [...] the right tool for the right job saves money and gets your product to market faster. https://www.aboutamazon.com/news/company-news/2018-letter-to-shareholders
6. Will the era of specialized databases be over? @ahachete AWS: specialized databases
7. Will the era of specialized databases be over? @ahachete AWS: specialized databases
8. Will the era of specialized databases be over? @ahachete The Right Tool for the Right Job
9. Will the era of specialized databases be over? @ahachete The Right Tool for the Right Job? Yay • Get the best performance • Optimal mapping app:database Nay • Learn many databases! • Learn how to do many types of Ops!
10. Will the era of specialized databases be over? Databases evolution at AWS: do you spot the trend?
11. Will the era of specialized databases be over? Aurora • 2014: MySQL • 2017: Postgres Aurora Brief Launch Timeline Aurora Serverless v1 • 2018: MySQL • 2019: Postgres Aurora Serverless v2 • 2022: MySQL & Postgres
12. Will the era of specialized databases be over? 2021: Adding SQL Server on top of Postgres https://babelfishpg.org/
13. Will the era of specialized databases be over? 2023: Amazon Aurora Limitless (PostgreSQL)
14. Will the era of specialized databases be over? Quantum Ledger Database: RIP
15. Will the era of specialized databases be over? Quantum Ledger Database: RIP https://aws.amazon.com/blogs/database/migrate-an-amazon-qldb-ledger-to-amazon-aurora-postgresql/
16. Will the era of specialized databases be over? Did you spot the trend? Is it limited to relational databases?
17. Will the era of specialized databases be over? Amazon DocumentDB (with MongoDB compatibility) https://news.ycombinator.com/item?id=18870397
18. Will the era of specialized databases be over? Brief departure (shameless plug) (but please bear with me)
19. Will the era of specialized databases be over? Dyna53.io
20. Will the era of specialized databases be over? Dyna53.io
21. Will the era of specialized databases be over? @ahachete Dyna53.io → Dynorora ???? • What if Dyna53 writes to RDS/Aurora instead? • After all, Postgres has very good JSON support… https://speakerdeck.com/ongres/can-postgres-scale-like-dynamodb
22. Will the era of specialized databases be over? @ahachete In reality… DynamoDB works on top of a RDBMS! Would it be MySQL if it were redesigned today?
23. Will the era of specialized databases be over? So why do we see this trend? Because AWS listens to customers
24. Will the era of specialized databases be over? Which database has been growing in this decade?
25. Will the era of specialized databases be over? What makes Postgres so unique? https://www.youtube.com/watch?v=EMldOiiG1Ko Extensions, Extensions, Extensions
26. Will the era of specialized databases be over? Postgres as a multi-purpose db https://www.timescale.com/blog/postgres-the-birdhorse-of-databases/
27. Will the era of specialized databases be over? Postgres is THE database for AI pgvector, pgvectorscale, pgcompute, pg_vectorize, … https://www.linkedin.com/pulse/generative-ai-augmented-recovery-generation-rag-its-data-sampaio-oll2f/
28. Will the era of specialized databases be over? Let me try to be the oracle
29. Will the era of specialized databases be over? AWS will eventually create a new Database to fill the missing gap: A Distributed Database which will be Postgres / Postgres-compatible
30. Will the era of specialized databases be over? Thank you!