Pimp my Platform: Using AlloyDB with GKE

AlloyDB is GCP's new, fully-managed, PostgreSQL-compatible database service that recently went GA. It combines the benefits of the well-known (and loved) open-source DBMS PostgreSQL with those of Google managed services.
In this talk, Tobias will explain how AlloyDB works and the benefits it provides over a standalone PostgreSQL integration. Additionally, he will show how to integrate AlloyDB with applications running in Google Kubernetes Engine (GKE), according to Google best practices.
Tobias is a Cloud Developer and IT Consultant at Posedio GmbH. He is also a passionate backend and DevOps engineer and a certified Professional Google Cloud Developer.
Recently, he worked on migrating a customer's GKE workloads from CloudSQL to AlloyDB and is now looking forward to sharing his newfound expertise.

Transcript

2. TALK OVERVIEW 1.What is AlloyDB? 2.Why would I use AlloyDB? 3.How does AlloyDB work? 4.What is the AlloyDB Auth Proxy? 5.Demo - Integrating AlloyDB with GKE
3. WHAT IS ALLOYDB? 1
4. 4 ANSWER • “Classic” PostgreSQL with modified kernel • Provides standard PostgreSQL API • Utilises Google’s infrastructure • Optimised for transactional and analytical workloads “AlloyDB is a fully-managed, PostgreSQL-compatible database for demanding transactional and analytical workloads.”1 1. https://cloud.google.com/alloydb/docs/overview
5. WHY WOULD I USE ALLOYDB? 2
6. 6 KEY BENEFITS • More than 4x faster for transactional workloads • Up to 100x faster for analytical queries • Enables dynamic vertical & horizontal scaling • Automatic protection from zonal failures • Fast creation of read replicas • Fast restart recovery • Backups don’t impact db layer performance & resources • Managed service, 99.99% uptime SLA - including maintenance Over “vanilla” PostgreSQL
7. HOW DOES ALLOYDB WORK? 3
8. 8 DESIGN OVERVIEW • Compute & storage disaggregated • Shared data store • DB layer only writes logs • Dynamic vertical scalability • Dynamic horizontal scalability of replicas • Fast read replica creation • DB layer not impacted by Backups Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
9. 9 DESIGN OVERVIEW • Distributed storage layer • High transaction throughput through very fast WAL • Seamlessly scalable log processing service (LPS) • Asynchronous log record processing (once per zone) • Very fast system restarts • Sharded block storage • Zonal failure protection Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
10. 10 DESIGN OVERVIEW • Intelligent caching • In-memory buffer cache for all instances and LPS • Additionally ultra-fast block cache for instances (hardware component) • WAL record streams to keep read replica caches up to date Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
11. 11 A WRITE OPERATION • Statement issued by client • Cached data updated • WAL record prepared • WAL record saved synchronously on transaction commit • WAL record streamed to read replicas • WAL records applied asynchronously to referenced blocks Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
12. 12 A READ OPERATION • Query sent to any instance • Standard PostgreSQL query- parsing, planning & processing • Required data blocks in cache? • No => Request from LPS • Data in LPS buffer cache? • No => Load from block storage • Data up to date? • No => stalled until redo- processing block completed (rare) Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
13. 13 REPLICATION • Data partitioned into shards • Shard-replication across three zones • Shard handled by one LPS • LPS can handle multiple shards • Extremely fast re-assignment • Re-assignment invisible to db layer • Zonal failures completely handled by storage layer • Standby for primary instance possible (for high availability) Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
14. 14 CROSS REGION REPLICATION & BACKUPS • Readonly cluster replica in another region [Pre-GA] • Enables disaster recovery • Enables geographic load balancing • Can help decrease latency • “Promotion” possible • On-demand backups • Automated backups according to defined schedule • Continuous backups (PITR) [Pre-GA] Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage?hl=en
15. 15 SUPPORTED EXTENSIONS • Supports common extensions like • hypopg • pgaudit • pg_stat_statements • PostGIS • Provides product specific extensions like google_columnar_engine • Some extensions require setting certain database flags
16. 16 THE COLUMNAR ENGINE EXTENSION • Installed per default in template1 • Requires explicit activation via database flag • Keeps certain data in columnar format in memory • Enables columnar and hybrid execution plans • Uses ML to determine data to keep in columnar format • Significantly increases performance of OLAP & HTAP Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-columnar-engine?hl=en
17. 17 THE COLUMNAR ENGINE EXTENSION • Creates and uses column specific metadata • Uses ML to determine metadata and data format • Utilizes metadata and vector processing instructions (SIMD) • Increases analytical query performance by up to 100x as measured by Google based on the SSB (scale 10, 16 vCPU AlloyDB instance) Image Credits: Google1 1. https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-columnar-engine?hl=en
18. WHAT IS THE ALLOYDB AUTH PROXY? 4
19. 19 THE AUTH PROXY • Lightweight Go application • Encrypts data traffic • Enables IAM based authorisation • Deploy on VM or as sidecar • Could become SPOF on VM • As sidecar availability needs to be assured • Application-authorisation through workload identity And what to mind using it in GKE
20. DEMO - INTEGRATING ALLOYDB WITH GKE 5
21. THANK YOU! POSEDIO GMBH +43 660 9154419 contact@posedio.com Weyringergasse 1-3, 1040 Wien, 
 Millenium Park 4, 6980 Lustenau www.posedio.com