Aurora Serverless / Goran Opacic

Aurora Serverless
Speaker: Goran Opacic
Nov 5, Belgrade
AWS User Group Belgrade
Meetup #3 DATABASES

Transcript

1. Aurora Serverless Goran Opacic madabout.cloud AWS User Group Belgrade Nov 5 2019 Meetup #3
2. AURORA –AWS MANAGED –Redundant –Storage autoscaling –MySQL & PostgreSQL
3. SERVERLESS –BILLING – PAY PER USE –SCALE TO/FROM ZERO –FORGET ABOUT THE INFRASTRUCTURE –DISTRIBUTED –UNLIMITED –EVENT SOURCED
4. AWS SERVERLESS –AWS Lambda FaaS –EventBridge –SQS –SNS –S3 –…
5. AURORA SERVERLESS DATABASE –SQL –AUTOSCALING - ZERO TO MAX AND BACK –NO PROVISIONING –BILLING
6. MICROSERVICES – Database Per Service IN PRACTICE
7. IN PRACTICE –REPORTING –UNPREDICTABLE WORKLOADS –CYCLICAL WORKLOADS –INTERMITTENT WORKLOADS –MULTI TENANT – SINGLE TENANT –DEVELOPMENT –TEST
8. NON- SERVERLESS AURORA © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Availability zone 1 Region Availability zone 2 Availability zone 3 Scale-out, distributed, multi-tenant design . . . Purpose-built log-structured, distributed storage Shared storage volume Storage nodes with SSDs Master SQL Transactions Caching Reader 5 Reader 9 SQL Transactions Caching SQL Transactions Caching Reader 1 SQL Transactions Caching Reader 8 SQL Transactions Caching Reader 15 SQL Transactions Caching … … … 10GiB stripes Master and up to 15 Readers all point to the same storage 6-way replication, 2 per AZ Quorums survive AZ+1 failure
9. TECH – 1 ACU = 1 VCPU + 2GB RAM – Lower BOUND – ZERO or 1 ACU – UPPER BOUND – 256 ACU = 64 VCPU + 488 GB RAM – BUFFER – CONNECTION POOL
11. SERVER SWITCH – SERVER POOL – 5 sec to SWITCH servers – Take server from pool – Set server to read from last good storage point – Copy buffer from LIVE server – Find “SAFE PLACE to SCALE” – FORCE Scaling on TIME-OUT – FREEZE workload and transfer session state – Route traffic to new server
15. AUTOSCALING – SCALING UP PARAMS – 3Min – 90% connection utilization OR 70% CPU utilization – SERVER SWITCH – SCALING DOWN PARAMS – 15Min – 40% connection utilization OR 30% CPU utilization – TURN OFF – default 5 min with no connections (optional) – WAKE-UP – to previous ACU, not 1 ACU or LOW BOUND ACU
17. BILLING – US East – $0.06 per acu hour = ~44 usd – $0.10 per GB/month – $0.20 per 1M IO Requests – Pay per SECOND – MINIMUM 1 min – ZERO ACU = pay for Storage ONLY – db.t2.small PRICE diff +13.8 usd – BUT because of AZ it is actually cheaper -16 usd
18. TO DO #1 – COLD START ~30 sec – 5.6 MySQL, 10.7 PostgreSQL only – NO READ REPLICAS –VERTICAL SCALING ONLY – NO SHARDS – NON Configurable scaling params – SAFE place to scale – not easy to find – Long-running queries or transactions are in progress – Temporary tables or table locks are in use
19. TO DO #2 – BUFFER copy – 5 sec to 5 min or more – No REPLICATION from/to other DBs – VPC ONLY – LIMITED Number of parameters – DB TCP port is fixed /3306 mysql – Loading data from S3 Not supported – NO Database Cloning – NO Backtrack
20. DEVELOPERS & DEVOPS – Connection timeout to 60 sec – Close your connections – Don’t feed connection pools – set min conn No to ZERO – 1 ACU ~90 MAX connections – 256 ACU ~6000 MAX connections – Use ProxySQL or pgBouncer – BACKUP to SNAPSHOT after 7 DAYS – longer restore – CONFIG CHANGES apply immediately (instance switch) – AUTOMATIC Upgrades & Security fixes – Maintenance windows don't apply to Aurora Serverless – ENCRYPTION is always ON
21. DATA API – REST API – JSON – EXTRA PROXY – IAM ROLES AUTH – COMING TO non serverless Aurora – LAMBDA - NO VPC – AWS CLI
22. DATA API # aws rds-data execute-statement --secret-arn {SECRETARN} --resource-arn {DBCLUSTERARN} --schema "mysql" --sql "SELECT * FROM demotable" --database "demodb" { "records": [ [ { "longValue": 1 }, { "stringValue": "test11111" } ] , ............ ] , "numberOfRecordsUpdated": 0 }
23. CATCH-UP – Deep dive on Amazon Aurora https://youtu.be/VrWH518sObk – Buffer pools https://youtu.be/uZ3-aeFYE5k – DATA API blog post https://madabout.cloud/2019/09/01/aws-data- api-for-amazon-aurora-serverless/