Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Aurora Serverless / Goran Opacic

Aurora Serverless / Goran Opacic

Aurora Serverless
Speaker: Goran Opacic
Nov 5, Belgrade
AWS User Group Belgrade
Meetup #3 DATABASES

Aa335d9f4a299100625e6f11efd33516?s=128

AWS User Group Belgrade

November 05, 2019
Tweet

Transcript

  1. Aurora Serverless Goran Opacic madabout.cloud AWS User Group Belgrade Nov

    5 2019 Meetup #3
  2. AURORA –AWS MANAGED –Redundant –Storage autoscaling –MySQL & PostgreSQL

  3. SERVERLESS –BILLING – PAY PER USE –SCALE TO/FROM ZERO –FORGET

    ABOUT THE INFRASTRUCTURE –DISTRIBUTED –UNLIMITED –EVENT SOURCED
  4. AWS SERVERLESS –AWS Lambda FaaS –EventBridge –SQS –SNS –S3 –…

  5. AURORA SERVERLESS DATABASE –SQL –AUTOSCALING - ZERO TO MAX AND

    BACK –NO PROVISIONING –BILLING
  6. MICROSERVICES – Database Per Service IN PRACTICE

  7. IN PRACTICE –REPORTING –UNPREDICTABLE WORKLOADS –CYCLICAL WORKLOADS –INTERMITTENT WORKLOADS –MULTI

    TENANT – SINGLE TENANT –DEVELOPMENT –TEST
  8. NON- SERVERLESS AURORA © 2018, Amazon Web Services, Inc. or

    its affiliates. All rights reserved. Availability zone 1 Region Availability zone 2 Availability zone 3 Scale-out, distributed, multi-tenant design . . . Purpose-built log-structured, distributed storage Shared storage volume Storage nodes with SSDs Master SQL Transactions Caching Reader 5 Reader 9 SQL Transactions Caching SQL Transactions Caching Reader 1 SQL Transactions Caching Reader 8 SQL Transactions Caching Reader 15 SQL Transactions Caching … … … 10GiB stripes Master and up to 15 Readers all point to the same storage 6-way replication, 2 per AZ Quorums survive AZ+1 failure
  9. TECH – 1 ACU = 1 VCPU + 2GB RAM

    – Lower BOUND – ZERO or 1 ACU – UPPER BOUND – 256 ACU = 64 VCPU + 488 GB RAM – BUFFER – CONNECTION POOL
  10. None
  11. SERVER SWITCH – SERVER POOL – 5 sec to SWITCH

    servers – Take server from pool – Set server to read from last good storage point – Copy buffer from LIVE server – Find “SAFE PLACE to SCALE” – FORCE Scaling on TIME-OUT – FREEZE workload and transfer session state – Route traffic to new server
  12. None
  13. None
  14. None
  15. AUTOSCALING – SCALING UP PARAMS – 3Min – 90% connection

    utilization OR 70% CPU utilization – SERVER SWITCH – SCALING DOWN PARAMS – 15Min – 40% connection utilization OR 30% CPU utilization – TURN OFF – default 5 min with no connections (optional) – WAKE-UP – to previous ACU, not 1 ACU or LOW BOUND ACU
  16. None
  17. BILLING – US East – $0.06 per acu hour =

    ~44 usd – $0.10 per GB/month – $0.20 per 1M IO Requests – Pay per SECOND – MINIMUM 1 min – ZERO ACU = pay for Storage ONLY – db.t2.small PRICE diff +13.8 usd – BUT because of AZ it is actually cheaper -16 usd
  18. TO DO #1 – COLD START ~30 sec – 5.6

    MySQL, 10.7 PostgreSQL only – NO READ REPLICAS –VERTICAL SCALING ONLY – NO SHARDS – NON Configurable scaling params – SAFE place to scale – not easy to find – Long-running queries or transactions are in progress – Temporary tables or table locks are in use
  19. TO DO #2 – BUFFER copy – 5 sec to

    5 min or more – No REPLICATION from/to other DBs – VPC ONLY – LIMITED Number of parameters – DB TCP port is fixed /3306 mysql – Loading data from S3 Not supported – NO Database Cloning – NO Backtrack
  20. DEVELOPERS & DEVOPS – Connection timeout to 60 sec –

    Close your connections – Don’t feed connection pools – set min conn No to ZERO – 1 ACU ~90 MAX connections – 256 ACU ~6000 MAX connections – Use ProxySQL or pgBouncer – BACKUP to SNAPSHOT after 7 DAYS – longer restore – CONFIG CHANGES apply immediately (instance switch) – AUTOMATIC Upgrades & Security fixes – Maintenance windows don't apply to Aurora Serverless – ENCRYPTION is always ON
  21. DATA API – REST API – JSON – EXTRA PROXY

    – IAM ROLES AUTH – COMING TO non serverless Aurora – LAMBDA - NO VPC – AWS CLI
  22. DATA API # aws rds-data execute-statement --secret-arn {SECRETARN} --resource-arn {DBCLUSTERARN}

    --schema "mysql" --sql "SELECT * FROM demotable" --database "demodb" { "records": [ [ { "longValue": 1 }, { "stringValue": "test11111" } ] , ............ ] , "numberOfRecordsUpdated": 0 }
  23. CATCH-UP – Deep dive on Amazon Aurora https://youtu.be/VrWH518sObk – Buffer

    pools https://youtu.be/uZ3-aeFYE5k – DATA API blog post https://madabout.cloud/2019/09/01/aws-data- api-for-amazon-aurora-serverless/