Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Resilient, Mission-Critical Applications with Azure Database for Postgres | Open Azure Day 2020 | Sridhar Ranganathan

Building Resilient, Mission-Critical Applications with Azure Database for Postgres | Open Azure Day 2020 | Sridhar Ranganathan

Are you looking to deploy your mission-critical application to a managed Postgres database service that is secure, reliable, available—as well as performant and scalable? If yes, this session is for you. You will learn about the new Flexible Server option in Azure Database for PostgreSQL that gives you zone-redundant HA (my favorite topic) as well as more control of your database configuration, networking options, and maintenance window. You will also learn how Citus—an open source extension to Postgres that transforms Postgres into a distributed database—is changing what’s possible for data-intensive applications. We’ll walk through some interesting applications that are already scaling out with Hyperscale (Citus), a built-in deployment option in Azure Database for PostgreSQL. And we’ll explore how Kubernetes has opened the door to running Postgres on Azure in hybrid scenarios, too.

143117954187136b825331f24da0e201?s=128

Azure Postgres

November 18, 2020
Tweet

Transcript

  1. Building resilient, mission-critical applications with Azure Database for PostgreSQL Sridhar

    Ranganathan, Principal Program Manager, Azure Database for PostgreSQL @sridrang / @AzureDBPostgres / @citusdata
  2. Agenda • Let us talk about PostgreSQL • PostgreSQL &

    Azure • Building applications with Azure Database for PostgreSQL • Making Azure the best destination for PostgreSQL
  3. None
  4. Why PostgreSQL? Solves a multitude of use cases • Rich

    indexing, text search Rich feature set • Extensible & tooling Open source Large developer community • Enterprise-ready • Zero data loss Proven resilience & stability Thousands of mission critical workloads
  5. PostgreSQL is more popular than ever Ranked 2017 & 2018

    DBMS of the Year by DB-Engines One of most loved & wanted databases in Stack Overflow 2019 Developer Survey Microsoft PostgreSQL https://insights.stackoverflow.com/survey/2019?utm_source=so-owned&utm_medium=blog&utm_campaign=dev-survey- 2019&utm_content=launch-blog https://db-engines.com/en/blog_post/76
  6. Microsoft acquired Citus Data in 2019

  7. Microsoft Azure Welcomes PostgreSQL Committers aka.ms/blog-postgres-committers

  8. Azure builds upon PostgreSQL PostgreSQL is built for the cloud

    Management Automation Extension support Global reach Security Scale up & out High Availability Compliance Intelligent performance Ecosystem integration
  9. Azure Database for PostgreSQL is fully-managed, intelligent, and scalable PostgreSQL

    aka.ms/azure-postgres
  10. Build innovative applications with Azure PostgreSQL Modern apps built with

    Azure Kubernetes Service (AKS) Engage cloud native developers with the flexibility of AKS and extensibility of PostgreSQL Enable real-time operational analytics apps Converge transactional and analytical data stores Build geospatial enabled Apps Leverage PostGIS, the world’s most popular OS geospatial solution to deliver location- aware experiences Combine time series and relational data Power just-in-time decision-making for event driven applications Using community PostgreSQL, and not a fork. No vendor lock-in.
  11. Easy to build with a fully-managed service on Azure Automatic

    updates Scale in seconds Built-in compliance and security Azure IP Advantage
  12. You want to migrate to cloud and/or modernize your application…

  13. Azure Database for PostgreSQL – Flexible Server (Preview)

  14. Flexible Server to run your enterprise workloads Maximum control for

    your databases VNET server parameters maintenance windows
  15. Flexible Server to run your enterprise workloads Build resilient apps

    across availability zones • Zone redundant High Availability zero data loss • Co-locate Maximum control for your databases VNET server parameters maintenance windows
  16. Flexible Server to run your enterprise workloads Build resilient apps

    across availability zones High Availability zero data loss • Co-locate Maximum control for your databases VNET server parameters maintenance windows end-to-end community PostgreSQL cost optimization Simplified developer experience aka.ms/postgres-flexible-blog
  17. North Europe Region – East US West US 2 AZ1

    AZ2 AZ3 Zone-redundant backup storage Flexible Server PostgreSQL: Linux & PostgreSQL Flexible Server Availability Zone 1 Availability Zone 2 Availability Zone 3 Premium Storage Data, Logs Linux VM PostgreSQL
  18. North Europe Region – East US West US 2 AZ1

    AZ2 AZ3 Zone-redundant backup storage Flexible Server PostgreSQL: Application co-location Flexible Server Availability Zone 1 Availability Zone 2 Availability Zone 3 Azure VM AKS App Service Premium Storage Data, Logs Linux VM PostgreSQL
  19. Region – East US Zone Redundant HA - Architecture Zone-redundant

    backup storage Availability Zone 1 Flexible Server (Primary) PostgreSQL Linux VM Premium Storage Data, WAL Azure VM AKS App Service Availability Zone 2 Flexible Server (Standby) Linux VM Premium Storage Data, Logs Synchronous Replication Azure VM AKS App Service PostgreSQL WAL archival Daily data backups
  20. Region – East US Zone Redundant HA – Automatic Failover

    Zone-redundant backup storage Availability Zone 1 Flexible Server (Standby) PostgreSQL Linux VM Premium Storage Data, WAL Azure VM AKS App Service Availability Zone 2 Flexible Server (Primary) Linux VM Premium Storage Data, WAL Synchronous Replication Azure VM AKS App Service PostgreSQL WAL archival Daily data backups aka.ms/flexible-pg-ha
  21. ž Hhh Security Network access - Public access (allowed IP

    addresses) - Firewall rules - Private access (VNet integration ) - Delegated subnet for Flexible server - Restricted access Client connectivity - TLS 1.2+ enforced aka.ms/flexible-pg-networking
  22. Let us build an application with Azure Database for PostgreSQL

  23. Build your application with Flexible Server 1) AKS – Containerized

    applications 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions 1 2 4 5 6 7 3 Zone redundant HA – Flexible server Azure Kubernetes Service (AKS)
  24. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  25. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  26. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  27. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  28. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  29. Build your application with Flexible Server 1) AKS – Containerized

    application 2) Flexible Server (HA) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server
  30. If you want to scale- out to 100s of nodes

    and handle TiB and PiB of data…
  31. Azure Database for PostgreSQL Hyperscale (Citus)

  32. Citus Data was founded 9 years ago to change what’s

    possible with Postgres
  33. Scaling out your Postgres database horizontally Each node has PostgreSQL

    with Citus installed. Each shard [ ] is 1 PostgreSQL table. Coordinator sends DMS/DDL queries to selective worker nodes. Coordinator node
  34. Azure PostgreSQL for PostgreSQL Hyperscale (Citus) • PostgreSQL database with

    data and queries distributed across nodes to provide unlimited horizontal scalability • Shard your PostgreSQL database across multiple nodes to give your application more memory, compute, and • Powered by the open source Citus extension and managed by Azure
  35. aka.ms/citus

  36. Scale-out your application with Hyperscale (Citus) scale out across 100s

    of nodes Scale out horizontally both transactions and analytics in one database Simplified infrastructure Blazing performance sub-second responses across billions of rows Stay current with PostgreSQL innovations open source extension
  37. Building applications with Hyperscale (Citus)

  38. 1) AKS – Containerized applications 2) Flexible Server (HA) –

    PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring 1 2 4 5 6 7 3 Azure Databricks Notification Hubs Browser Azure Cache for Redis Application Insights AKS Azure Functions Zone redundant HA – Flexible server If you want to scale-out your previous built application… Content Delivery Network
  39. Easily scale-out with Hyperscale (Citus) 1) AKS – Containerized OLTP

    applications 2) Hyperscale (Citus) – PostgreSQL database 3) Azure Cache – Session states, asset caching 4) Azure Functions – Alerts 5) Notification Hubs – Push notice 6) Azure Databricks – ML analytics and deeper insights 7) Application Insights – Performance monitoring Azure Databricks Notification Hubs Browser Content Delivery Network Azure Cache for Redis Application Insights AKS Azure Functions 1 2 4 5 6 7 3 Hyperscale (Citus)
  40. 4 4 3 5 2 1 6 Notification Hubs Devices

    Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Building Real-time analytics with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Hyperscale (Citus) in Azure Database for PostgreSQL
  41. 4 4 3 5 2 1 6 Building Real-time analytics

    with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Hyperscale (Citus) in Azure Database for PostgreSQL
  42. 4 4 3 5 2 1 6 Building Real-time analytics

    with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Hyperscale (Citus) in Azure Database for PostgreSQL
  43. 4 4 3 5 2 1 6 Building Real-time analytics

    with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Hyperscale (Citus) in Azure Database for PostgreSQL
  44. 4 4 3 5 2 1 6 Building Real-time analytics

    with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Hyperscale (Citus) in Azure Database for PostgreSQL
  45. 4 4 3 5 2 1 6 Building Real-time analytics

    with Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Event Hubs App Service/AKS Hyperscale (Citus) in Azure Database for PostgreSQL
  46. 4 4 3 5 2 1 6 Real-time analytics with

    Hyperscale (Citus) 1) Event Hubs - Stream millions of events 2) Hyperscale (Citus) – ingest raw transactional events 3) Hyperscale (Citus) – perform incremental rollups 4) Azure Databricks – transform and analyze from both raw/structured 5) AKS – Provide insights to users 6) Push notifications Notification Hubs Devices Raw Events Azure Databricks Scheduled process Browser Aggregations Hyperscale (Citus) in Azure Database for PostgreSQL Event Hubs App Service/AKS aka.ms/hyperscale-citus-real-time-tutorial
  47. Microsoft Windows team relies on Citus and Postgres (on Azure)

    for mission- critical shiproom decisions “Distributed PostgreSQL is a game changer. We can support more than 6M queries every day, on 2 PB of data. With Citus, most of our queries respond in less than 0.2 seconds.” aka.ms/blog-petabyte-scale-analytics
  48. Citus helps ASB onboard customers 20x faster “After migrating to

    Citus, we can onboard Vonto customers 20X faster, in 2 minutes vs. the 40+ minutes it used to take. And with the launch of Hyperscale (Citus) on Azure Database for PostgreSQL, we are excited to see what we can build next on Azure.”
  49. 10s–1,000s of apps Diverse infrastructure Multi-cloud IoT devices Edge Datacenters

    Branch offices Hosters OEM hardware VMs Containers Databases Serverless
  50. Now, if you want to run fully managed Azure Database

    for PostgreSQL in your infrastructure...
  51. Azure Arc enabled PostgreSQL Hyperscale - part of Azure Arc

    enabled data services
  52. Azure IoT Any edge device Azure Arc Any datacenter, any

    cloud Integrated systems Azure Stack Microsoft Azure Management | Security + Identity | App + Data Services | Dev Tools + DevOps
  53. Unmatched security Azure Policy Role-based Access Control Elastic scale Deploy

    in seconds Scale up, scale out Automation at scale Always current Automated updates Hyperscale (Citus) on-premises Cloud billing Cloud billing on- premises Cost efficiency Unified management Single view for on-prem and clouds Use familiar tools Any hardware, any Kubernetes Connected or Disconnected Azure Arc enabled PostgreSQL Hyperscale Bring Azure data services to on-premises, multi-cloud (Azure, AWS, GCP), and edge with Azure Arc aka.ms/azurearcdata
  54. Making Azure the best destination for PostgreSQL

  55. Multiple deployment modes to meet your application needs Flexible Server

    (Preview) Control, Flexibility, HA, and simplified developer experience PostgreSQL Hyperscale – Azure Arc Run Hyperscale (Citus) workloads in your infrastructure Single Server Fully managed PostgreSQL service Hyperscale (Citus) Distributed database that provides unlimited scalability aka.ms/azure-postgres
  56. Open Source Proven resilience & stability Rich feature set Identity,

    security, management, and compliance Available in Microsoft Azure
  57. Open Source Proven resilience & stability Rich feature set Fully

    managed Intelligent Performance Highly scalable High Availability Maximum control Identity, security, management, and compliance Available in Microsoft Azure
  58. Open Source Proven resilience & stability Rich feature set Fully

    managed Intelligent Performance Highly scalable High Availability Maximum control Integrated with Azure data ecosystem Identity, security, management, and compliance Available in Microsoft Azure
  59. Identity, security, management, and compliance All Available in Microsoft Azure

    Best of open source community along with the manageability and integration benefits of Azure Open Source Proven resilience & stability Rich feature set Fully managed Intelligent Performance Highly scalable High Availability Maximum control Integrated with Azure data ecosystem
  60. Identity, security, management, and compliance All Available in Microsoft Azure

    Best of open source community along with the manageability and integration benefits of Azure Open Source Proven resilience & stability Rich feature set Fully managed Intelligent Performance Highly scalable High Availability Maximum control Integrated with Azure data ecosystem Extend to non-Azure architectures using Azure Arc
  61. Citus Newsletter aka.ms/citus-newsletter October 2020 Next step: Subscribe

  62. Additional resources Azure service page: aka.ms/azure-postgres aka.ms/postgres-cloud9 aka.ms/postgres-flexible-blog aka.ms/postgres-uservoice Email

    us: AskAzureDBforPostgreSQL@service.microsoft.com
  63. thank you merci grazie danke ध"यवाद ευχαριστώ gracias ధన#$ా&'ల) 谢谢

    @sridrang @AzureDBPostgres @citusdata ந"றி AskAzureDBforPostgreSQL@service.microsoft.com
  64. BACKUP SLIDES

  65. Single Server Hyperscale (Citus) Worry-free PostgreSQL in the cloud with

    an architecture built to scale out Example use cases • Scaling PostgreSQL multi- tenant, SaaS applications • Real-time operational analytics • Building high throughput transactional apps Fully-managed, single- node PostgreSQL database service Example use cases • Transactional and operational analytics workloads • Apps requiring JSON, geospatial support, or full- text search • Cloud-native apps built with modern frameworks PostgreSQL deployment options in Azure Flexible Server Maximum control for your database with a simplified developer experience Example use cases • A new simplified architecture • High-performance apps utilizing zone co-location for low latency • Apps built on the latest Postgres version Arc Hyperscale (Citus) Run PostgreSQL Hyperscale in your infrastructure Example use cases • Scaling PostgreSQL multi- tenant, SaaS applications • Real-time operational analytics • Building high throughput transactional apps
  66. Speaker Sridhar Ranganathan, Principal PM Azure Database for PostgreSQL @sridrang

    @AzureDBPostgres @citusdata
  67. Building resilient, mission-critical applications with Azure Database for PostgreSQL Sridhar

    Ranganathan, Principal Program Manager Azure Open Source Databases @SridRang / @AzureDBPostgres / @citusdata
  68. North Europe Region – East US West US 2 AZ1

    AZ2 AZ3 Flexible Server: Azure Regions, Zones Availability Zone 1 Availability Zone 2 Availability Zone 3 aka.ms/azureregions
  69. Scaling out your Postgres database horizontally Each node has PostgreSQL

    with Citus installed. Each shard [ ] is 1 PostgreSQL table. Coordinator sends DMS/DDL queries to selective worker nodes. Coordinator node
  70. Scaling out your Postgres database horizontally 2. Coordinator aggregates and

    sends the result back to the client Coordinator node aka.ms/hyperscalecitus