Slide 1

Slide 1 text

What is Azure Arc enabled PostgreSQL Hyperscale? Jean-Yves Devant (JY) Principal Program Manager Microsoft

Slide 2

Slide 2 text

What is Azure Arc enabled PostgreSQL Hyperscale?

Slide 3

Slide 3 text

Where Does That Fit? How does it work? Why PostgreSQL? What is Azure Database for PostgreSQL Hyperscale (Citus)? What is Azure Arc? What is Azure Arc enabled data services? Where does Azure Arc and Azure PostgreSQL Hyperscale meet? Show it to me!

Slide 4

Slide 4 text

WHY POSTGRESQL?

Slide 5

Slide 5 text

loved wanted https://insights.stackoverflow.com/survey/2019 https://db-engines.com/en/blog_post/76 DBMS of the Year Why PostgreSQL?

Slide 6

Slide 6 text

Open source Large developer community Proven resilience & stability Thousands of mission critical workloads Rich feature set Solves a multitude of use cases Why PostgreSQL?

Slide 7

Slide 7 text

High performances Open source Relational JSON/B support Key/value pairs with hstore Extensions Highly customizable Flexible datatypes Python, Ruby, R, V8… Frequent releases Rich indexing Geospatial Full text search Why PostgreSQL?

Slide 8

Slide 8 text

WHAT IS AZURE DATABASE FOR POSTGRESQL HYPERSCALE (CITUS)?

Slide 9

Slide 9 text

What is Azure Database For PostgreSQL Hyperscale (Citus)? Managed service in Azure Runs the Citus extension Cluster of multiple PostgreSQL instances Scales out compute horizontally Distributes data and queries Superior performance

Slide 10

Slide 10 text

APPLICATION COORDINATOR NODE WORKER NODES W1 W2 W3 … Wn A cluster of multiple PostgreSQL servers with the Citus extension. How Does It Work?

Slide 11

Slide 11 text

APPLICATION CREATE TABLE campaigns (…); SELECT create_distributed_table( 'campaigns','company_id'); METADATA COORDINATOR NODE WORKER NODES W1 W2 W3 … Wn CREATE TABLE campaigns_102 CREATE TABLE campaigns_105 CREATE TABLE campaigns_101 CREATE TABLE campaigns_104 CREATE TABLE campaigns_103 CREATE TABLE campaigns_106 Distributes tables across the cluster How Does It Work?

Slide 12

Slide 12 text

APPLICATION SELECT FROM GROUP BY company_id, avg(spend) AS avg_campaign_spend campaigns company_id; METADATA COORDINATOR NODE WORKER NODES W1 W2 W3 … Wn SELECT company_id sum(spend), count(spend) … FROM campaigns_2009 … SELECT company_id sum(spend), count(spend) … FROM campaigns_2001 … SELECT company_id sum(spend), count(spend) … FROM campaigns_2017 … Distributes queries across the cluster How Does It Work?

Slide 13

Slide 13 text

Distributes transactions in the cluster, example 1 BEGIN; UPDATE SET WHERE COMMIT; campaigns start_date = '2018-03-17' company_id = 'Pat Co'; METADATA W1 W2 W3 … Wn BEGIN; UPDATE Campaigns_2012 SET …; COMMIT; APPLICATION COORDINATOR NODE WORKER NODES How Does It Work?

Slide 14

Slide 14 text

APPLICATION BEGIN; UPDATE SET WHERE UPDATE SET WHERE COMMIT; campaigns feedback = 'relevance' company_type = 'platinum'; ads feedback = 'relevance' company_type = 'platinum'; METADATA W1 W2 W3 … Wn BEGIN … assign_distributed_ transaction_id … UPDATE campaigns_2009 … COMMIT PREPARED … BEGIN … assign_distributed_ transaction_id … UPDATE campaigns_2001 … COMMIT PREPARED … BEGIN … assign_distributed_ transaction_id … UPDATE campaigns_2017 … COMMIT PREPARED … COORDINATOR NODE WORKER NODES Distributes transactions in the cluster, example 2 How Does It Work?

Slide 15

Slide 15 text

How Far Can Citus Scale? Algolia 5-10B rows ingested per day Heap 700+ billion events 1.4PB data on a 70-node Citus Chartbeat >2.6B rows added per month Mixrank 1.6PB time series data Microsoft Petabyte-scale analytics from 800M+ Windows devices “Distributed PostgreSQL is a game changer. We can support more than 6M queries every day, on 2 PB of data. With Citus, response times for 75% of queries are less than 0.2 seconds.” https://aka.ms/blog-petabyte-scale-analytics Pex 80B rows updated/day 20-node Citus 2.4TB memory, 1280 cores, 80TB of data Customer stories: https://www.citusdata.com/customers

Slide 16

Slide 16 text

Citus helps ASB onboard customers 20x faster “After migrating to Citus, we can onboard Vonto customers 20X faster, in 2 minutes vs. the 40+ minutes it used to take. And with the launch of Hyperscale (Citus) on Azure Database for PostgreSQL, we are excited to see what we can build next on Azure.”

Slide 17

Slide 17 text

WHAT IS AZURE ARC?

Slide 18

Slide 18 text

Hybrid cloud is the norm

Slide 19

Slide 19 text

Managing the complexity of hybrid cloud is the challenge

Slide 20

Slide 20 text

10s–1,000s of apps Diverse infrastructure Multi-cloud IoT devices Edge Datacenters Branch offices Hosters OEM hardware VMs Containers Databases Serverless Customer Environments Are Increasingly Complex

Slide 21

Slide 21 text

Elastic scalability Self-service provisioning Built-in monitoring and security Pay for just what you use Management from anywhere Automation at scale Azure Arc Helps You Realize Cloud Benefits Everywhere!

Slide 22

Slide 22 text

Azure IoT Any edge device Azure Arc Any datacenter, any cloud Integrated systems Azure Stack Microsoft Azure Azure Hybrid Innovation anywhere with Azure Management | Security + Identity | App + Data Services | Dev Tools + DevOps

Slide 23

Slide 23 text

Azure Arc Bring Azure services and management to any infrastructure Azure Arc is a set of technologies that extends Azure management and enables Azure services to run across on-premises, multi-cloud, and edge Implement Azure security anywhere Run Azure Data Services anywhere Extend Azure management across your environments Adopt cloud practices on-premises

Slide 24

Slide 24 text

Across Any Infrastructure Public cloud On-premises datacenter Edge site

Slide 25

Slide 25 text

At-scale Kubernetes app management Organize and govern across environments Multi-cloud Datacenter & hosted Azure Arc Customer use cases Use cloud services on prem and still meet compliance and regulatory requirements Azure Arc enabled servers https://aka.ms/arc-serversdocs Azure Arc enabled Kubernetes https://aka.ms/Azure-Arc-Kubernetes Azure Arc enabled data services https://aka.ms/azurearcdata All Azure Arc services https://aka.ms/azurearc Run data services anywhere

Slide 26

Slide 26 text

How Do I Get Started With Azure Arc? http://aka.ms/azurearc https://docs.microsoft.com/azure/azure-arc

Slide 27

Slide 27 text

WHAT IS AZURE ARC ENABLED DATA SERVICES?

Slide 28

Slide 28 text

Elastic scale PostgreSQL Hyperscale Scale up, scale out on demand Automation at scale Always current Self-service provisioning in seconds Automated updates Evergreen SQL Managed Instance Unified management Single view for on-prem, clouds, and edge Consistent tools and workflows Built-in monitoring and security Azure Arc Enabled Data Services Azure data services in your datacenter, multi-cloud, and edge Connected or Disconnected

Slide 29

Slide 29 text

Azure Arc enabled PostgreSQL Hyperscale Azure Arc Enabled Data Services In Preview Now! Azure Arc enabled SQL Managed Instance Azure Arc enabled SQL Server Try Azure Arc enabled data services for free and let us know what you think https://aka.ms/AzureArcData

Slide 30

Slide 30 text

Azure Data Services Anywhere At A Glance Applications Custom apps Analytics BI … Any Kubernetes AKS Any hardware Azure data services OEM hardware Azure data controller Kubernetes OpenShift Microsoft Azure Site Recovery Azure Site Recovery Monitoring Azure Security Provisioning HA/DR Scaling Updates Backup Diagnostics Amazon EC2

Slide 31

Slide 31 text

Why Kubernetes? Leading application containers technology Abstraction layer, runs on any infrastructure Consistent & at-scale deployment and management in seconds Automation and CI/CD at scale with GitOps https://www.gitops.tech

Slide 32

Slide 32 text

Connectivity Modes Indirectly connected (preview) Local provisioning/de-provisioning Local elastic scaling Local monitoring Local log analytics Local backup/restore Upload logs and metrics to Monitor View inventory in Azure Upload billing data to Azure Use Kubernetes authentication and authorization Azure DevOps, GitOps operations Directly connected (future) More details to be announced later…

Slide 33

Slide 33 text

Azure Arc data controller Backup Monitoring and logs Controller API Azure Arc integration HA/DR Scaling Patching/updates Provisioning Persistent storage Node Node Node Node Node Node Azure Data Studio Identity Azure RBAC & Policy Advanced Data Security Deployments Resource Inventory Logs & Telemetry Backup Retention Consumption azdata CLI kubectl CLI Microsoft Container Registry Azure Portal Azure Data Studio CLI 3rd Party Kubernetes API Azure Arc enabled PostgreSQL Hyperscale Other Database service Analytics services Azure Arc Data Services Architecture Deeper Dive

Slide 34

Slide 34 text

Roles And Responsibilities: PaaS Vs. Hybrid Who’s in charge of SLAs? Azure Platform As A Service (PaaS) Azure Arc hybrid services Microsoft Yes Microsoft Microsoft Microsoft Customer No Customer Microsoft Does Microsoft provide SLAs? Who does the operations? Who provides the software*? Who provides the infrastructure? *Azure services Customer

Slide 35

Slide 35 text

How Do I Get Started With Azure Arc Enabled Data Services? https://docs.microsoft.com/azure/azure- arc/data/

Slide 36

Slide 36 text

AZURE ARC ENABLED POSTGRESQL HYPERSCALE + =

Slide 37

Slide 37 text

This Is Where It All Comes Together Azure Arc enabled PostgreSQL Hyperscale is:

Slide 38

Slide 38 text

How Do I Get Started With Azure Arc Enabled PostgreSQL Hyperscale? Get started https://aka.ms/arcpostgresqlhyperscale Deploy https://aka.ms/deployarcpostgresqlhyperscale Accelerated experience with a test deployment https://github.com/microsoft/azure_arc#azure-arc- enabled-data-services In preview now. Free

Slide 39

Slide 39 text

SHOW IT TO ME!

Slide 40

Slide 40 text

Postgres In Azure Vs. Other Clouds? The Choice Hyperscale (Citus) Worry-free PostgreSQL in the cloud with an architecture built to scale out Single Server Enterprise-ready, fully managed community OSS engines Azure Arc enabled PostgreSQL Hyperscale NEW Hybrid, scale out PostgreSQL in environment of your choice Flexible Server (Preview) NEW Maximum control with a simplified developer experience Open source & community PostgreSQL committers at Microsoft: https://aka.ms/blog-postgres-committers

Slide 41

Slide 41 text

Q&A Get started https://aka.ms/arcpostgresqlhyperscale Follow us: @AzureDBPostgres, @CitusData

Slide 42

Slide 42 text

Special Thanks To for supporting DataPlatformGeeks & SQLServerGeeks Community Initiatives

Slide 43

Slide 43 text

Three Ways to Win Prizes Post your selfie with hash tag #DPS2020 Give Session & Conference Feedback Visit our Sponsors & Exhibitors Thank You Follow us on Twitter @TheDataGeeks @DataAISummit

Slide 44

Slide 44 text

Data Platform Virtual Summit 2020 is a community initiative by DataPlatformGeeks RESOURCES

Slide 45

Slide 45 text

Go Deeper Into Postgres & Hyperscale (Citus) • https://www.citusdata.com/ • http://docs.citusdata.com/en/v9.5/ Why Scale Out Postgres? https://youtu.be/g3H4nGsJsl0 DEMO - High performance HTAP with Postgres & Hyperscale (Citus) https://youtu.be/W_3e07nGFxY DEMO – Building HTAP Applications with Python & Postgres on Azurehttps://youtu.be/YDT8_riLLs0

Slide 46

Slide 46 text

DELETE BEFORE PUBLISHING