Making Azure the best destination to run Postgres | European Virtual Open Source Summit 2020 | Umur Cubukcu

Developers love Postgres, and so does our open source database team at Microsoft. And we’ve been investing more and more in Postgres—by acquiring Citus Data to enable apps that need horizontal scale, by hiring and investing in Postgres open source committers, and of course by launching the Azure Database for PostgreSQL database service back in the day. In this talk—Umur Cubukcu, head of the Azure Postgres product team and former CEO of Citus Data—will explain why he believes Azure is the best destination to run Postgres.

This talk will show you how Hyperscale (Citus)—now built into Azure Database for PostgreSQL—is changing what is possible with Postgres. Based on the Citus open source extension to Postgres, Hyperscale (Citus) transforms Postgres into a distributed database, empowering developers and applications to do things that were simply not possible before. Umur also plans to share insights into real-world Azure scenarios that run on Postgres, including multi-tenant SaaS, real-time analytics, time series, and geospatial use cases.

Transcript

1. Making Azure the best
   destination to run Postgres
   Umur Cubukcu Principal Group PM Manager,
   Azure & former CEO of Citus Data
   @umurc / @AzureDBPostgres / @citusdata
   

   View Slide

2. View Slide

3. 9 years ago we founded
   a company to change
   what’s possible with Postgres
   

   View Slide

4. PostgreSQL
   transformed into a
   horizontally
   scalable database
   

   View Slide

5. Packaged as
   extension,
   not a fork
   PostgreSQL
   transformed into a
   horizontally
   scalable database
   Open source
   &
   Fully managed
   

   View Slide

6. Packaged as
   extension,
   not a fork
   PostgreSQL
   transformed into a
   horizontally
   scalable database
   Open source
   & fully-managed
   

   View Slide

7. “Distributed PostgreSQL is a game changer.
   We can support more than 6M queries every day, on 2 PB
   of data. With Citus, response times for 75% of queries are
   less than 0.2 seconds.”
   

   View Slide

8. Microsoft Azure Welcomes PostgreSQL Committers
   aka.ms/blog-postgres-committers
   

   View Slide

9. View Slide

10. Azure
    Why PostgreSQL?
    Postgres-as-a-Service on Azure
    Postgres at any scale: Hyperscale (Citus)
    What makes Azure unique for PostgreSQL?
    Use cases & resources
    

    View Slide

11. DB-Engines Ranking
    PostgreSQL (+ OSS Relational) more & more popular
    Source: DB-Engines.com.
    

    View Slide

12. of companies invest
    in public cloud
    technologies3
    of new in-house
    applications being built
    on open source DBMS2
    https://insights.stackoverflow.com/survey/2019
    Postgres one of most
    loved & wanted databases
    in Stack Overflow
    Developer Survey 20191
    >70%
    growing in popularity
    80%
    

    View Slide

13. Open source
    Large developer
    community
    Rich feature set
    Solves a multitude of
    use cases
    Proven resilience
    & stability
    Thousands of mission
    critical workloads
    Why PostgreSQL?
    • Enterprise-ready
    • Zero data loss
    • Rich indexing, text search,
    arrays
    • Extensible
    & tooling
    

    View Slide

14. Why We Postgres
    • Open Source
    • Constraints
    • Extensions
    • PostGIS
    • Citus
    • B-tree, GIN, BRIN, &
    GiST
    • Available as a database
    service
    • Decades of robustness
    • Millions of happy users
    • Foreign data wrappers
    • Window functions
    • CTEs
    • ACID
    • Full text search
    • JSONB
    • Rich datatypes
    • pg_stat_statements
    • Rollups
    

    View Slide

15. Azure
    Why PostgreSQL?
    Postgres-as-a-Service on Azure
    Postgres at any scale: Hyperscale (Citus)
    What makes Azure unique for PostgreSQL?
    Use cases & resources
    

    View Slide

16. PostgreSQL
    built for cloud:
    Simpler &
    better on
    Azure
    Global
    reach
    Security
    Scale up
    & out
    High
    Availability
    Compliance
    Intelligent
    performance
    Easy ecosystem
    integration
    Extension
    support
    

    View Slide

17. You have choices.
    We are fans of
    fully-managed
    database
    service (PaaS)
    On-prem
    PostgreSQL
    Managed by you
    Managed by Postgres
    experts at Microsoft
    Datacenter
    management
    Hardware
    O/S provision /
    patching
    Database provision /
    Patch / Scaling
    Virtualization
    Security / Compliance
    Performance
    optimization
    High availability
    / DR / Backups
    Data
    Applications
    

    View Slide

18. You have choices.
    We are fans of
    fully-managed
    database
    service (PaaS)
    On-prem
    PostgreSQL
    IaaS
    Azure VMs with
    PostgreSQL
    Managed by you
    Managed by Postgres
    experts at Microsoft
    Datacenter
    management
    Hardware
    Virtualization
    O/S patching
    Database provision /
    Patch / Scaling
    Security / Compliance
    Performance
    optimization
    High availability
    / DR / Backups
    Data
    Applications
    Datacenter
    management
    Hardware
    O/S provision /
    patching
    Database provision /
    Patch / Scaling
    Virtualization
    Security / Compliance
    Performance
    optimization
    High availability
    / DR / Backups
    Data
    Applications
    

    View Slide

19. You have choices.
    We are fans of
    fully-managed
    database
    service (PaaS)
    On-prem
    PostgreSQL
    IaaS
    Azure VMs with
    PostgreSQL
    PaaS
    Azure Database for
    PostgreSQL
    Managed by you
    Managed by Postgres
    experts at Microsoft
    Security / Compliance
    Intelligent performance
    Datacenter
    management
    Hardware
    Virtualization
    O/S
    Database provision /
    Patch / Scaling
    High availability
    / DR / Backups
    Data
    Applications
    Datacenter
    management
    Hardware
    Virtualization
    O/S patching
    Database provision /
    Patch / Scaling
    Security / Compliance
    Performance
    optimization
    High availability
    / DR / Backups
    Data
    Applications
    Datacenter
    management
    Hardware
    O/S provision /
    patching
    Database provision /
    Patch / Scaling
    Virtualization
    Security / Compliance
    Performance
    optimization
    High availability
    / DR / Backups
    Data
    Applications
    

    View Slide

20. Azure
    Why PostgreSQL?
    Postgres-as-a-Service on Azure
    Postgres at any scale: Hyperscale
    (Citus)
    What makes Azure unique for PostgreSQL?
    Use cases & resources
    

    View Slide

21. Changing what's possible with Hyperscale (Citus)
    Grow to 100s of database nodes,
    without re-architecting your application
    Block growth on 1
    (monolithic) database
    18
    Total Nodes
    

    View Slide

22. View Slide

23. aka.ms/citus
    

    View Slide

24. Elastic scale
    Always current Unified management
    Azure data services anywhere: On-premises, multi-cloud
    Cloud benefits for both disconnected and connected scenarios
    Disconnected support
    aka.ms/AzureArc (PREVIEW)
    Azure SQL Database
    PREVIEW Azure Database for PostgreSQL Hyperscale
    PREVIEW
    

    View Slide

25. Azure
    Why PostgreSQL?
    Postgres-as-a-Service on Azure
    Postgres at any scale: Hyperscale (Citus)
    What makes Azure unique for
    PostgreSQL?
    Use cases & resources
    

    View Slide

26. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Collaborate & contribute
    

    View Slide

27. Making Azure the best destination for PostgreSQL
    Rich feature set
    Proven resilience
    & stability
    Open source
    Highly scalable
    Fully managed Run anywhere Security &
    Compliance
    Hyperscale (Citus) Azure Arc
    

    View Slide

28. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Highly scalable
    Fully managed Run anywhere Security &
    Compliance
    

    View Slide

29. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Highly scalable
    Fully managed Run anywhere Security &
    Compliance
    Available on Microsoft Azure
    

    View Slide

30. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Highly scalable
    Fully managed Run anywhere Security &
    Compliance
    Available on Microsoft Azure
    

    View Slide

31. aka.ms/ads-postgres-extension
    

    View Slide

32. Azure
    Why PostgreSQL?
    Postgres-as-a-Service on Azure
    Postgres at any scale: Hyperscale (Citus)
    What makes Azure unique for PostgreSQL?
    Use cases & resources
    

    View Slide

33. Single Server Use Cases for PostgreSQL on Azure
    Digital transformations & data estate modernization
    AKS,
    Python/Django, Ruby/Rails,
    Java/Hibernate
    New
    Cloud-Native
    

    View Slide

34. Lift & shift
    to cloud
    Single Server Use Cases for PostgreSQL on Azure
    Digital transformations & data estate modernization
    AKS,
    Python/Django, Ruby/Rails,
    Java/Hibernate
    PaaS v. IaaS: Focusing on
    your application & not your
    database.
    New
    Cloud-Native
    

    View Slide

35. Lift & shift
    to cloud
    Oracle to
    PostgreSQL
    Single Server Use Cases for PostgreSQL on Azure
    Digital transformations & data estate modernization
    AKS,
    Python/Django, Ruby/Rails,
    Java/Hibernate
    PaaS v. IaaS: Focusing on
    your application & not your
    database.
    Modernize data estate,
    build new analytics, lower
    costs
    New
    Cloud-Native
    

    View Slide

36. Lift & shift
    to cloud
    Oracle to
    PostgreSQL
    Single Server Use Cases for PostgreSQL on Azure
    Digital transformations & data estate modernization
    AKS,
    Python/Django, Ruby/Rails,
    Java/Hibernate
    PaaS v. IaaS: Focusing on
    your application & not your
    database.
    Modernize data estate,
    build new analytics, lower
    costs
    New
    Cloud-Native
    Digital transformations + Modern relational applications built with OSS
    

    View Slide

37. Hyperscale (Citus) use cases for Postgres on Azure
    Digital transformations & data estate modernization
    Real-time,
    operational analytics
    JSON, Geospatial, Timeseries,
    HTAP workloads
    

    View Slide

38. Hyperscale (Citus) use cases for Postgres on Azure
    Digital transformations & data estate modernization
    Multi-tenant
    & SaaS
    Real-time,
    operational analytics
    JSON, Geospatial, Timeseries,
    HTAP workloads
    Enterprises, B2B,
    Sharding, ISVs building
    SaaS apps
    

    View Slide

39. Hyperscale (Citus) use cases for Postgres on Azure
    Digital transformations & data estate modernization
    Multi-tenant
    & SaaS
    Real-time,
    operational analytics
    JSON, Geospatial, Timeseries,
    HTAP workloads
    Transactional /
    OLTP
    Enterprises, B2B,
    Sharding, ISVs building
    SaaS apps
    Strong consistency, Relational
    (foreign keys, joins), limitless
    data
    

    View Slide

40. Scaling from as low
    as 10s of GBs up to
    Petabytes
    

    View Slide

41. Architecting petabyte-scale analytics by scaling out
    Postgres on Azure with the Citus extension
    aka.ms/blog-petabyte-scale-analytics
    

    View Slide

42. Citus helps ASB onboard
    customers 20x faster
    “After migrating to Citus, we
    can onboard Vonto customers
    20X faster, in 2 minutes vs. the
    40+ minutes it used to take.
    And with the launch of
    Hyperscale (Citus) on Azure
    Database for PostgreSQL, we
    are excited to see what we can
    build next on Azure.”
    

    View Slide

43. Superpowers in Azure Database for PostgreSQL
    Hyperscale
    (Citus) to
    scale out
    horizontally
    

    View Slide

44. Know that Microsoft loves open source (& Postgres)
    Hyperscale
    (Citus) to
    scale out
    horizontally
    Open source
    commitment
    

    View Slide

45. If thinking of using Postgres, 3 reasons to think of Azure
    Hyperscale
    (Citus) to
    scale out
    horizontally
    Complemented
    with Azure
    ecosystem
    Open source
    commitment
    

    View Slide

46. Our Azure Postgres service page—and our blog!
    Azure Database for PostgreSQL
    https://aka.ms/azure-postgres
    Azure Postgres Blog
    https://aka.ms/azure-postgres-blog
    

    View Slide

47. Migrations
    https://aka.ms/postgres-migration-tutorial
    Wealth of documentation resources, too
    Azure Postgres Quickstart Docs
    https://aka.ms/azure-postgres-quickstart
    Azure Database for PostgreSQL
    https://aka.ms/azure-postgres
    Azure Postgres Blog
    https://aka.ms/azure-postgres-blog
    

    View Slide

48. Migrations
    https://aka.ms/postgres-migration-tutorial
    AskAzureDBforPostgreSQL@service.microsoft.com
    Citus open source packages on GitHub—also, Email
    https://aka.ms/citus
    Azure Postgres Quickstart Docs
    https://aka.ms/azure-postgres-quickstart
    Azure Database for PostgreSQL
    https://aka.ms/azure-postgres
    Azure Postgres Blog
    https://aka.ms/azure-postgres-blog
    

    View Slide

49. Citus Newsletter
    aka.ms/citus-newsletter
    

    View Slide

50. © Copyright Microsoft Corporation. All rights reserved.
    danke schön
    dank u
    merci
    teşekkürler
    thank you
    grazie
    gracias
    tack
    @umurc
    @AzureDBPostgres
    @citusdata
    Umur Cubukcu
    

    View Slide

51. BACKUP
    

    View Slide

52. “Distributed
    PostgreSQL is a game
    changer.
    We can support more
    than 6M queries
    every day, on 2 PB of
    data. With Citus,
    response times for
    75% of queries are
    less than 0.2
    seconds.”
    

    View Slide

53. Hyperscale (Citus) use cases for Postgres on Azure
    JSON, Geospatial,
    Timeseries, HTAP
    workloads
    Enterprises, B2B,
    Sharding, ISVs
    building SaaS apps
    Strong consistency,
    Relational (foreign
    keys, joins), limitless
    data
    Real-time,
    operational
    analytics
    Multi-tenant
    & SaaS
    Transactional
    / OLTP
    

    View Slide

54. Digital transformations with data at the heart
    +
    

    View Slide

55. Single Server Use Cases for PostgreSQL on Azure
    AKS, Python/Django,
    Ruby/Rails,
    Java/Hibernate
    PaaS v. IaaS: Focusing
    on your application &
    not your database
    Modernize data
    estate, build new
    analytics, lower costs
    New
    Cloud-Native
    Apps
    Lift/shift
    to cloud
    Oracle to
    Postgres
    

    View Slide

56. Digital transformations with data at the heart
    

    View Slide

57. Digital transformations with data at the heart
    

    View Slide

58. Scaling from as low as 10s of GBs up to PBs
    

    View Slide

59. Making Azure the best destination for PostgreSQL
    Rich feature set
    Proven resilience
    & stability
    Open source
    Highly scalable
    Fully managed Run anywhere Intelligent Perf &
    Security
    Hyperscale (Citus) Azure Arc
    

    View Slide

60. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Highly scalable
    Fully managed Run anywhere Intelligent Perf &
    Security
    

    View Slide

61. Rich feature set
    Proven resilience
    & stability
    Open source
    Making Azure the best destination for PostgreSQL
    Highly scalable
    Fully managed Run anywhere Intelligent Perf &
    Security
    Available on Microsoft Azure
    

    View Slide

62. Rich feature set
    Proven resilience
    & stability
    Open source
    Fully
    managed
    Highly
    scalable
    Security &
    Compliance
    Intelligent
    performance
    Making Azure the best destination for PostgreSQL
    

    View Slide

63. Making Azure the best destination for PostgreSQL
    Available on Microsoft Azure
    Rich feature set
    Proven resilience
    & stability
    Open source
    Fully
    managed
    Highly
    scalable
    Security &
    Compliance
    Intelligent
    performance
    Hyperscale (Citus)
    

    View Slide

64. Unmatched
    security
    Elastic
    scale
    Always
    current
    Cloud
    billing
    Unified
    management
    Azure SQL Database
    PREVIEW Azure Database for PostgreSQL Hyperscale
    PREVIEW
    Azure data services anywhere
    Bring Azure data services to on-premises, multi-cloud, and edge with Azure Arc
    aka.ms/AzureArc
    

    View Slide

65. Changing what's possible with Hyperscale (Citus)
    Grow to 100s of database nodes,
    without re-architecting your application
    Block growth on 1
    (monolithic) database
    18
    Total Nodes
    

    View Slide

66. Source: DB-Engines.com.
    DB-Engines Ranking
    PostgreSQL is more popular than ever
    

    View Slide

67. of companies
    invest in
    public cloud
    technologies1
    PostgreSQL named
    2018 and 2017
    “Database of the year”
    by DB-Engines.com2
    https://db-engines.com/en/blog_post/76
    https://insights.stackoverflow.com/survey/2019
    Postgres one of most
    loved & wanted databases
    in Stack Overflow
    Developer Survey 20193
    80%
    growing in popularity
    

    View Slide

68. Microsoft acquired Citus Data in early 2019
    Umur I added this new slide. Screenshot of the MSFT blog about the
    acquisition. You tripped in the dry run here. So I thought it would make more
    sense to give you an explicit slide
    to mention the acquisition (after talking about how you founded a company 9
    years ago)
    

    View Slide

69. Slide palette info
    PowerPoint Theme Accent colors
    Accent 1 Accent 2 Accent 3
    Accent 4 Accent 5 Accent 6
    

    View Slide

70. Ensure slide content is accessible
    Take the following steps to create accessible content that everyone can consume effectively.
    Contrast
    Use high contrast colors for
    maximum readability
    The recommended contrast
    ratio is at least 4.5:1
    Text Text
    Color Contrast Analyzer
    Download this tool to determine
    the legibility of text and the
    contrast of visual elements
    Download
    Shape and color
    Use different shapes with a
    legend to indicate statuses
    to accommodate for color
    blindness
    Example:
    Alt text
    Alt text helps people with
    screen readers understand
    the content of slides
    You should create alternative
    text for shapes, pictures,
    charts, tables, SmartArt
    graphics, or other objects
    Here’s how:
    Right click on item
    Select Edit Alt Text
    Enter a Description
    If an object is meant to add visual
    styling and doesn’t contain
    meaning (e.g. stylistic borders), do
    not add alt text and instead check
    Mark as decorative
    Slide layouts
    Using a built-in slide layout
    that matches your content
    ensures a hierarchical
    reading order of text blocks
    Example:
    If a new slide will have a title,
    rather than starting with a blank
    layout and adding a text block for
    the title, choose one of the built-in
    layouts with a title placeholder
    Reading order
    Screen readers describe
    content on the screen in the
    order it was created
    To ensure your content is
    read back in the order you
    prefer, arrange your objects
    in the Selection Pane
    appropriately. Objects on the
    bottom of the selection pane
    are read first
    Here’s how:
    Click the Home tab
    In the Drawing group, select the
    Arrange drop-down menu
    Click Selection Pane…
    Additional tips
    Be sure to run the Accessibility Checker! Go to File click the Check for Issues drop down menu click Check Accessibility
    Videos need to be accessible: If your presentation includes a video, ensure it is captioned and audio described (if appropriate)
    Visit the Office Accessibility Center to learn more about accessibility in PowerPoint
    

    View Slide

71. Brand and accent colors accessibility
    Type and background combinations must
    meet a minimum 4.5:1 contrast ratio to help
    ensure people of all abilities can access and
    understand our communications.
    Pick colors carefully and avoid relying on
    color alone to convey information. Always
    provide text or other graphical cues to guide
    customers to our message.
    Examples demonstrate correct color use
    for on-screen applications.
    Get the Color Contrast Analyzer
    Get more information on accessibility
    Get the Microsoft color guidelines PDF
    Dark Orange
    Rich Black
    R255 G147 B73
    Hex #FF9349
    Dark Yellow
    Rich Black
    R254 G240 B0
    Hex #FEF000
    Dark Green
    Rich Black
    R155 G240 B11
    Hex #9BF00B
    Dark Teal
    Rich Black
    R48 G229 B208
    Hex #30E5D0
    Dark Blue
    Rich Black
    R80 G230 B255
    Hex #50E6FF
    Dark Purple
    Rich Black
    R213 G157 B255
    Hex #D59DFF
    White
    Rich Black
    R216 G59 B1
    Hex #D83B01
    Rich Black
    R255 G185 B0
    Hex #FFB900
    White
    R16 G124 B16
    Hex #107C10
    White
    Rich Black
    R0 G133 B117
    Hex #008575
    White
    Rich Black
    R0 G120 B212
    Hex #0078D4
    White
    Rich Black
    R134 G97 B197
    Hex #8661C5
    Light Orange
    White
    R107 G41 B41
    Hex #6B2929
    Light Yellow
    White
    R106 G75 B22
    Hex #6A4B16
    Light Green
    White
    R5 G75 B22
    Hex #054B16
    Light Teal
    White
    R39 G75 B71
    Hex #274B47
    Light Blue
    White
    R36 G58 B94
    Hex #243A5E
    Light Purple
    White
    R59 G46 B88
    Hex #3B2E58
    White
    R255 G255 B255
    Hex #FFFFFF
    Orange
    Dark Orange
    Dark Yellow Green
    Dark Green
    Teal
    Dark Teal
    Blue
    Dark Blue
    Purple
    Dark Purple
    Rich Black
    R0 G0 B0
    Hex #000000
    White
    Light Orange
    Orange
    White
    Light Yellow
    Yellow
    White
    Light Green
    White
    Light Teal
    Teal
    White
    Light Blue
    Blue
    White
    Light Purple
    Purple
    Extra Light Gray
    R242 G242 B242
    Hex #F2F2F2
    Light Gray
    R230 G230 B230
    Hex #E6E6E6
    Gray
    R210 G210 B210
    Hex #D2D2D2
    Mid Gray
    R115 G115 B115
    Hex #737373
    Dark Gray
    R80 G80 B80
    Hex #505050
    Extra Dark Gray
    R47 G47 B47
    Hex #2F2F2F
    

    View Slide

72. Fully-managed database
    service on Azure means
    you can focus on your
    application
    On-prem
    PostgreSQL
    IaaS
    Azure VMs with
    PostgreSQL
    PaaS
    Azure Database
    for PostgreSQL
    Datacenter
    management
    Hardware
    O/S provision
    /patching
    Database provision/
    Patch/Scaling
    Virtualization
    Data
    Applications
    High availability
    /DR/Backups
    Datacenter
    management
    Hardware
    Virtualization
    O/S
    Database provision/
    Patch/Scaling
    Data
    Applications
    High availability
    /DR/Backups
    Data
    Applications
    Datacenter
    management
    Hardware
    Virtualization
    O/S
    Database provision/
    Patch/Scaling
    High availability/
    DR/Backups
    Intelligent
    performance/security
    Managed by our Postgres experts at Microsoft
    Managed by you
    Only on Azure—machine learning capability
    

    View Slide

73. “After migrating to Citus, we can
    onboard Vonto customers 20X faster,
    in 2 minutes vs. the 40+ minutes it
    used to take. And with the launch of
    Hyperscale (Citus) on Azure
    Database for PostgreSQL, we are
    excited to see what we can build
    next on Azure.”
    Citus helps ASB onboard
    customers 20x faster
    

    View Slide

74. “
    At ASB, our Vonto service had a vast amount of signups
    in a short period, causing our single-node Postgres
    database to creak at the seams—no matter how much
    we vertically scaled our single node infrastructure.
    After migrating to Citus, we can onboard Vonto customers
    20X faster, in 2 minutes vs. the 40+ minutes it used
    to take. And with the launch of Hyperscale (Citus)
    on Azure Database for PostgreSQL, we are excited
    to see what we can build next on Azure.”
    Citus was a game changer for us.
    Vonto / ASB
    

    View Slide

75. Why Azure is the best
    destination to run Postgres
    Umur Cubukcu Principal Group PM Manager, Azure
    & former CEO of Citus Data
    

    View Slide