Yugabyte DB - Distributed SQL database on Kubernetes Amey Banarse Principal Data Architect Yugabyte Inc. @ameybanarse

2 © 2019 All rights reserved. Introduction 2  Amey Banarse Principal Data Architect, Yugabyte Inc. ♦ Pivotal ♦ FINRA ♦ NYSE University of Pennsylvania (UPenn) @ameybanarse about.me/amey

Kubernetes is massively popular in Fortune 500s ● Walmart - Edge Computing KubeCon 2019 https://www.youtube.com/watch?v=sfPFrvDvdlk ● Target - Data @ Edge https://tech.target.com/2018/08/08/running-cassandra-in-kubernetes -across-1800-stores.html ● eBay - Platform Modernization https://www.ebayinc.com/stories/news/ebay-builds-own-servers-intends -to-open-source/

Data on K8s Ecosystem is evolving rapidly

Why Data services on K8s? Containerized data workloads running on Kubernetes offer several advantages over traditional VM / bare metal based data workloads including but not limited to: ● Better cluster resource utilization ● Portability between cloud and on-premises ● Frictionless multi-tenancy with versioning ● Simple and selective instant upgrades ● Robust automation framework can be embedded inside CRDs (Custom Resource Definitions) or commonly referred as ‘K8S Operator’.

6 Yugabyte Confidential © 2020 All rights reserved. A brief history of Yugabyte Builders of multiple popular DBs Part of Facebook’s Cloud-Native DB Evolution Yugabyte team dealt with this growth first hand Massive geo-distributed deployment given global users Worked with world class infra team to solve these issues Yugabyte founding team ran Facebook’s public-cloud scale DBaaS +1 Trillion ops/day +100 Petabytes data set sizes

7 Yugabyte Confidential © 2020 All rights reserved. What is Distributed SQL? SQL & Transactions SQL Massive Scalability Geo Distribution Ultra Resilience A Revolutionary Database Architecture

8 Yugabyte Confidential © 2020 All rights reserved. Open source, high performance, cloud native, distributed SQL database 100% Apache 2.0 Low Latency (Sub-ms) Kubernetes & Multi-Cloud

9 Yugabyte Confidential © 2020 All rights reserved. Designing the Perfect Distributed SQL DB PostgreSQL more popular than MongoDB Aurora much more popular than Spanner bit.ly/distributed-sql-deconstructed Amazon Aurora Google Spanner A highly available MySQL and PostgreSQL-compatible relational database service Not scalable but HA All RDBMS features PostgreSQL & MySQL The first horizontally scalable, strongly consistent, relational database service Scalable and HA Missing RDBMS features New SQL syntax

tablet 1’ tablet 1’ tablet 1’ Self-Healing, Fault-Tolerant Auto Sharding & Rebalancing ACID Transactions Global Data Distribution High Throughput, Low Latency YCQL SQL-Based Flexible Schema API YSQL Distributed Postgres API Distributed SQL APIs Natively runs in Containers, self service Design Follows a Layered Approach

11 Yugabyte Confidential © 2020 All rights reserved. o YSQL - Fully relational SQL API that is wire compatible with PostgreSQL o YCQL - Optimized Cassandra Query Language API o DocDB – High-performance distributed Document store – Offers strong consistency and multi row ACID transactions Design Follows a Layered Approach

12 © 2019 All rights reserved. All Nodes are Identical … … YugabyteDB Query Layer YugabyteDB Query Layer YugabyteDB Query Layer DocDB Storage Layer DocDB Storage Layer DocDB Storage Layer Can connect to ANY node Add/remove nodes anytime YugabyteDB Node YugabyteDB Node YugabyteDB Node microservices platform

Scaling out PostgreSQL with YugabyteDB

14 © 2019 All rights reserved. 1. Single Region, Multi-Zone Availability Zone 1 Availability Zone 2 Availability Zone 3 Consistent Across Zones No WAN Latency But No Region-Level Failover/Repair 2. Single Cloud, Multi-Region Region 1 Region 2 Region 3 Consistent Across Regions Cross-Region WAN Latency with Auto Region-Level Failover/Repair 3. Multi-Cloud, Multi-Region Cloud 1 Cloud 2 Cloud 3 Consistent Across Clouds Cross-Cloud WAN Latency with Auto Cloud-Level Failover/Repair Deployment Topologies

15 Yugabyte Confidential © 2020 All rights reserved. Deploying Geo-Distributed DB on K8s ● Scalable & Highly Available data tier ● Business Continuity ● Geo-Partitioning & Data Compliance

16 Yugabyte Confidential © 2019 All rights reserved. Yugabyte DB on K8S

17 Yugabyte Confidential © 2019 All rights reserved. Yugabyte Platform Live Demo

18 © 2018 All rights reserved. YugaByte DB Deployed as StatefulSets node2 node1 node4 node3 yb-master StatefulSet yugabytedb yb-master-1 pod yugabytedb yb-master-0 pod yugabytedb yb-master-2 pod yb-tserver StatefulSet tablet 1’ yugabytedb yb-tserver-1 pod tablet 1’ yugabytedb yb-tserver-0 pod tablet 1’ yugabytedb yb-tserver-3 pod tablet 1’ yugabytedb yb-tserver-2 pod … Local/Remote Persistent Volume Local/Remote Persistent Volume Local/Remote Persistent Volume Local/Remote Persistent Volume yb-masters Headless Service yb-tservers Headless Service App Clients Admin Clients

19 © 2018 All rights reserved. Ensuring High Performance LOCAL STORAGE Since v1.10 REMOTE STORAGE Lower latency, Higher throughput Recommended for workloads that do their own replication Pre-provision outside of K8s Use SSDs for latency-sensitive apps Higher latency, Lower throughput Recommended for workloads do not perform any replication on their own Provision dynamically in K8s Use alongside local storage for cost-efficient tiering Most used

20 © 2018 All rights reserved. Configuring Data Resilience POD ANTI-AFFINITY MULTI-ZONE/REGIONAL/MULTI-REGION POD SCHEDULING Pods of the same type should not be scheduled on the same node Keeps impact of node failures to absolute minimum Multi-Zone - Tolerate zone failures for k8s worker nodes Regional – Tolerate zone failures for both k8s worker and master nodes Multi-Region* – Requires federation of k8s clusters or submariner.io

21 © 2018 All rights reserved. Automating Day 2 Operations BACKUP & RESTORE Backups and restores are a database level construct YugaByte DB can perform distributed snapshot and copy to a target for a backup Restore the backup into an existing cluster or a new cluster with a different number of tservers ROLLING UPGRADES Supports two upgradeStrategies: onDelete (default) and rollingUpgrade Pick rolling upgrade strategy for DBs that support zero downtime upgrades such as YugaByte DB New instance of the pod spawned with same network id and storage HANDLING FAILURES Pod failure handled by K8S automatically Node failure has to be handled manually by adding a new slave node to K8S cluster Local storage failure has to be handled manually by mounting new local volume to K8S

22 © 2018 All rights reserved. Extending StatefulSets with Operators https://github.com/yugabyte/yugabyte-operator Based on Custom Controllers that have direct access to lower level K8S API Excellent fit for stateful apps requiring human operational knowledge to correctly scale, reconfigure and upgrade while simultaneously ensuring high performance and data resilience Complementary to Helm for packaging CPU usage in the yb-tserver StatefulSet Scale yb-tserver by 1 pod CPU > 80% for 1min and max_threshold not exceeded

23 © 2018 All rights reserved. YugabyteDB Quickstart docs.yugabyte.com/quick-start

A platform built for a new way of thinking ➔ Event + Microservice first design ➔ Team autonomy with platform efficiency ➔ 100% Cloud Native operating model on k8s ➔ Turnkey multi-cloud ➔ Full Spring Data support

25 Yugabyte Confidential © 2020 All rights reserved. Kubernetes-Native YugabyteDB from a user perspective

26 Yugabyte Confidential © 2020 All rights reserved. Who Installs YugabyteDB?

27 Yugabyte Confidential © 2020 All rights reserved. Key Differentiator Cloud-agnostic, Kubernetes-native, high performance ● Flexibility to run internal DBaaS on AWS or on-prem ● Integration with PKS, Service Broker and Marketplace for internal PaaS ● Multi-master deployment required ✓ This is a technical win ✓ Starting out with a key OLTP application ✓ Interested in building out cloud-agnostic private DBaaS to power private PaaS Other Appealing Features Scalability & High Availability Operational efficiency (zero-downtime day 2 operations) Full PostgreSQL support (eventually replace Oracle) Kubernetes-Ready A Large US Healthcare provider

28 Yugabyte Confidential © 2020 All rights reserved. Key Differentiator Kubernetes-native, multi-region database ● Enable users to build stateful edge applications ● Move data across multiple geographies ● Deployments become cloud-agnostic ✓ Building an edge cloud solution ✓ Looked at multiple DBs before picking YugabyteDB ✓ Repeatable “telco” and “edge” play Other Appealing Features Scalability & High Availability Yugabyte Platform (turnkey solution for service providers) Low Latency Kubernetes-Ready A Large US Telco provider

A classic Enterprise App scenario

30 © 2018 All rights reserved. A Real-World Demo Yugastore – E-Commerce app on the cloud native stack Deployed on https://github.com/yugabyte/yugastore-java

Yugastore - Kronos Marketplace

Classic Enterprise Microservices Architecture CART MICROSERVICE PRODUCT MICROSERVICE API Gateway CHECKOUT MICROSERVICE UIU Yugabyte Cluster SQL CQL CQL UI APP REST .

Istio Traffic Management for Microservices CART MICROSERVICE PRODUCT MICROSERVICE API Gateway CHECKOUT MICROSERVICE UIU UI APP Galley Citadel Pilot Istio Edge Proxy Istio Control Plane Istio Service Discovery Istio Edge Gateway Istio Route Configuration using Envoy Proxy

Yugabyte Spark Integration ● Analytics and Aggregate Queries ● ML workloads - Recommendations and Rankings ● Uses pySpark, OSS Spark-Cassandra connectors Source Tables Derived Tables Enrichment / pre-aggregation Batch Aggregates

35 © 2019 All rights reserved. Join the community yugabyte.com/slack We stars! github.com/yugabyte/yugabyte-db We’re Hiring! bit.ly/yugabyte-careers

36 Yugabyte Confidential © 2020 All rights reserved. The default database for the cloud Thank You!