TiDB Operator: LA Kuberentes Meetup, May 22, 2019

This deck was delivered at the LA Kubernetes meetup on May 22, 2019, to introduce TiDB, TiKV and how to run the various components of the TiDB platform on Kubernetes using the operator pattern.

Transcript

1. “Smooth” Operator Kevin Xu (@kevinsxu)
2. Agenda • History and Community • Technical Walkthrough • Kubernetes Operator ◦ to run cloud-native stateful app
3. A little about PingCAP • Founded in April 2015 by 3 infrastructure engineers • Created and maintains TiDB, TiKV • Offices throughout North America and China
4. PingCAP.com Community GitHub Stars • TiDB: 18,000+ • TiKV: 5,000+ Contributors • TiDB: 250+ • TiKV: 150+
5. PingCAP.com TiKV -> Incubating Status!
6. PingCAP.com Who’s Using TiDB? 300+ Deployments
7. PingCAP.com Mobike + TiDB • 200 million users • 200 cities • 9 million smart bikes • ~35 TB in TiDB
8. Technical Walkthrough
9. Technical Inspiration TiDB is a NewSQL database that speaks the MySQL protocol It is not based on the MySQL source code It is an ACID/strongly consistent database The inspiration is Google Spanner + F1 It separates SQL processing and storage into separate components Both of them are independently scalable The SQL processing layer is stateless It is designed for both Transaction and Analytical Processing (HTAP)
10. Use Cases 1. Approaching the maximum size for MySQL on a single server. Debating whether or not to shard. 2. Already sharded MySQL, but having a hard time doing analytics on up-to-date data.
11. TiDB TiDB Region 1 L TiKV Node 1 Region 2 Region 3 Region 4 Region 2 L TiKV Node 3 Region 3 Region 4 L Region 1 Region 4 TiKV Node 2 Region 3 L Region 2 Region 1 TiKV Cluster PD Cluster TiDB Core
12. TiDB HTAP: Row + Column Storage Spark Cluster TiDB TiDB Region 1 TiKV Node 1 Region 2 Region 3 Region 4 Region 2 TiKV Node 3 Region 3 Region 4 Region 1 Region 4 TiKV Node 2 Region 3 Region 2 Region 1 TiFlash Node 2 TiFlash Extension Cluster TiKV Cluster TiSpark Worker TiSpark Worker TiFlash Node 1
13. Kubernetes -- Operator
14. Kubernetes on 1 slide • Pod: Smallest unit of work (analogous to a node, an instance) ◦ can hold 1 or more containers ◦ it dies? it dies. just get a new pod (cattle v. pet) • Deployment: API for a Pod ◦ great for stateless apps, like RESTful API ◦ use pod resources, declare # of replicas, rolling deployment, etc. • StatefulSet: Another API for a Pod, designed for stateful apps ◦ maintain order, “sticky” identity, no interchangeability, etc.
16. Operator History • Operator pattern pioneered by CoreOS...now Red Hat...now IBM • Introduced in 2016, Operator Framework in 2018 ◦ First 2: etcd operator, Prometheus operator • TiDB Operator (2017); Predated Operator Framework
17. Why Do We (As in TiDB) Care? • Manage multiple clusters (multi-tenancy) • Safe scaling (up or down, in or out) • Use different types of Network or Local Storage (different performance) • Automatic monitoring • Rolling updates • Automatic failover • *Multi-Cloud* (as long as it has k8s)
18. Why Should YOU Care? • Manages stateful applications: ◦ databases, caches, monitoring system, etc. • Encodes application domain knowledge ◦ Extension of your SRE team • Kubernetes-enabled Hybrid / Multi-Cloud • Growing popularity in database community: ◦ https://thenewstack.io/databases-operators-bring-stateful-workloads-to-kubernet es/
19. PingCAP.com High Level Resources => Controllers => Clusters
20. Resources -- CRD • Custom Resource Definition (CRD): ◦ An application-specific YAML file ◦ End user writes the domain operation logic in CRD ◦ Simple to implement and deploy • (There is another way): ◦ API Aggregation: ▪ More control, more powerful but… ▪ Hard to deploy, not well-supported by k8s engines
21. Cluster State -- StatefulSet StatefulSet... • Guarantees ordering and uniqueness of pods ◦ pd -> tikv -> tidb • Gives “sticky” identity -- network and storage • *No* interchangeable pods ◦ always map the same volume to the same pod • Stable since Kubernetes 1.9
22. TiDB TiDB Region 1 L TiKV Node 1 Region 2 Region 3 Region 4 Region 2 L TiKV Node 3 Region 3 Region 4 L Region 1 Region 4 TiKV Node 2 Region 3 L Region 2 Region 1 TiKV Cluster PD Cluster TiDB Core
23. How TiDB manages state -- Custom Controller Spec: component: image: replicas: ... Status: image replicas state CRD (provided by user) Custom Controller Cluster State
24. How TiDB manages state apiVersion: pingcap.com/v1alpha1 kind: TidbCluster metadata: name: demo spec: pd: image: pingcap/pd:v2.1.0 replicas: 3 requests: cpu: “4” memory: “8Gi” … tikv: image: pingcap/tikv:v2.1.0 … status: tikv: stores: “5”: podName: demo-tikv-2 state: Up ...
25. “Smooth” Operator
26. PingCAP.com All Open Sourced https://github.com/pingcap/tidb-operator/blob/ master/docs/user-guide.md
27. PingCAP.com On GCP Marketplace
28. PingCAP.com TiDB, Managed As A Cloud Service Early Access: https://www.pingcap.com/tidb-cloud/
29. Thank You @kevinsxu pingcap.com github.com/pingcap tikv.org