TiDB: Overview, New Features, Multi-Cloud

This keynote talk was delivered with Li Shen, VP of Engineering at PingCAP, at Percona Live Europe 2018, to provide a technical overview of TiDB, new features in version 2.1, and multi-cloud deployment using a Kubernetes Operator.

Transcript

1. TiDB: Overview, New Features, Multi-Cloud Shen Li, Kevin Xu PingCAP
2. !2 What is Ti(tanium) DB? • NewSQL (RDBMS at Scale) • MySQL Compatible (5.7) • Hybrid OLTP + OLAP Workloads • Multi-Cloud
3. !3 Product: the TiDB Platform • TiDB Platform: • TiDB (stateless SQL layer) • TiKV (distributed transactional key-value store on RocksDB) • TiSpark (Apache Spark plug-in to TiKV) • Open Sourced from Day 1 • GA 2.0: April 2018
4. Community !4 • Stars: • TiDB: 15,600+ • TiKV: 3,800+ • Contributors: • TiDB: 210+ • TiKV: 105+
5. Recent News !5
6. TiDB Overview
7. Platform Architecture !7 Monitoring & Observability: Prometheus + Grafana
8. TiKV: the Storage Foundation !8
9. TiDB: the (My)SQL Layer !9
10. 300+ Companies in Production !10
11. MySQL Compatible Tools and Migration
12. Tools !12 • TiDB-DM (Data Migration) • TiDB-Lightning • TiDB-Binlog
13. TiDB-DM !13 • Why? To support migrating data from MySQL/MariaDB to TiDB • Full data migration + binlog replication • Features: • Data migration job management • Monitoring system • Sharded table merge migration • Data transformation
14. DM Architecture !14
15. Sharded Table Merge !15
16. Data Transformation !16
17. TiDB-Lightning !17 • Why? Enables fast, full import of SQL dump -> TiDB cluster • Fast: • Can import > 100 GB / hour on production grade hardware • Full Import: • Import the entire database once from scratch; not incremental import • Suitable for migrating an existing database to a new TiDB cluster • TiDB Cluster: • Lightning interacts with TiDB’s internal structures • Only works with TiDB cluster
18. TiDB-Lightning Architecture !18
19. TiDB-Binlog !19 • Why? Provides real-time data synchronization and backup • Synchronizing downstream data platform in real-time to: • Hot backup • MySQL slave • OLAP data warehouses • Writing incremental backup files • Event subscription & trigger
20. TiDB-Binlog Architecture !20
21. Migration: in and out of TiDB !21
22. Sneak Peek of TiDB 2.1
23. TiDB 2.1 - SQL Layer Smarter SQL optimizer • More efficient plan • Dynamic stats update based on query feedback Faster aggregation operator • New aggregation framework • Multi-thread hash aggregation operator 50% faster OLTP point query !23 Faster online DDL • Run DDL concurrently • Faster “Add Index” operation Improved debugging & diagnosis • More readable EXPLAIN result • “Explain Analyze” statement • Expose cluster information through HTTP API
24. TiDB 2.1 - Storage Engine (aka TiKV) Raft learner • Improve cross-DC data availability Hotspot scheduling • Automatically distribute hotspots to more idle instances Batch region-splits • Avoid large region formation under heavy write workloads 2X read performance improvement !24
25. 1.0 vs 2.0 on TPC-H Scale 50 !25
26. 2.0 vs 2.1 on TPC-H Scale 50 !26
27. 2.1 vs 2.0 on Sysbench Point Select !27
28. TiDB Cloud
29. TiDB As A Service !29 • Fully Managed by PingCAP • Upgrades, Patching, Online DDL with No Downtime • Backup and Restore • Multi-Cloud Deployment https://www.pingcap.com/tidb-cloud/
30. TiDB Operator !30
31. GCP Marketplace !31
32. See it to believe it !32 • PingCAP Booth #6 • Demo by Greg Weber TODAY at 1:20PM
33. Oh…one more thing, TiDB Academy !33 https://www.pingcap.com/tidb-academy/
34. Thank you! @PingCAP, @shenli3514, @kevinsxu