Slide 1
Slide 1 text
Momento Confidential
Beyond the Server: Crafting TiDB
Serverless and Chat2Query for Developers
Max Liu
Co-founder and CEO, PingCAP
Slide 2
Slide 2 text
Momento Confidential
Agenda
● Brief introduction of PingCAP & TiDB
● The journey of building TiDB Serverless
● Example: AI powered OSSInsight.io on TiDB Serverless
● TiDB Serverless key capabilities
● Developer resources
Slide 3
Slide 3 text
Momento Confidential
Silicon Valley
Amsterdam
Beijing
Tokyo
The creators of TiDB
Founded in 2015
Based in Sunnyvale, CA
More than 550+ employees globally
Over $600M in total funding
Mission:
Empower engineers and enterprises
to innovate with speed, agility,
and scale
Singapore
Slide 4
Slide 4 text
Momento Confidential
The most advanced,
open source,
distributed SQL database
for modern applications.
Scalable. Versatile. Titanium (Ti) Reliable.
35K+ GitHub Stars 800+ Contributors
Trusted By
FlipKart
Pinterest
Slide 5
Slide 5 text
Momento Confidential
Where Does the Journey of
TiDB Serverless Begin?
Slide 6
Slide 6 text
Momento Confidential
“How about offering a free
database for every developer?”
Is it possible?
Is it affordable?
Let's do some math
Slide 7
Slide 7 text
Momento Confidential
26.9 million
Well, A big surprise?
Slide 8
Slide 8 text
Momento Confidential
26.9 million clusters
Even we provide a tiny tidb (aws t2.nano) cluster.
For each year it will cost:
26.9 million * 4 vCPUs (3 storage node+ 1 compute
node + 1 placement driver) * (0.003 * 365 * 24)
= $2.83 billion / year
Really? what a shocking number!
Slide 9
Slide 9 text
Momento Confidential
ELB ~ 30$ for each cluster/year, we need another
$807 million.
We need total $3.6 billion/year.
Oh, I forget to mention ELB, storage, HA and more…
Slide 10
Slide 10 text
Momento Confidential
ELB ~ 30$ for each cluster/year, we need another
$807 million.
We need total $3.6 billion/year.
Let’s forget about the cost of the storage, monitoring system
and so on, that’s too scary to calculate.
What’s more, AI transforms individuals into developers. The
number could be 10x larger in the next few years.
Oh, I forget to mention ELB, storage, HA and more…
Slide 11
Slide 11 text
Momento Confidential
So, We gave up
Well, that’s too hard
Slide 12
Slide 12 text
Momento Confidential
We gave up the traditional architecture
Well, that’s too hard
Slide 13
Slide 13 text
Momento Confidential
Why Not Shut Down the Cluster When It's Not in Use?
Well, let’s try one more step
Slide 14
Slide 14 text
Momento Confidential
Let's rethink the architecture
Slide 15
Slide 15 text
Momento Confidential
Yes! Serverless architecture
Slide 16
Slide 16 text
Momento Confidential
To Make it possible
● Reuse as many spot instances as we can. ~70% off discount
● 80/20 theory. 80% of the cluster are not active. 80% off
Quick math in your mind: $3.6 billion/year * 30% * 20%
One more thing
Slide 17
Slide 17 text
Momento Confidential
ChatGPT works
Slide 18
Slide 18 text
Momento Confidential
Load
balancer
Isolated
SQL Layer
Shared
Disk Cache Layer
Object storage
Gateway Gateway
Row
Engine
Row
Engine
Shared Storage( Amazon S3, Google Cloud Storage…)
Resource
Pool
Virtual Cluster - Tenant n
Micro
Services Compression Analyze DDL
Virtual Cluster - Tenant 1
Slide 19
Slide 19 text
Momento Confidential
I know it!
The classical principle:
Separation of compute and storage.
Slide 20
Slide 20 text
Momento Confidential
Yeah! That’s not enough
Separation of compute and compute.
?
Slide 21
Slide 21 text
Momento Confidential
Yeah! That’s not enough
Separation of lightweight compute and heavy compute.
Not all computations are equal
Slide 22
Slide 22 text
Momento Confidential
Tell me more about
● lightweight compute
○ Read/Write a few records or indexes
○ Small OLTP transactions
● heavy compute
○ Adding index
○ Change/Remove a column from a table
○ Collect table statistics(Analyze table)
○ Scan a whole table or lots of records
○ Compression
○ And more
Slide 23
Slide 23 text
Momento Confidential
Tell me more about
● lightweight compute
○ Read/Write a few records or indexes
○ Small OLTP transactions (e.g. primary key lookup)
● heavy compute (offload to dynamic micro services)
○ Adding index
○ Change/Remove a column from a table
○ Collect table statistics(Analyze table)
○ Scan a whole table or lots of records
○ Compression
○ And more
Slide 24
Slide 24 text
Momento Confidential
Results of Lightweight compute
● 2x better cpu utilization ( Saved 50% money, again! )
● Lower latency for lightweight compute, less spike
Now it’s $100 million per year. Wow, it’s close to $3.7 per developer
per year.
Slide 25
Slide 25 text
Momento Confidential
It is possible! We made it.
Wow, It’s hard to believe.
Innovation changed everything
Slide 26
Slide 26 text
Momento Confidential
That’s TiDB Serverless
Slide 27
Slide 27 text
Momento Confidential
We didn’t stop there
Slide 28
Slide 28 text
Momento Confidential
Separation of compute and compute
● Better user experience ( after offload heavy compute)
○ 100x Faster adding index (WIP)
○ Fast table statistics collection (Done. leverage the distributed
architecture)
○ Backup large clusters in seconds (Done. e.g., hundreds of TBs,
with the help of S3)
● Higher Durability & Availability
○ Leverage the durability of S3
○ Recover from a failed server faster by loading data from S3
concurrently
Slide 29
Slide 29 text
Momento Confidential
TiDB Serverless Architecture
Offload Compute
:
Virtual Cluster - Tenant 1
SQL SQL
Storage
S3 / Blob / Cloud Storage
Virtual Cluster - Tenant n
SQL SQL
DDL worker
MPP Compute service
Data Ingestion service
Storage
Storage
Isolated
SQL Layer
(For
OLTP)
Storage
Cache
Layer
(For OLTP)
Shared
Storage
Pool
…
Slide 30
Slide 30 text
Momento Confidential
Example: OSSInsight.io
https://github.com/pingcap/ossinsight
OSSInsight is a powerful open-source tool that gives detailed and up-to-date
information about the open source world by analyzing over 6 billion GitHub
events.
OSS Insight's Data Explorer makes exploring GitHub data easy. Simply ask a
question in natural language, and Data Explorer will create a SQL, execute the
SQL, and show you the results visually.
Slide 31
Slide 31 text
Momento Confidential
Slide 32
Slide 32 text
Momento Confidential
OSSInsight on TiDB Serverless
Data Volume - 12 TBs, over 6 Billion rows of
data - all growing in real-time.
Random Mixed Workloads - Serving online
transactions plus analytical queries
generated by AI
Unpredictable Traffic Spikes - Featured on
HackerNews with 7x traffic in one day.
72.7%
Cost Reduction
Zero
Manual Intervention
During Spikes
5X
Development Efficiency
for LLM Data Apps
https://github.com/pingcap/ossinsight
Slide 33
Slide 33 text
Momento Confidential
Unpredictable Traic Spikes
https://github.com/pingcap/ossinsight
Slide 34
Slide 34 text
Momento Confidential
TiDB Serverless: Key capabilities
Slide 35
Slide 35 text
Momento Confidential
MySQL Compatible
Use a familiar language including MySQL drivers and ORMs.
Slide 36
Slide 36 text
Momento Confidential
TiDB Serverless
Lalency
Time
Workload
Peak
100
Time
PVs
Automatically scale in and out to meet workload
demands in real-time
Resource Pool
Real-time elasticity
Slide 37
Slide 37 text
Momento Confidential
What is
scalability?
And why
should we
care?
Slide 38
Slide 38 text
Momento Confidential
Sysbench
Scale out from 0 to
~1.3M QPS
After the peak, scale
back to zero.
Within 30 mins
Without any
configuration or
tuning 🎉
Slide 39
Slide 39 text
Momento Confidential
12,491ms vs 187.5ms
67x Faster with HTAP
Versatile to run Hybrid Transactional and Analytical (HTAP)
workloads with smart optimizer and two distinct storage layers
Slide 40
Slide 40 text
Momento Confidential
Pay as you go
Start free, then only pay for the storage and data processing you
use - and never overspend with a spending limit
Slide 41
Slide 41 text
Momento Confidential
AI-Assisted Enhancements
With OpenAI powered features like Chat2Query and
TiDB-Bot, it’s designed to boost developer productivity
Slide 42
Slide 42 text
Momento Confidential
Developer Resources
TiDB Cloud CLI
VS Code
@TiDB_Developer
Partner integrations to meet
where developers are
Comprehensive docs to
learn about TiDB Cloud
Resource hub and online
courses for developers
Slide 43
Slide 43 text
Momento Confidential
[NEW] Serverless Driver for Edge Computing!
https://docs.pingcap.com/tidbcloud/serverless-driver
Slide 44
Slide 44 text
Momento Confidential
AJU Center of Independent Living
& TiDB Serverless
AJU migrated some websites below (built with Wordpress) to TiDB Serverless:
1) こころの絆創膏 2) ゲートキーパー研修 3) なごや食育ひろば (coming soon)
AJU Center for Independent
Living is an association run
by colleagues of wheelchair
users in Aichi Prefecture.
Slide 45
Slide 45 text
Momento Confidential
THANK YOU.
https://pingcap.co.jp/tidb-cloud/
Sign up Today!
No creditcard required.