NewSQL: what, when and how

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Cover Title Goes here Month XX, 20XX NewSQL: What, When and How Nikolay Stoitsev @ Uber Engineering Sofia

Slide 3

Slide 3 text

RDBMS → NoSQL → NewSQL

Slide 4

Slide 4 text

RDBMS

Slide 5

Slide 5 text

ACID Transactions

Slide 6

Slide 6 text

ACID Transactions

Slide 7

Slide 7 text

ACID Transactions

Slide 8

Slide 8 text

ACID Transactions

Slide 9

Slide 9 text

ACID Transactions

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

DC1 DC2

Slide 12

Slide 12 text

DC1 DC2

Slide 13

Slide 13 text

Availability

Slide 14

Slide 14 text

Consistency

Slide 15

Slide 15 text

PSQL can really scale

Slide 16

Slide 16 text

2K Database size 10+TB Rows fetched per second 500 Rows updated per second Our cluster

Slide 17

Slide 17 text

Vertical scaling

Slide 18

Slide 18 text

“Bigger servers don’t exist”

Slide 19

Slide 19 text

Sharding Data 1. Migrating to NoSQL 2.

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

NoSQL

Slide 24

Slide 24 text

Vertical scaling

Slide 25

Slide 25 text

Sharding and Replication

Slide 26

Slide 26 text

F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A distributed storage system for structured data. ACM Trans. Comput. Syst., 26:4:1–4:26, June 2008.

Slide 27

Slide 27 text

G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: amazon’s highly available key-value store. SIGOPS Oper. Syst. Rev., 41:205–220, October 2007.

Slide 28

Slide 28 text

Apache HBase Apache Cassandra

Slide 29

Slide 29 text

We picked Apache Cassandra

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Configurable Consistency

Slide 32

Slide 32 text

No ACID Transactions

Slide 33

Slide 33 text

NewSQL

Slide 34

Slide 34 text

Scalability of NoSQL + ACID Transactions

Slide 35

Slide 35 text

Data is partitioned and replicated

Slide 36

Slide 36 text

https://ai.google/research/pubs/pub27898

Slide 37

Slide 37 text

True Time API

Slide 38

Slide 38 text

Decetralized transaction management

Slide 39

Slide 39 text

Tablet 1 Tablet 2 Tablet 3

Slide 40

Slide 40 text

Leader Follower Follower Paxos

Slide 41

Slide 41 text

Leader Follower Follower Read

Slide 42

Slide 42 text

Leader Follower Follower Read Can I read?

Slide 43

Slide 43 text

Leader Follower Follower Write

Slide 44

Slide 44 text

Leader 1 Leader 2 Distributed Write Transaction Manager

Slide 45

Slide 45 text

https://github.com/cockroachdb/cockroach

Slide 46

Slide 46 text

https://www.cockroachlabs.com/blog/living-without-atomic-clocks/

Slide 47

Slide 47 text

Apache Ignite

Slide 48

Slide 48 text

Used at Uber for in-house framework for massive distributed apps

Slide 49

Slide 49 text

In-memory distributed database

Slide 50

Slide 50 text

Distributed data structures

Slide 51

Slide 51 text

SQL Support

Slide 52

Slide 52 text

Brings the computation to the data

Slide 53

Slide 53 text

https://dzone.com/articles/apache-ignite- transactions-architecture-ignite-per

Slide 54

Slide 54 text

https://ignite.apache.org/features/sql.html

Slide 55

Slide 55 text

Which database to pick?

Slide 56

Slide 56 text

No one size fits all database

Slide 57

Slide 57 text

RDBMS Easy to run it Has transactions Can scale Provides availability NoSQL NewSQL Can scale Has Transactions

Slide 58

Slide 58 text

Thanks! Nikolay Stoitsev Uber Engineering Sofia