Slide 1

Slide 1 text

ScalarDL Technical Overview 1 Oct, 2022 1

Slide 2

Slide 2 text

© 2022 Scalar, inc. Confidential ScalarDL: Byzantine Fault Detection Middleware 2 Scalar DB Storages / Databases Applications OS Scalar DL Ledger Auditor Client SDK Contract Function Scalable and Practical Byzantine Fault Detection Middleware for Transactional Database Systems

Slide 3

Slide 3 text

© 2022 Scalar, inc. Confidential Key Characteristics in ScalarDL 3 Tamper-evident Scalable Correct Database-agnostic Cloud-agnostic Byzantine faults can be detected as long as N > f Performance and availability can be linearly scaled No intermediate states are visible, and data is always up-to-date No dependency on particular database products No dependency on particular public clouds N: # of administrative domains in a database system, f: # of faulty domains Cassandra, Amazon DynamoDB, Azure Cosmos DB, and JDBC-supported major RDBMSs (MySQL, PostgreSQL, Oracle Database) are supported. AWS, Azure, EKS, and AKS are supported. ACID-compliant and strong consistent database system (i.e., strict serializable) DVi = DV / N・RF (RF: Replication Factor << N)

Slide 4

Slide 4 text

© 2022 Scalar, inc. Confidential ScalarDL System Architecture 4 Pattern 1 Pattern 1: Ledger only. ScalarDL provides data integrity in a database system. But, it does not guarantee to detect Byzantine faults. Pattern 2: Ledger+Auditor. ScalarDL guarantees to detect Byzantine faults. (as long as either Ledger or Auditor is correct.) Primary ScalarDL Ledger Primary Databases Administrative domain 1 Database System Scalar DL Client SDK Application programs Primary Secondary ScalarDL Ledger Primary Databases Administrative domain 1 Database System ScalarDL Auditor Secondary Databases Administrative domain 2

Slide 5

Slide 5 text

© 2022 Scalar, inc. Confidential ScalarDL: Ledger 5 User (Client) Java Contract Signature Private Key ScalarDL Ledger function invoke() { if (accounts[0].data.balance < args.val) { throw new Error(“not enough balance”); } accounts[0].data.balance -= args.val; accounts[1].data.balance += args.val; results = { … }; } Payment Contract Request : (contract, args, sig) Asset ID Age Data (before) Data (after) Sigs Func (ref) Args Hash A 1 { } { balance = 100, …} charge { val = 100 } B 1 { } { balance = 200, …} charge { val = 200 } A 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 90, …} payment { val = 10 } SN = Func (SN-1 , Args) Deterministic & TE TE If S0 is TE ⇒ SN is TE Public Key B 2 { A: {balance = 100}, B: {balance = 200} …} { balance = 210, …} payment { val = 10 } H(A1) H(B1) State hash chain Data integrity of ledger entries: Arguments { accounts = [“A”, “B”], val = 10, …} * includes other accounts data Argument

Slide 6

Slide 6 text

© 2022 Scalar, inc. Confidential ScalarDL: Auditor 6 managed by a different AD Execute Asset Proof Request Proof Asset Proof T Compare to detect Byzantine faults Auditor manages the proofs of execution to recompute assets without trusting the integrity of data given from Ledger (1 patented, 1 filed, accepted by PVLDB) User (Client) ScalarDL Ledger ScalarDL Auditor See our PVLDB paper for more details

Slide 7

Slide 7 text

© 2022 Scalar, inc. Confidential Key Difference between Public Blockchains and Scalar DL 7 Public Blockchains (Ethereum …) ScalarDL • History diverges in normal cases – It diverges even if no malicious activities ⇒ Impossible to guarantee finality • History diverges only in abnormal cases – If it diverges, it is caused by malicious activities ⇒ Possible to guarantee finality IEEE ICDCS Workshop’16

Slide 8

Slide 8 text

© 2022 Scalar, inc. Confidential Key Difference between BFT/BFD SMR (Private Blockchains*) and Scalar DL 8 BFT/BFD SMR, Private Blockchains ScalarDL Data is totally ordered ⇒ Hard to parallelize/scale Data is partially ordered ⇒ Easy to parallelize/scale Sequential processing is required Parallelizable * Private blockchains are thought to be variants of techniques called Byzantine-fault tolerance state machine replication (BFT SMR) See our PVLDB paper for more details

Slide 9

Slide 9 text

© 2022 Scalar, inc. Confidential Unique Features 1 : ACID Nested Contract Execution • Multiple contracts can be executed atomically 9 Contract 1 Transaction Tamper-evident ledger Contract 2 Contract 3 Atomic execution

Slide 10

Slide 10 text

© 2022 Scalar, inc. Confidential Unique Features 2 : User-defined Function (patented) • Function is a business logic for creating deletable and easy to search records • Remedy the downsides of ledger structure and tamper-evidence property 10 Mutable database Function Contract Transaction Tamper-evident ledger Atomic execution Business logic Evidence that business logic is executed

Slide 11

Slide 11 text

© 2022 Scalar, inc. Confidential Benefits of User-defined Function 11 Not deletable Not searchable flexibly Not tamper-evident Inconsistent Tamper-evident Deletable Searchable flexibly Consistent Applications Applications BFT/BFD (e.g., Blockchain) DB Applications DB BFT/BFD (e.g., Blockchain) Applications log log Mutable records Tamper-evident records ScalarDL Ledger Scalar DL Auditor Replicated in a decentralized way

Slide 12

Slide 12 text

© 2022 Scalar, inc. Confidential Benchmark Results (YCSB、TPC-C) • Benchmarked Systems: ScalarDL、BFD-SMR (PeerReviewTx) [SOSPʼ07] • Databases: PostgreSQL (1 node / AD), Cassandra (3 nodes / AD) • YCSB-F: 100M records, 1RMW (1tx: 1read 1write) • TPC-C: 100 warehouses, NewOrder-Payment (NP) • Environment: AWS. 2 ADs. c5d.4xlarge for each database instance. 12 YCBS-F (read-modify-write) TPC-C (NP) ScalarDL scaled its throughput as client threads increased. See our PVLDB paper for more details

Slide 13

Slide 13 text

© 2022 Scalar, inc. Confidential Scalability (TPC-C) 13 ScalarDL achieved near-linear scalability. See our PVLDB paper for more details

Slide 14

Slide 14 text

© 2022 Scalar, inc. Confidential Use Cases : Where to use ScalarDL 14 • Data integrity is required – Data can not be altered or deleted maliciously • Scalability is required – Many requests from many end-users or devices • There is a main organization – And there are a few auditors • Byzantine fault detection is acceptable Digital evidence / Database forensics Our focuses:

Slide 15

Slide 15 text

© 2022 Scalar, inc. Confidential For More Details • ScalarDL internal mechanism, fundumental differences between ScalarDL and other technologies/producuts. – https://dl.acm.org/doi/abs/10.14778/3523210.3523212 – PVDLB is a top-tier international conference in database and data engineering. • GitHub (Usage) – https://github.com/scalar-labs/scalardl • Engineering Blog – https://medium.com/scalar-engineering 15