ashwanth kumar @_ashwanthkumar Building Distributed RocksDB 

overview - What’s RocksDB - Why Distributed RocksDB - Lessons from running it in production 

rocksdb 

From facebook Fast persistent KV store Server Workloads Embeddable Optimized for SSDs rocksdb Fork of LevelDB Modelled after BigTable LSM Tree based SST files Written in C++ 

Simple C++ API Has bindings in C Java Go Python rocksdb 

but why build a distributed database? 

data system paradigms 

sum of numbers 

sum of numbers select col from table; 

sum of numbers select col from table; <> 

sum of numbers select col from table; <> 

sum of numbers (attempt 2) 

sum of numbers select sum(col) from table; 

sum of numbers select sum(col) from table; 

sum of numbers <> select sum(col) from table; 

data shipping function shipping select sum(col) from table; select col from table; 

but why ship functions? 

data locality for low latency / data intensive applications 

recursive reduction aggregations in distributed systems 

recursive reduction select sum(col) from table; 

recursive reduction select sum(col) from table; 

recursive reduction select sum(col) from table; 

recursive reduction select sum(col) from table; <> 

- sum / multiplication - (sorted) top-K elements - operations on a graph - eg. link reach on twitter graph - function should be associative and optionally commutative recursive reduction 

rocksdb @indix 

- Serving our API in production for 2+ years - Search on hierarchical documents - Dynamic fields didn’t scale well on Solr - Brand / Store / Category Counts for a filter - Price History Service - More than a billion prices and serve online to REST queries rocksdb @indix 

- Stats (as Monoids) Storage System - All we want was approximate aggregates real-time - HTML Archive System - Stores ~120TB of url and timestamp indexed HTML pages - Real-time scheduler for our crawlers - Finds out which of the 20 urls to crawl now out of 3+ billion urls - Helps crawler crawl 20+ million urls everyday rocksdb @indix 

suuchi toolkit for building distributed function shipping applications github.com/ashwanthkumar/suuchi 

ops lessons running rocksdb in production 

leveled compaction 

https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide Tuning RocksDB for Read amplification Write amplification Space amplification compaction + other tuning 

backup & restore - Incremental backups - Store backups in S3 - Sometimes high CPU during backups - Restore happens outside the app lifecycle https://github.com/indix/rocks 

questions? https://github.com/ashwanthkumar/distributed-rocksdb-talk