of scaling in database § Scaling up § Scaling out § NoSQL workloads and scalability model § Core data operations, indexing and querying § Homogenous HW scaling § Introducing multi-‐dimensional scaling § Isolation of workloads through a services architecture § Independent HW scaling § Q & A
§ Disclaimer: The views expressed in this presentation are our own and do not necessarily reflect the views of Couchbase § Most of the content in this presentation is originally created by Anil, Product Manager, Couchbase § Thanks Anil, for allowing me to re-‐use your slides J 3
§ Clarence J M Tauro – [email protected] - Senior Instructor, Couchbase - ~11 Years Professional Teaching and Consulting Experience - Worked at Pivotal – Instructor/Consultant for Spring/Spring Security/Spring Web/Enterprise Integration with Spring/Spring JMS/Spring Web/Spring Batch, Pivotal Hadoop/Cloud Foundry - PhD in Computer Science from Christ University [thesis accepted] - Hard-core Dog lover 4
& Cons Pros § Can result in major performance improvement § Machines can now support having many cores and terabytes of RAM Cons § Expensive § Requires downtime § Performance bounded – at some point the database engine itself becomes the bottleneck § Limited in scalability and elasticity
& Cons Pros § Simple and easily scalable § Data evenly split across cluster of nodes § Scales linearly with throughput § Highly available § No single point of failure Cons § Not great for all workloads – data, index and query
database, many workloads § Core data processing: GETs & SETs for a given key § Indexing: Index maintenance and lookups § Querying: Combine index and data with complex just-‐in-‐time data re-‐shaping, ordering, grouping, aggregations, and more Varying resource requirements -‐ CPU, RAM, I/O, Network Varying methods to optimize latency & throughput for each
Homogenous Scaling § Each node get a slice of the workload § Simple to do… But... § Workloads compete and interfere with each other § Can’t fine tune each workload § Core Data operation are partition-‐able so great with wider fan-‐out § Indexing and queries aren’t always partitionable, so worse with wider fan-‐out Index Workload Couchbase Cluster Query Workload Data Workload node1 node8
is Multi-‐Dimensional Scalability? MDS is the architecture that enables independent scaling of data, query and indexing workloads. Index Service Couchbase Cluster Query Service Data Service node1 node8
node8 Modern Architecture § Isolated Service for minimized interference § Independent “zones” for Query, Index and Data Services Minimize indexing and query overhead on core key-‐value operations. Index Service Query Service Data Service
Independent Scalability for Best Computational Capacity per Service Heavier indexing (index more fields) : scale up index service nodes More RAM for query processing: scale up query service nodes Couchbase Cluster node1 node8 node9 Data Service Index Service Query Service
STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
STORAGE Couchbase Server 1 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
Manager Data Service Projector & Router New Services in Couchbase Server 4.0 Query Service Index Service Supervisor Index maintenance & Scan coordinator Index#2 Index#1 Query Processor cbq-‐engine Bucket#1 Bucket#2 DCP Stream Index#4 Index#3 ... Bucket#2 18093 11211 18901 Managed Cache
unprecedented control of scalability with Couchbase Server § Separate out competing workloads to independent services § Independently scale each service “zone” within the cluster § Couchbase Server with MDS maximizes scalability and performance § Improves scale and performance to degrees not possible with other NoSQL or big-‐data engines on premise or in the cloud § Improved price/performance and squeezes more performance and throughput for mission-‐critical systems
on Couchbase: http://training.couchbase.com CD220: Developing Couchbase NoSQL Applications Oct 20 – Oct 23 2015 CS300: Couchbase NoSQL Server Administration Nov 17 – Nov 20 Enroll Today!