Scaling PHP Applications with Couchbase

Slide 1

Slide 1 text

Scaling PHP Applica-ons with Couchbase Michael Nitschinger

Slide 2

Slide 2 text

Who‘s this guy? •  SDK Engineer at Couchbase   Maintaining •  Oﬃcial Java SDK •  Spymemcached •  Spring-‐Data-‐Couchbase •  Former Lithium Core Contributor •  @daschl on TwiGer

Slide 3

Slide 3 text

RDBMS are not Enough?

Slide 4

Slide 4 text

Instagram gained nearly 1 million users overnight when then expanded to Android Growth is the New Reality

Slide 5

Slide 5 text

Draw Something Viral Growth 19 17 15 13 11 9 7 5 3 3/1 28 26 24 22 20 18 16 14 12 10 8 2/6 Draw Something by OMGPOP Daily Ac)ve Users (millions) 21 2 4 6 8 10 12 14 16

Slide 6

Slide 6 text

How do you take this growth? RDBMS is good for many thing, but hard to scale RDBMS Scales Up Get a bigger, more complex server Users ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance RelaUonal Database Web/App Server Tier System Cost ApplicaUon Performance Won’t scale beyond this point

Slide 7

Slide 7 text

Scaling out RDBMS • Run Many SQL Servers • Data could be shared -‐ Done by the applicaUon code • Caching for faster response Ume Web/App Server Tier Memcached Tier MySQL Tier

Slide 8

Slide 8 text

Scaling out ﬂa]ens the cost and performance curves NoSQL Database Scales Out Cost and performance mirrors app Uer Users NoSQL Distributed Data Store Web/App Server Tier ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance ApplicaUon Performance System Cost NoSQL Technology Scales Out

Slide 9

Slide 9 text

Couchbase

Slide 10

Slide 10 text

Couchbase Open Source Project •  Leading NoSQL database project focused on distributed database technology and surrounding ecosystem •  Supports both key-‐value and document-‐oriented use cases •  All components are available under the Apache 2.0 Public License •  Obtained as packaged so`ware in both enterprise and community ediUons.

Slide 11

Slide 11 text

Couchbase Handles Real World Scale

Slide 12

Slide 12 text

Easy Scalability Consistent High Performance Always On 24x365 Grow cluster without applicaUon changes, without downUme with a single click Consistent sub-‐millisecond read and write response Umes with consistent high throughput No downUme for so`ware upgrades, hardware maintenance, etc. Flexible Data Model JSON document model with no ﬁxed schema. Couchbase Server Core Principles

Slide 13

Slide 13 text

New in 2.0+ JSON support Indexing and Querying Cross data center replica-on Incremental Map Reduce JSON JSON JSON JSON JSON

Slide 14

Slide 14 text

Architecture & Opera-ons

Slide 15

Slide 15 text

Couchbase Server Architecture Heartbeat Process monitor Global singleton supervisor ConﬁguraUon manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replicaUon manager hGp REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 -‐ 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data Manager Cluster Manager

Slide 16

Slide 16 text

Couchbase Server Architecture ReplicaUon, Rebalance, Shard State Manager REST management API/Web UI 8091 Admin Console Erlang /OTP 11210 / 11211 Data access ports Object-‐managed Cache Mul--‐threaded Persistence Engine 8092 Query API Query Engine hGp Data Manager Cluster Manager

Slide 17

Slide 17 text

3 3 2 Single node -‐ Couchbase Write Opera-on Managed Cache Disk Queue Disk ReplicaUon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node

Slide 18

Slide 18 text

COUCHBASE SERVER CLUSTER Cluster-‐wide Basic Opera-on •  Docs distributed evenly across servers •  Each server stores both ac-ve and replica docs Only one server acUve at a Ume •  Client library provides app with simple interface to database •  Cluster map provides map to which server doc is on App never needs to know •  App reads, writes, updates docs •  Mul-ple app servers can access same document at same -me User Conﬁgured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 ACTIVE Doc 4 Doc 7 Doc Doc Doc SERVER 2 Doc 8 ACTIVE Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9

Slide 19

Slide 19 text

Add Nodes to Cluster •  Two servers added One-‐click opera-on •  Docs automa-cally rebalanced across cluster Even distribuUon of docs Minimum doc movement •  Cluster map updated •  App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Conﬁgured Replica Count = 1

Slide 20

Slide 20 text

Fail Over Node REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc •  App servers accessing docs •  Requests to Server 3 fail •  Cluster detects server failed Promotes replicas of docs to acUve Updates cluster map •  Requests for docs now go to appropriate server •  Typically rebalance would follow Doc Doc 1 Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Conﬁgured Replica Count = 1 COUCHBASE SERVER CLUSTER

Slide 21

Slide 21 text

COUCHBASE SERVER CLUSTER Indexing and Querying User Conﬁgured Replica Count = 1 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 •  Indexing work is distributed amongst nodes •  Large data set possible •  Parallelize the eﬀort •  Each node has index for data stored on it •  Queries combine the results from required nodes ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Query

Slide 22

Slide 22 text

SERVER 3 SERVER 1 SERVER 2 Couchbase Server – San Francisco SERVER 3 SERVER 1 SERVER 2 Couchbase Server – New York Op-mis-c replica-on Per replica-on Tunable Parameters Op-mized protocol based on memcached Reliability and performance at scale Cross Data Center Replica-on (XDCR)

Slide 23

Slide 23 text

www.couchbase.com/develop Clojure Python Ruby libcouchbase Go SDKs

Slide 24

Slide 24 text

Couchbase & PHP

Slide 25

Slide 25 text

Oﬃcial PHP SDK •  Current Release: 1.2.1 (October 2013) •  Based on top of libcouchbase •  Does the heavy liping for you   Knows the cluster topology   „Routes“ operaUons to target servers   Abstracts protocol semanUcs (memcache, h]p,...)

Slide 26

Slide 26 text

Integra-on •  Doctrine Cache Provider   Since 2.4 •  Fully featured ODM in the making •  For now, try Basement!   h]ps://github.com/Basement/Basement