Slide 1

Slide 1 text

Casablanca  Big  Data  Meetup   Jan.  2015 Simon  Baslé  -­‐  SDK  Engineer

Slide 2

Slide 2 text

Casablanca  Big  Data  Meetup   Jan.  2015 Simon  Baslé  -­‐  SDK  Engineer Couchbase  Technical  Deep-­‐Dive   Admin  Console  Demo   Overview  of  the  Java  SDK

Slide 3

Slide 3 text

Couchbase  Technical  Deep-­‐Dive  

Slide 4

Slide 4 text

©2014  Couchbase  Inc. Agenda ▪ What  is  Couchbase?   ▪ Couchbase  Architecture-­‐   ▪ Single  Node  Architecture  and  operations   ▪ Clustered  architecture  and  operations   ▪ Cross  Data  Center  Replication  (XDCR)   ▪ Indexing/Querying/Full-­‐Text   ▪ Couchbase  Mobile  Architecture   ▪ Backup  and  Restore   ▪ Security  with  Couchbase 4

Slide 5

Slide 5 text

©2014  Couchbase  Inc. Big  Data  =  Operational  +  Analytic  (NoSQL  +  Hadoop) 5 ▪ Online   ▪ Web/Mobile/IoT  apps   ▪ Millions  of  customers/ consumers ▪ Offline   ▪ Analytics  apps   ▪ Hundreds  of  business  analysts

Slide 6

Slide 6 text

Use  Cases 6

Slide 7

Slide 7 text

©2014  Couchbase  Inc. Couchbase  provides  a  complete  Data  Management  solution 7 High  availability   cache Key-­‐value   store Document   database Embedded  database Sync  
 management Multi-­‐purpose  capabilities  support  a  broad  range  of  apps  and  use  cases   Enterprises  often  start  with  cache,  then  broaden  usage  to  other  apps  and  use  cases  

Slide 8

Slide 8 text

©2014  Couchbase  Inc. Use  case  Spotlights Profile   Management Store,  access,  and  update   profile  data  to  support  user   interactions,  such  as   authentication,  customer   lookup,  and  preference   tracking Internet  of   Things Ingest  data  from  endpoint   devices,  such  as  sensors,   smart  meters,  and  other   network  devices,  and  scale   to  support  massive   datasets HA  Caching Consistently  low  latency  and   high  throughput  access  to   any  type  of  data,  alleviating   load  on  backend  systems   and  maintaining  snappy   user  experience

Slide 9

Slide 9 text

Couchbase  Technical  Overview 9

Slide 10

Slide 10 text

©2014  Couchbase  Inc. What  makes  Couchbase  unique? 10 Performance/ Scalability  leader General  purpose Simplified   administration Cache Key  Value Document Always-­‐on   availability 24  x   365 Enterprises  choose  Couchbase  for  several  key  reasons   Local  /  Mobile

Slide 11

Slide 11 text

©2014  Couchbase  Inc. Performance  &  Scalability Auto  Sharding Memory-­‐memory   XDCR No  manual  sharding   Database  manages  data   movement  to  scale  out  -­‐   Not  the  user   Market’s  only  memory  to   memory  database  replication   across  clusters    and  geos   Provides  disaster  recover    /  
 data  locality Single  Node  Type Hugely  simplifies   management  of  clusters   Easy  to  scale  clusters  by   adding  any  #  of  nodes

Slide 12

Slide 12 text

©2014  Couchbase  Inc. General  purpose  database 12 22 Tunable  built-­‐in  
 cache Consolidated  cache  and   database   Tune  memory  required  based   on  application  requirements Flexible  schemas   with  JSON Represent  data  with  varying   schemas  using  JSON  on  the   server  or  on  the  device   Index  and  query  data  with   javascript  views     Cache Key  Value Document Local  /  Mobile Couchbase   Lite Light  weight  embedded  DB  for   always  available  apps   Sync  gateway  syncs  data   seamlessly  with  Couchbase   server

Slide 13

Slide 13 text

©2014  Couchbase  Inc. Simplified  administration Online  upgrades  and   operations Built-­‐in  enterprise   class  admin  console Online  software,  hardware   and  DB  upgrades   Indexing,  Compaction,   Rebalance,  backup  &  Restore Perform  all  administrative  tasks   with  the  click  of  a    button   Monitor  status  of  the  system   visual  at  cluster  level,  database   level,  server  level Restful  APIs All  admin  operations   available  via  UI,  REST  APIs  or   CLI  commands   Integrate  third  party   monitoring  tools  easily  using   REST    

Slide 14

Slide 14 text

©2014  Couchbase  Inc. Always-­‐on  Availability 14 High   Availability 24  x   365 Backup  &  Restore Full  backup  or  Incremental   backup  with  online  restore   Delta  node  catch-­‐ups  for   faster  recovery  after  failures   In-­‐memory  replication  with   manual  or  automatic  fail  over   Rack-­‐zone  awareness  to   minimize  data  unavailability Disaster   Recovery Memory-­‐to-­‐memory  cross   cluster  replication  across  data   centers  or  geos   Active-­‐active  topology  with   bi-­‐directional  setup

Slide 15

Slide 15 text

Couchbase  Architecture:  Single  Node 15

Slide 16

Slide 16 text

©2014  Couchbase  Inc. Couchbase  Server  Architecture Query   Engine Object-­‐ managed   Cache Storage  Engine DATA  MANAGER 11210  /  11211   Data  access  ports 8092   Query  API HTTP REST  management     API/Web  UI Replication,   Rebalance,    Shard   State  Manager Erlang  / OTP   CLUSTE R     MANAGE R 8091   Admin  Console • Single-­‐node  type  means   easier  administration  and   scaling   • Single  installation   • Two  major  components/processes:   Data  manager  cluster  manager   • Data  manager:   C/C++   Layer  consolidation  of  caching  and  persistence   • Cluster  manager:   Erlang/OTP   Administration  UI’s   Out-­‐of-­‐band  for  data  requests

Slide 17

Slide 17 text

©2014  Couchbase  Inc. Write  (‘set’)  Operation 3 3 2 2 Managed  Cache Disk  Queue Disk Replication   Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 Doc  1 To  other  node • Single-­‐node  type  means  easier   administration  and  scaling   • Writes  are  async  by  default   • Application  gets  acknowledgement  when   successfully  in  RAM  and  can  trade-­‐off   waiting  for  replication  or  persistence  per-­‐ write   • Replication  to  1,  2  or  3  other  nodes   • Replication  is  RAM-­‐based  so  extremely   fast   • Off-­‐node  replication  is  primary  level  of  HA   • Disk  written  to  as  fast  as  possible  –  no   waiting

Slide 18

Slide 18 text

©2014  Couchbase  Inc. Read  (‘get’)  Operation GET
 Doc  1 3 3 2 2 Disk  Queue Replicatio n  Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 Doc  1 Managed  Cache Disk To  other  node • Single-­‐node  type  means  easier   administration  and  scaling   • Reads  coming  out  of  cache  are  extremely   fast   • No  other  process  or  system  to   communicate  with   • Data  connection  is  a  TCP-­‐binary  protocol

Slide 19

Slide 19 text

©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Single-­‐node  type:  Simple  administration  and  scaling.    Other  technologies  have   multiple  types  and  processes  that  need  monitoring  and  analysis  for  scaling   ▪ Built-­‐in,  managed  caching  layer:  Super  high-­‐performance  for  reads  and  writes.     Other  technologies  have  much  more  limited  caching  layer  or  use  filesystem,   block-­‐level  buffering.    Also,  key  separation  of  RAM  from  disk  allows  for  much   more  consistent  performance  as  data  sets  grow  beyond  RAM  available.   ▪ Replication  and  disk  queues:  Everything  done  asynchronously  from  RAM  by   default  for  best  performance,  application  has  the  ability  to  wait  (per-­‐write)  for   replication  and/or  persistence. 19

Slide 20

Slide 20 text

Couchbase  Architecture:  Cluster-­‐Wide 20

Slide 21

Slide 21 text

©2014  Couchbase  Inc. Auto  sharding  –  Bucket  and  vBuckets   21 ▪ A  bucket  is  a  logical,  unique  key  space   ▪ Multiple  buckets  can  exist  within  a  single  cluster  of  nodes   ▪ Each  bucket  has  active  and  replica  data  sets     ▪ Each  data  set  has  1024  Virtual  Bucket  (vBuckets)     ▪ Documents  get  logically  mapped  to  vBuckets   ▪ Document  IDs  always  get  hashed  to  the  same  virtual  bucket   ▪ Virtual  buckets  to  do  not  have  a  fixed  physical  server  location   ▪ Mapping  between  the  virtual  buckets  and  physical  server  is  called  the  cluster  map   ▪ Each  virtual  bucket  contains  1/1024th  portion  of  the  data  set vB Data buckets vB 1 ….. 1024 Virtual buckets

Slide 22

Slide 22 text

©2014  Couchbase  Inc. Auto  sharding  –  Bucket  and  vBuckets   21 ▪ A  bucket  is  a  logical,  unique  key  space   ▪ Multiple  buckets  can  exist  within  a  single  cluster  of  nodes   ▪ Each  bucket  has  active  and  replica  data  sets     ▪ Each  data  set  has  1024  Virtual  Bucket  (vBuckets)     ▪ Documents  get  logically  mapped  to  vBuckets   ▪ Document  IDs  always  get  hashed  to  the  same  virtual  bucket   ▪ Virtual  buckets  to  do  not  have  a  fixed  physical  server  location   ▪ Mapping  between  the  virtual  buckets  and  physical  server  is  called  the  cluster  map   ▪ Each  virtual  bucket  contains  1/1024th  portion  of  the  data  set vB Data buckets vB 1 ….. 1024 Virtual buckets

Slide 23

Slide 23 text

©2014  Couchbase  Inc. Cluster  Map App  Server Key     NYC  MQ1     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster

Slide 24

Slide 24 text

©2014  Couchbase  Inc. Cluster  Map App  Server Key     NYC  MQ1     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster

Slide 25

Slide 25 text

©2014  Couchbase  Inc. Cluster  Map App  Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster

Slide 26

Slide 26 text

©2014  Couchbase  Inc. Cluster  Map App  Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster

Slide 27

Slide 27 text

©2014  Couchbase  Inc. Cluster  Map  –  2  nodes  added App  Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster Server  4 Server  5

Slide 28

Slide 28 text

©2014  Couchbase  Inc. Cluster  Map  –  2  nodes  added App  Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster Server  4 Server  5

Slide 29

Slide 29 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object)

Slide 30

Slide 30 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster

Slide 31

Slide 31 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication)

Slide 32

Slide 32 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication)

Slide 33

Slide 33 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’

Slide 34

Slide 34 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’ • Cluster  map  provides  location  of  which  server  a  shard   is  on

Slide 35

Slide 35 text

©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. read/write/update Active SERVER 1 Active SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’ • Cluster  map  provides  location  of  which  server  a  shard   is  on • Every  read/write/update/delete  goes  to  same  node   for  a  given  key • Strongly  consistent  data  access  (“read  your  own   writes”) • A  single  Couchbase  node  can  achieve  100k’s  ops/sec   so  no  need  to  scale  reads

Slide 36

Slide 36 text

©2014  Couchbase  Inc. Cluster  wide  -­‐  Add  Nodes  to  Cluster REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 Doc Doc  8 Doc Doc  9 Doc Doc  2 Doc Doc  8 Doc Doc  5 Doc Doc  6 APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER User  Configured  Replica  Count  =  1 • Multiple  nodes  added  or   removed  at  once   • One-­‐click  operation   • Incremental  movement  of   active  and  replica  vbuckets   and  data   • Client  library  updated  via   cluster  map   • Fully  online  operation,  no   downtime  or  loss  of   performance

Slide 37

Slide 37 text

©2014  Couchbase  Inc. Cluster  wide  -­‐  Add  Nodes  to  Cluster REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc  8 Doc Doc  9 Doc Doc  2 Doc Doc  8 Doc Doc  5 Doc Doc  6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER User  Configured  Replica  Count  =  1 • Multiple  nodes  added  or   removed  at  once   • One-­‐click  operation   • Incremental  movement  of   active  and  replica  vbuckets   and  data   • Client  library  updated  via   cluster  map   • Fully  online  operation,  no   downtime  or  loss  of   performance

Slide 38

Slide 38 text

©2014  Couchbase  Inc. Cluster  wide  -­‐  Fail  Over  Node REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc  9 Doc  8 Doc Doc  6 Doc Doc Doc  5 Doc Doc  2 Doc  8 Doc Doc APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 User  Configured  Replica  Count  =  1 COUCHBASE  SERVER    CLUSTER • When  node  goes  down,   some  requests  will  fail   • Failover  is  either  automatic   or  manual   • Client  library  is   automatically  updated  via   cluster  map   • Replicas  not  recreated  to   preserve  stability   • Best  practice  to  replace  node   and  rebalance

Slide 39

Slide 39 text

©2014  Couchbase  Inc. Cluster  wide  -­‐  Fail  Over  Node REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc  9 Doc  8 Doc Doc  6 Doc Doc Doc  5 Doc  2 Doc  8 Doc Doc Doc Doc  1 Doc  3 APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 User  Configured  Replica  Count  =  1 COUCHBASE  SERVER    CLUSTER • When  node  goes  down,   some  requests  will  fail   • Failover  is  either  automatic   or  manual   • Client  library  is   automatically  updated  via   cluster  map   • Replicas  not  recreated  to   preserve  stability   • Best  practice  to  replace  node   and  rebalance

Slide 40

Slide 40 text

©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Hash  partitioning/sharding:  No  need  for  application  to  define  a  shard  key.    No  hot   spots.    Strongly  consistent,  immediately  “read  your  own  writes”   ▪ Clustering  and  topology  awareness:  Abstraction  layer  for  application  means  no   architecture  or  config  changes  when  growing  cluster.       ▪ Truly  shared  nothing:  No  single-­‐point  of  failure  and  linearly  scalable.   ▪ No  load  balancer:  Couchbase  client  library  handles  load  balancing  and  distribution.   ▪ Failover:  Manual  or  automatic,  activates  replica  vbuckets,  prevents  split-­‐brain 28

Slide 41

Slide 41 text

Couchbase  Architecture:  XDCR 29

Slide 42

Slide 42 text

©2014  Couchbase  Inc. XDCR:  Cross  Data  Center  Replication ▪ Application  can  access  both  clusters  (master  –  master)   ▪ Scales  out  linearly   ▪ Different  from  intra-­‐cluster  replication  (“CP”  versus  “AP”)

Slide 43

Slide 43 text

©2014  Couchbase  Inc. XDCR  after  Write 3 3 2 2 Managed  Cache Disk  Queue Disk Replication   Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 To  other  node XDCR   Queue To  other  cluster Doc  1 Doc  1 • Single-­‐node  type  means   easier  administration  and   scaling   • Writes  are  async  by  default   • XDCR  is  also  RAM-­‐based   • XDCR  happens  between  individual   nodes  –  no  gateway   • All  queues  are  processed  in  parallel

Slide 44

Slide 44 text

©2014  Couchbase  Inc. XDCR  in  a  cluster 32 DOC   DOC   DOC   DOC   DOC   DOC   DOC   DOC   2 DOC   9 DOC   4 DOC   1 DOC   8 DOC   6 DOC   3 DOC   2 DOC   7 DOC   9 DOC   5 ACTIVE  (RAM) DISK ACTIVE(RAM) DISK ACTIVE(RAM) DISK SERVER  1 SERVER  3 SERVER  2 DOC   DOC   DOC   DOC   DOC   DOC   DOC   DOC   2 DOC   9 DOC   4 DOC   1 DOC   8 DOC   6 DOC   3 DOC   2 DOC   7 DOC   9 DOC   5 ACTIVE  (RAM) DISK ACTIVE(RAM) DISK ACTIVE(RAM) DISK SERVER  1 SERVER  3 SERVER  2 New  York  City  Datacenter   Couchbase  Server  Cluster San  Francisco  Datacenter   Couchbase  Server  Cluster

Slide 45

Slide 45 text

©2014  Couchbase  Inc. XDCR:  Flexible  topologies ▪ One-­‐one,  one-­‐many,  many-­‐one   ▪ Differently  sized  and  resourced  clusters  supported

Slide 46

Slide 46 text

©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Simple  setup  and  topology  aware:  Setup  replication  once,  no  need  to  change   afterwards.    Supports  differently  sized  clusters   ▪ Linearly  Scalable:  Node-­‐node  communication,  no  central  gateway   ▪ High-­‐performance:  RAM-­‐RAM  replication,  designed  for  WAN  connections   ▪ Multi-­‐master:  All  clusters  able  to  service  reads  and/or  writes,  application  controls.   ▪ Automatic  conflict  resolution:  Eventually  consistent  writes 34

Slide 47

Slide 47 text

Couchbase  Architecture:  Indexing  and  Querying 35

Slide 48

Slide 48 text

©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Highest  performance  from  K/V  operations   ▪ Views:  Incremental  map-­‐reduce,  great  for  complex  computations  and  handling  very  large   datasets.   ▪ Elastic  Search:  Couchbase  as  primary  data-­‐store,  feeds  Elastic  Search  for  full-­‐text  querying   ▪ Hadoop:  Couchbase  as  operational  data-­‐store,  dumping  data  to  Hadoop  for  longer-­‐ running  analytics,  potentially  storing  results  back  in  Couchbase  for  application  access.   ▪ N1QL:  Next-­‐generation  query  language  coming  in  2015.    “SQL-­‐like”,  extended  for  JSON 36

Slide 49

Slide 49 text

Couchbase  Architecture:  Mobile 37

Slide 50

Slide 50 text

©2014  Couchbase  Inc. Couchbase  Mobile  Overview Couchbase  Lite   On-­‐device,  lightweight,  native   embedded  JSON  database Sync  Gateway   Synchronize  on-­‐device   Couchbase  Lite  with  Couchbase   Server  in  the  cloud Couchbase  Server   High  performance,  scalable,   always-­‐on  JSON  database  in   the  cloud

Slide 51

Slide 51 text

©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Couchbase  Mobile  =  Couchbase  Lite  +  Couchbase  Sync  Gateway  +  Couchbase  Server   ▪ Couchbase  Lite:  Only  NoSQL  database  for  mobile  devices  (phones,  tables,  embedded   systems)   ▪ Couchbase  Sync  Gateway:  Offline/Online  synchronization 39

Slide 52

Slide 52 text

©2014  Couchbase  Inc. Security  with  Couchbase ▪ Administrative  Security:   ▪ Admin  user  (full  access)  vs.  Read-­‐only  user  (monitoring,  developer  access)   ▪ SSL  encryption  to  REST  API  and  Web  UI   ▪ HTTP  access  log   ▪ Data  Security:   ▪ Applications  connect  via  SASL  with  single  user/pass   ▪ Data-­‐at-­‐Rest  encryption  via  partnership  with  Vormetric   ▪ SSL  encryption  for  over-­‐the-­‐wire   ▪ Coming  soon:   ▪ LDAP/Kerberos  integration   ▪ More  extensive  administrative  action  auditing

Slide 53

Slide 53 text

©2014  Couchbase  Inc. Backup  and  Restore  with  Couchbase ▪ Zer0-­‐downtime  backup  and  restore   ▪ Built-­‐in  utilities:  cbbackup  /  cbrestore   ▪ Full,  differential  and  cumulative  backup  available  per-­‐bucket.       ▪ Restore  from  any  point,  to  any  bucket  or  topology  

Slide 54

Slide 54 text

©2014  Couchbase  Inc. Conclusion 42 Performance/ Scalability  leader Sub  millisecond  latency   with  high  throughput;   memory-­‐centric   architecture General  purpose Simplified   administration Easy  to  depl0y  &  manage;   integrated  Admin  Console,   single-­‐click  cluster   expansion  &  rebalance Cache Key  Value Document Cache,  key  value  store,   document  database,  and   local/mobile  database  in   single  platform   Always-­‐on   availability Data  replication  across   nodes,  clusters,  and  data   centers 24  x   365 Enterprises  choose  Couchbase  for  several  key  reasons   Local  /  Mobile

Slide 55

Slide 55 text

Demo

Slide 56

Slide 56 text

Overview  of  the  Java  SDK

Slide 57

Slide 57 text

©2014  Couchbase  Inc. The  Couchbase  SDK  Family 45 C  /  C++ JVM

Slide 58

Slide 58 text

©2014  Couchbase  Inc. The  Couchbase  SDK  Family 46 C  /  C++ JVM ? ?

Slide 59

Slide 59 text

©2014  Couchbase  Inc. The  Couchbase  SDK  Family 47 C  /  C++ JVM ? ? upcoming

Slide 60

Slide 60 text

©2014  Couchbase  Inc. Why  A  SDK? ▪ Work  with  the  Database  using  familiar  patterns  and  language   ▪ The  SDK  does  the  heavy  lifting  for  you   – Cluster  topology   – Routing  operations  to  the  nodes   – Dealing  with  the  protocol(s)   ▪ Mutualize  work  on  performance  and  offer  good  abstractions 48

Slide 61

Slide 61 text

The  Java  SDK  2.0

Slide 62

Slide 62 text

©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Complete  re-­‐write  (vs  branch  1.x  based  on  spymemcached)   –     ▪ Asynchronous  &  Reactive  at  the  core,  but  with  Synchronous  API 50 ReactiveX

Slide 63

Slide 63 text

©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Core  abstracts  low-­‐level   ▪ Fully  async  &  message-­‐ oriented   ▪ Low  overhead,  high   performance   ▪ Java  6+ 51 Core

Slide 64

Slide 64 text

©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Java  Client  is  a  higher  level   abstraction   ▪ Exposes  RxJava  Observables  in   the  Asynchronous  API   ▪ Adds  a  Synchronous  API  on  top   of  it   ▪ Java  6+  (especially  great  for  clients  in   Java  8!) 52 Core Java

Slide 65

Slide 65 text

©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ All  that  could  be  leveraged  to   build  SDKs  for  other  JVM   languages   ▪ Also  adapters  for  popular   frameworks   –  There's  a  Spring  Data   connector  for  the  1.4  branch,  to   be  upgraded  to  2.x   – Hadoop  connector  planned 53 Core Java Scala JRuby,  ... Hadoop Spring Play!2

Slide 66

Slide 66 text

Working  With  The  Client

Slide 67

Slide 67 text

©2014  Couchbase  Inc. Working  With  The  Client //connecting  to  the  cluster  via  known  node(s)   CouchbaseCluster  cluster  =  CouchbaseCluster.create("192.168.1.101");   //opening  a  bucket,  establishing  resources   Bucket  bucket  =  cluster.openBucket("customBucket",  "password");   //creating  JSON  and  a  Document   JsonObject  json  =  JsonObject.create().put("name",  "John");   JsonDocument  doc  =  JsonDocument.create("key1",  json);   //storing  the  Document   Document  inDb  =  bucket.insert(doc); 55

Slide 68

Slide 68 text

©2014  Couchbase  Inc. Working  With  The  Client //connecting  to  the  cluster  via  known  node(s)   CouchbaseCluster  cluster  =  CouchbaseCluster.create("192.168.1.101");   //opening  a  bucket,  establishing  resources   Bucket  bucket  =  cluster.openBucket("customBucket",  "password");   //creating  JSON  and  a  Document   JsonObject  json  =  JsonObject.create().put("name",  "John");   JsonDocument  doc  =  JsonDocument.create("key1",  json);   //storing  the  Document   Document  inDb  =  bucket.insert(doc); 56 That's  Synchronous

Slide 69

Slide 69 text

©2014  Couchbase  Inc. Going  From  Sync  To  Async ▪ The  Cluster  and  Bucket  both  have  async  versions,  obtained   by  calling  async()  method.   ▪ Asynchronous  API  exposes  RxJava  Observables.   ▪ Very  rich    and  expressive  API  in  terms  of  combinations  and   transformations. 57

Slide 70

Slide 70 text

©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 58

Slide 71

Slide 71 text

©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 59 This  is  from  the  async  API,  exposing  an   Observable

Slide 72

Slide 72 text

©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 60 The  Observable  is  a  stream,  can  be  connected  to  an  Observer.   It  is  the  dual  to  Iterable/Iterator,  in  a  push  model  instead  of  a   pull  model.

Slide 73

Slide 73 text

©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 61 This  is  from  RxJava

Slide 74

Slide 74 text

©2014  Couchbase  Inc. RxJava  101 //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 62 One  of  the  simplest  transformation  operators,  T  -­‐>  R   Here  gets  a  JsonDocument  and  extract  the  name  String  value   Resulting  in  an  Observable

Slide 75

Slide 75 text

©2014  Couchbase  Inc. RxJava  101 //retrieving  a  document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 63 Then  we  subscribe,  connecting  an  Observer  to  the  stream   "For  each  name  in  the  stream,  print  out  "Hello  name"  "

Slide 76

Slide 76 text

©2014  Couchbase  Inc. RxJava  101 Observer  has  3  "hooks":   ▪ onNext   – each  time  a  new  item  is  emitted  in  the  stream   ▪ onError   – terminating  event  in  case  of  Exception   ▪ onCompleted   – terminating  event  when  the  stream  is  finished   – an  infinite  stream  could  exist,  not  calling  this 64

Slide 77

Slide 77 text

©2014  Couchbase  Inc. RxJava  101 Key  Strength  Is  In  Composition   RxJava  offers  tons  of  operators  in  Observable  to:   ▪ combine  (merge,  zip,  ...)   ▪ transform  (map,  flatMap,  reduce,  ...)   ▪ filter  (take,  skip,  filter,  ...)   ▪ manage  errors  (retry,  onErrorReturn,  ...) 65

Slide 78

Slide 78 text

Going  Further

Slide 79

Slide 79 text

©2014  Couchbase  Inc. Going  Further ▪ Couchbase  Official  Documentation   http://docs.couchbase.com/developer/java-­‐2.0/overview.html   – Section  on  RxJava   ▪ ReactiveX   http://reactivex.io   – RxJava  is  now  part  of  ReactiveX,  which  offers  Rx  bindings  in   other  languages   ▪ RxJava  has  a  great  documentation,  check  it  out!   https://github.com/ReactiveX/RxJava/wiki 67

Slide 80

Slide 80 text

The  End

Slide 81

Slide 81 text

Thank  You! Q&A