CasablancaBigData Meetup

CasablancaBigData Meetup

Couchbase general presentation + Java SDK 2.0 presentation made at the Casablanca Big Data Meetup in January 2015

Fda20bf9d9c85c4390ca7237beba45a2?s=128

Simon Baslé

January 14, 2015
Tweet

Transcript

  1. Casablanca  Big  Data  Meetup   Jan.  2015 Simon  Baslé  -­‐

     SDK  Engineer
  2. Casablanca  Big  Data  Meetup   Jan.  2015 Simon  Baslé  -­‐

     SDK  Engineer Couchbase  Technical  Deep-­‐Dive   Admin  Console  Demo   Overview  of  the  Java  SDK
  3. Couchbase  Technical  Deep-­‐Dive  

  4. ©2014  Couchbase  Inc. Agenda ▪ What  is  Couchbase?   ▪

    Couchbase  Architecture-­‐   ▪ Single  Node  Architecture  and  operations   ▪ Clustered  architecture  and  operations   ▪ Cross  Data  Center  Replication  (XDCR)   ▪ Indexing/Querying/Full-­‐Text   ▪ Couchbase  Mobile  Architecture   ▪ Backup  and  Restore   ▪ Security  with  Couchbase 4
  5. ©2014  Couchbase  Inc. Big  Data  =  Operational  +  Analytic  (NoSQL

     +  Hadoop) 5 ▪ Online   ▪ Web/Mobile/IoT  apps   ▪ Millions  of  customers/ consumers ▪ Offline   ▪ Analytics  apps   ▪ Hundreds  of  business  analysts
  6. Use  Cases 6

  7. ©2014  Couchbase  Inc. Couchbase  provides  a  complete  Data  Management  solution

    7 High  availability   cache Key-­‐value   store Document   database Embedded  database Sync  
 management Multi-­‐purpose  capabilities  support  a  broad  range  of  apps  and  use  cases   Enterprises  often  start  with  cache,  then  broaden  usage  to  other  apps  and  use  cases  
  8. ©2014  Couchbase  Inc. Use  case  Spotlights Profile   Management Store,

     access,  and  update   profile  data  to  support  user   interactions,  such  as   authentication,  customer   lookup,  and  preference   tracking Internet  of   Things Ingest  data  from  endpoint   devices,  such  as  sensors,   smart  meters,  and  other   network  devices,  and  scale   to  support  massive   datasets HA  Caching Consistently  low  latency  and   high  throughput  access  to   any  type  of  data,  alleviating   load  on  backend  systems   and  maintaining  snappy   user  experience
  9. Couchbase  Technical  Overview 9

  10. ©2014  Couchbase  Inc. What  makes  Couchbase  unique? 10 Performance/ Scalability

     leader General  purpose Simplified   administration Cache Key  Value Document Always-­‐on   availability 24  x   365 Enterprises  choose  Couchbase  for  several  key  reasons   Local  /  Mobile
  11. ©2014  Couchbase  Inc. Performance  &  Scalability Auto  Sharding Memory-­‐memory  

    XDCR No  manual  sharding   Database  manages  data   movement  to  scale  out  -­‐   Not  the  user   Market’s  only  memory  to   memory  database  replication   across  clusters    and  geos   Provides  disaster  recover    /  
 data  locality Single  Node  Type Hugely  simplifies   management  of  clusters   Easy  to  scale  clusters  by   adding  any  #  of  nodes
  12. ©2014  Couchbase  Inc. General  purpose  database 12 22 Tunable  built-­‐in

     
 cache Consolidated  cache  and   database   Tune  memory  required  based   on  application  requirements Flexible  schemas   with  JSON Represent  data  with  varying   schemas  using  JSON  on  the   server  or  on  the  device   Index  and  query  data  with   javascript  views     Cache Key  Value Document Local  /  Mobile Couchbase   Lite Light  weight  embedded  DB  for   always  available  apps   Sync  gateway  syncs  data   seamlessly  with  Couchbase   server
  13. ©2014  Couchbase  Inc. Simplified  administration Online  upgrades  and   operations

    Built-­‐in  enterprise   class  admin  console Online  software,  hardware   and  DB  upgrades   Indexing,  Compaction,   Rebalance,  backup  &  Restore Perform  all  administrative  tasks   with  the  click  of  a    button   Monitor  status  of  the  system   visual  at  cluster  level,  database   level,  server  level Restful  APIs All  admin  operations   available  via  UI,  REST  APIs  or   CLI  commands   Integrate  third  party   monitoring  tools  easily  using   REST    
  14. ©2014  Couchbase  Inc. Always-­‐on  Availability 14 High   Availability 24

     x   365 Backup  &  Restore Full  backup  or  Incremental   backup  with  online  restore   Delta  node  catch-­‐ups  for   faster  recovery  after  failures   In-­‐memory  replication  with   manual  or  automatic  fail  over   Rack-­‐zone  awareness  to   minimize  data  unavailability Disaster   Recovery Memory-­‐to-­‐memory  cross   cluster  replication  across  data   centers  or  geos   Active-­‐active  topology  with   bi-­‐directional  setup
  15. Couchbase  Architecture:  Single  Node 15

  16. ©2014  Couchbase  Inc. Couchbase  Server  Architecture Query   Engine Object-­‐

    managed   Cache Storage  Engine DATA  MANAGER 11210  /  11211   Data  access  ports 8092   Query  API HTTP REST  management     API/Web  UI Replication,   Rebalance,    Shard   State  Manager Erlang  / OTP   CLUSTE R     MANAGE R 8091   Admin  Console • Single-­‐node  type  means   easier  administration  and   scaling   • Single  installation   • Two  major  components/processes:   Data  manager  cluster  manager   • Data  manager:   C/C++   Layer  consolidation  of  caching  and  persistence   • Cluster  manager:   Erlang/OTP   Administration  UI’s   Out-­‐of-­‐band  for  data  requests
  17. ©2014  Couchbase  Inc. Write  (‘set’)  Operation 3 3 2 2

    Managed  Cache Disk  Queue Disk Replication   Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 Doc  1 To  other  node • Single-­‐node  type  means  easier   administration  and  scaling   • Writes  are  async  by  default   • Application  gets  acknowledgement  when   successfully  in  RAM  and  can  trade-­‐off   waiting  for  replication  or  persistence  per-­‐ write   • Replication  to  1,  2  or  3  other  nodes   • Replication  is  RAM-­‐based  so  extremely   fast   • Off-­‐node  replication  is  primary  level  of  HA   • Disk  written  to  as  fast  as  possible  –  no   waiting
  18. ©2014  Couchbase  Inc. Read  (‘get’)  Operation GET
 Doc  1 3

    3 2 2 Disk  Queue Replicatio n  Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 Doc  1 Managed  Cache Disk To  other  node • Single-­‐node  type  means  easier   administration  and  scaling   • Reads  coming  out  of  cache  are  extremely   fast   • No  other  process  or  system  to   communicate  with   • Data  connection  is  a  TCP-­‐binary  protocol
  19. ©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Single-­‐node  type:  Simple  administration

     and  scaling.    Other  technologies  have   multiple  types  and  processes  that  need  monitoring  and  analysis  for  scaling   ▪ Built-­‐in,  managed  caching  layer:  Super  high-­‐performance  for  reads  and  writes.     Other  technologies  have  much  more  limited  caching  layer  or  use  filesystem,   block-­‐level  buffering.    Also,  key  separation  of  RAM  from  disk  allows  for  much   more  consistent  performance  as  data  sets  grow  beyond  RAM  available.   ▪ Replication  and  disk  queues:  Everything  done  asynchronously  from  RAM  by   default  for  best  performance,  application  has  the  ability  to  wait  (per-­‐write)  for   replication  and/or  persistence. 19
  20. Couchbase  Architecture:  Cluster-­‐Wide 20

  21. ©2014  Couchbase  Inc. Auto  sharding  –  Bucket  and  vBuckets  

    21 ▪ A  bucket  is  a  logical,  unique  key  space   ▪ Multiple  buckets  can  exist  within  a  single  cluster  of  nodes   ▪ Each  bucket  has  active  and  replica  data  sets     ▪ Each  data  set  has  1024  Virtual  Bucket  (vBuckets)     ▪ Documents  get  logically  mapped  to  vBuckets   ▪ Document  IDs  always  get  hashed  to  the  same  virtual  bucket   ▪ Virtual  buckets  to  do  not  have  a  fixed  physical  server  location   ▪ Mapping  between  the  virtual  buckets  and  physical  server  is  called  the  cluster  map   ▪ Each  virtual  bucket  contains  1/1024th  portion  of  the  data  set vB Data buckets vB 1 ….. 1024 Virtual buckets
  22. ©2014  Couchbase  Inc. Auto  sharding  –  Bucket  and  vBuckets  

    21 ▪ A  bucket  is  a  logical,  unique  key  space   ▪ Multiple  buckets  can  exist  within  a  single  cluster  of  nodes   ▪ Each  bucket  has  active  and  replica  data  sets     ▪ Each  data  set  has  1024  Virtual  Bucket  (vBuckets)     ▪ Documents  get  logically  mapped  to  vBuckets   ▪ Document  IDs  always  get  hashed  to  the  same  virtual  bucket   ▪ Virtual  buckets  to  do  not  have  a  fixed  physical  server  location   ▪ Mapping  between  the  virtual  buckets  and  physical  server  is  called  the  cluster  map   ▪ Each  virtual  bucket  contains  1/1024th  portion  of  the  data  set vB Data buckets vB 1 ….. 1024 Virtual buckets
  23. ©2014  Couchbase  Inc. Cluster  Map App  Server Key    

    NYC  MQ1     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster
  24. ©2014  Couchbase  Inc. Cluster  Map App  Server Key    

    NYC  MQ1     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster
  25. ©2014  Couchbase  Inc. Cluster  Map App  Server Key    

    SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster
  26. ©2014  Couchbase  Inc. Cluster  Map App  Server Key    

    SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster
  27. ©2014  Couchbase  Inc. Cluster  Map  –  2  nodes  added App

     Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster Server  4 Server  5
  28. ©2014  Couchbase  Inc. Cluster  Map  –  2  nodes  added App

     Server Key     SFO  MQ2     Key  vBucket   (hash  function) vBucket  Server   (table  lookup) CRC  32 vBucket1   vBucket2   vBucket3   vBucket4   …   vBucket1024 Server  1 Server  2 Server  3 Couchbase  Cluster Server  4 Server  5
  29. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object)
  30. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster
  31. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication)
  32. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication)
  33. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’
  34. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. Active SERVER 1 Active

    SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’ • Cluster  map  provides  location  of  which  server  a  shard   is  on
  35. ©2014  Couchbase  Inc. ©2014  Couchbase,  Inc. read/write/update Active SERVER 1

    Active SERVER 2 Active SERVER 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Shard 5 Shard 2 Shard 9 Shard Shard Shard Shard 4 Shard 7 Shard 8 Shard Shard Shard Shard 1 Shard 3 Shard 6 Shard Shard Shard Replica Replica Replica Shard 4 Shard 1 Shard 8 Shard Shard Shard Shard 6 Shard 3 Shard 2 Shard Shard Shard Shard 7 Shard 9 Shard 5 Shard Shard Shard Couchbase  Clustering 25 • Application  has  single  logical   connection  to  cluster  (client  object) • Data  is  automatically  sharded  resulting  in  even   document  data  distribution  across  cluster • Each  vbucket  replicated  1,  2  or  3  times  (“peer-­‐to-­‐ peer”  replication) • Docs  are  automatically  hashed  by  the  client  to  a   shard’ • Cluster  map  provides  location  of  which  server  a  shard   is  on • Every  read/write/update/delete  goes  to  same  node   for  a  given  key • Strongly  consistent  data  access  (“read  your  own   writes”) • A  single  Couchbase  node  can  achieve  100k’s  ops/sec   so  no  need  to  scale  reads
  36. ©2014  Couchbase  Inc. Cluster  wide  -­‐  Add  Nodes  to  Cluster

    REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 Doc Doc  8 Doc Doc  9 Doc Doc  2 Doc Doc  8 Doc Doc  5 Doc Doc  6 APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER User  Configured  Replica  Count  =  1 • Multiple  nodes  added  or   removed  at  once   • One-­‐click  operation   • Incremental  movement  of   active  and  replica  vbuckets   and  data   • Client  library  updated  via   cluster  map   • Fully  online  operation,  no   downtime  or  loss  of   performance
  37. ©2014  Couchbase  Inc. Cluster  wide  -­‐  Add  Nodes  to  Cluster

    REPLICA ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc  8 Doc Doc  9 Doc Doc  2 Doc Doc  8 Doc Doc  5 Doc Doc  6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 COUCHBASE  SERVER    CLUSTER User  Configured  Replica  Count  =  1 • Multiple  nodes  added  or   removed  at  once   • One-­‐click  operation   • Incremental  movement  of   active  and  replica  vbuckets   and  data   • Client  library  updated  via   cluster  map   • Fully  online  operation,  no   downtime  or  loss  of   performance
  38. ©2014  Couchbase  Inc. Cluster  wide  -­‐  Fail  Over  Node REPLICA

    ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc  1 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc  3 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc  9 Doc  8 Doc Doc  6 Doc Doc Doc  5 Doc Doc  2 Doc  8 Doc Doc APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 User  Configured  Replica  Count  =  1 COUCHBASE  SERVER    CLUSTER • When  node  goes  down,   some  requests  will  fail   • Failover  is  either  automatic   or  manual   • Client  library  is   automatically  updated  via   cluster  map   • Replicas  not  recreated  to   preserve  stability   • Best  practice  to  replace  node   and  rebalance
  39. ©2014  Couchbase  Inc. Cluster  wide  -­‐  Fail  Over  Node REPLICA

    ACTIVE Doc  5 Doc  2 Doc Doc Doc  4 Doc Doc SERVER  1 REPLICA ACTIVE Doc  4 Doc  7 Doc Doc Doc  6 Doc Doc SERVER  2 REPLICA ACTIVE Doc  1 Doc  2 Doc Doc Doc  7 Doc  9 Doc Doc SERVER  3 SERVER  4 SERVER  5 REPLICA ACTIVE REPLICA ACTIVE Doc  9 Doc  8 Doc Doc  6 Doc Doc Doc  5 Doc  2 Doc  8 Doc Doc Doc Doc  1 Doc  3 APP  SERVER  1 COUCHBASE  Client  Library   CLUSTER  MAP COUCHBASE  Client  Library   CLUSTER  MAP APP  SERVER  2 User  Configured  Replica  Count  =  1 COUCHBASE  SERVER    CLUSTER • When  node  goes  down,   some  requests  will  fail   • Failover  is  either  automatic   or  manual   • Client  library  is   automatically  updated  via   cluster  map   • Replicas  not  recreated  to   preserve  stability   • Best  practice  to  replace  node   and  rebalance
  40. ©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Hash  partitioning/sharding:  No  need

     for  application  to  define  a  shard  key.    No  hot   spots.    Strongly  consistent,  immediately  “read  your  own  writes”   ▪ Clustering  and  topology  awareness:  Abstraction  layer  for  application  means  no   architecture  or  config  changes  when  growing  cluster.       ▪ Truly  shared  nothing:  No  single-­‐point  of  failure  and  linearly  scalable.   ▪ No  load  balancer:  Couchbase  client  library  handles  load  balancing  and  distribution.   ▪ Failover:  Manual  or  automatic,  activates  replica  vbuckets,  prevents  split-­‐brain 28
  41. Couchbase  Architecture:  XDCR 29

  42. ©2014  Couchbase  Inc. XDCR:  Cross  Data  Center  Replication ▪ Application

     can  access  both  clusters  (master  –  master)   ▪ Scales  out  linearly   ▪ Different  from  intra-­‐cluster  replication  (“CP”  versus  “AP”)
  43. ©2014  Couchbase  Inc. XDCR  after  Write 3 3 2 2

    Managed  Cache Disk  Queue Disk Replication   Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 To  other  node XDCR   Queue To  other  cluster Doc  1 Doc  1 • Single-­‐node  type  means   easier  administration  and   scaling   • Writes  are  async  by  default   • XDCR  is  also  RAM-­‐based   • XDCR  happens  between  individual   nodes  –  no  gateway   • All  queues  are  processed  in  parallel
  44. ©2014  Couchbase  Inc. XDCR  in  a  cluster 32 DOC  

    DOC   DOC   DOC   DOC   DOC   DOC   DOC   2 DOC   9 DOC   4 DOC   1 DOC   8 DOC   6 DOC   3 DOC   2 DOC   7 DOC   9 DOC   5 ACTIVE  (RAM) DISK ACTIVE(RAM) DISK ACTIVE(RAM) DISK SERVER  1 SERVER  3 SERVER  2 DOC   DOC   DOC   DOC   DOC   DOC   DOC   DOC   2 DOC   9 DOC   4 DOC   1 DOC   8 DOC   6 DOC   3 DOC   2 DOC   7 DOC   9 DOC   5 ACTIVE  (RAM) DISK ACTIVE(RAM) DISK ACTIVE(RAM) DISK SERVER  1 SERVER  3 SERVER  2 New  York  City  Datacenter   Couchbase  Server  Cluster San  Francisco  Datacenter   Couchbase  Server  Cluster
  45. ©2014  Couchbase  Inc. XDCR:  Flexible  topologies ▪ One-­‐one,  one-­‐many,  many-­‐one

      ▪ Differently  sized  and  resourced  clusters  supported
  46. ©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Simple  setup  and  topology

     aware:  Setup  replication  once,  no  need  to  change   afterwards.    Supports  differently  sized  clusters   ▪ Linearly  Scalable:  Node-­‐node  communication,  no  central  gateway   ▪ High-­‐performance:  RAM-­‐RAM  replication,  designed  for  WAN  connections   ▪ Multi-­‐master:  All  clusters  able  to  service  reads  and/or  writes,  application  controls.   ▪ Automatic  conflict  resolution:  Eventually  consistent  writes 34
  47. Couchbase  Architecture:  Indexing  and  Querying 35

  48. ©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Highest  performance  from  K/V

     operations   ▪ Views:  Incremental  map-­‐reduce,  great  for  complex  computations  and  handling  very  large   datasets.   ▪ Elastic  Search:  Couchbase  as  primary  data-­‐store,  feeds  Elastic  Search  for  full-­‐text  querying   ▪ Hadoop:  Couchbase  as  operational  data-­‐store,  dumping  data  to  Hadoop  for  longer-­‐ running  analytics,  potentially  storing  results  back  in  Couchbase  for  application  access.   ▪ N1QL:  Next-­‐generation  query  language  coming  in  2015.    “SQL-­‐like”,  extended  for  JSON 36
  49. Couchbase  Architecture:  Mobile 37

  50. ©2014  Couchbase  Inc. Couchbase  Mobile  Overview Couchbase  Lite   On-­‐device,

     lightweight,  native   embedded  JSON  database Sync  Gateway   Synchronize  on-­‐device   Couchbase  Lite  with  Couchbase   Server  in  the  cloud Couchbase  Server   High  performance,  scalable,   always-­‐on  JSON  database  in   the  cloud
  51. ©2014  Couchbase  Inc. Key  Take-­‐aways ▪ Couchbase  Mobile  =  Couchbase

     Lite  +  Couchbase  Sync  Gateway  +  Couchbase  Server   ▪ Couchbase  Lite:  Only  NoSQL  database  for  mobile  devices  (phones,  tables,  embedded   systems)   ▪ Couchbase  Sync  Gateway:  Offline/Online  synchronization 39
  52. ©2014  Couchbase  Inc. Security  with  Couchbase ▪ Administrative  Security:  

    ▪ Admin  user  (full  access)  vs.  Read-­‐only  user  (monitoring,  developer  access)   ▪ SSL  encryption  to  REST  API  and  Web  UI   ▪ HTTP  access  log   ▪ Data  Security:   ▪ Applications  connect  via  SASL  with  single  user/pass   ▪ Data-­‐at-­‐Rest  encryption  via  partnership  with  Vormetric   ▪ SSL  encryption  for  over-­‐the-­‐wire   ▪ Coming  soon:   ▪ LDAP/Kerberos  integration   ▪ More  extensive  administrative  action  auditing
  53. ©2014  Couchbase  Inc. Backup  and  Restore  with  Couchbase ▪ Zer0-­‐downtime

     backup  and  restore   ▪ Built-­‐in  utilities:  cbbackup  /  cbrestore   ▪ Full,  differential  and  cumulative  backup  available  per-­‐bucket.       ▪ Restore  from  any  point,  to  any  bucket  or  topology  
  54. ©2014  Couchbase  Inc. Conclusion 42 Performance/ Scalability  leader Sub  millisecond

     latency   with  high  throughput;   memory-­‐centric   architecture General  purpose Simplified   administration Easy  to  depl0y  &  manage;   integrated  Admin  Console,   single-­‐click  cluster   expansion  &  rebalance Cache Key  Value Document Cache,  key  value  store,   document  database,  and   local/mobile  database  in   single  platform   Always-­‐on   availability Data  replication  across   nodes,  clusters,  and  data   centers 24  x   365 Enterprises  choose  Couchbase  for  several  key  reasons   Local  /  Mobile
  55. Demo

  56. Overview  of  the  Java  SDK

  57. ©2014  Couchbase  Inc. The  Couchbase  SDK  Family 45 C  /

     C++ JVM
  58. ©2014  Couchbase  Inc. The  Couchbase  SDK  Family 46 C  /

     C++ JVM ? ?
  59. ©2014  Couchbase  Inc. The  Couchbase  SDK  Family 47 C  /

     C++ JVM ? ? upcoming
  60. ©2014  Couchbase  Inc. Why  A  SDK? ▪ Work  with  the

     Database  using  familiar  patterns  and  language   ▪ The  SDK  does  the  heavy  lifting  for  you   – Cluster  topology   – Routing  operations  to  the  nodes   – Dealing  with  the  protocol(s)   ▪ Mutualize  work  on  performance  and  offer  good  abstractions 48
  61. The  Java  SDK  2.0

  62. ©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Complete  re-­‐write

     (vs  branch  1.x  based  on  spymemcached)   –     ▪ Asynchronous  &  Reactive  at  the  core,  but  with  Synchronous  API 50 ReactiveX
  63. ©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Core  abstracts

     low-­‐level   ▪ Fully  async  &  message-­‐ oriented   ▪ Low  overhead,  high   performance   ▪ Java  6+ 51 Core
  64. ©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ Java  Client

     is  a  higher  level   abstraction   ▪ Exposes  RxJava  Observables  in   the  Asynchronous  API   ▪ Adds  a  Synchronous  API  on  top   of  it   ▪ Java  6+  (especially  great  for  clients  in   Java  8!) 52 Core Java
  65. ©2014  Couchbase  Inc. The  Java  SDK  2.0 ▪ All  that

     could  be  leveraged  to   build  SDKs  for  other  JVM   languages   ▪ Also  adapters  for  popular   frameworks   –  There's  a  Spring  Data   connector  for  the  1.4  branch,  to   be  upgraded  to  2.x   – Hadoop  connector  planned 53 Core Java Scala JRuby,  ... Hadoop Spring Play!2
  66. Working  With  The  Client

  67. ©2014  Couchbase  Inc. Working  With  The  Client //connecting  to  the

     cluster  via  known  node(s)   CouchbaseCluster  cluster  =  CouchbaseCluster.create("192.168.1.101");   //opening  a  bucket,  establishing  resources   Bucket  bucket  =  cluster.openBucket("customBucket",  "password");   //creating  JSON  and  a  Document   JsonObject  json  =  JsonObject.create().put("name",  "John");   JsonDocument  doc  =  JsonDocument.create("key1",  json);   //storing  the  Document   Document  inDb  =  bucket.insert(doc); 55
  68. ©2014  Couchbase  Inc. Working  With  The  Client //connecting  to  the

     cluster  via  known  node(s)   CouchbaseCluster  cluster  =  CouchbaseCluster.create("192.168.1.101");   //opening  a  bucket,  establishing  resources   Bucket  bucket  =  cluster.openBucket("customBucket",  "password");   //creating  JSON  and  a  Document   JsonObject  json  =  JsonObject.create().put("name",  "John");   JsonDocument  doc  =  JsonDocument.create("key1",  json);   //storing  the  Document   Document  inDb  =  bucket.insert(doc); 56 That's  Synchronous
  69. ©2014  Couchbase  Inc. Going  From  Sync  To  Async ▪ The

     Cluster  and  Bucket  both  have  async  versions,  obtained   by  calling  async()  method.   ▪ Asynchronous  API  exposes  RxJava  Observables.   ▪ Very  rich    and  expressive  API  in  terms  of  combinations  and   transformations. 57
  70. ©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a

     document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 58
  71. ©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a

     document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 59 This  is  from  the  async  API,  exposing  an   Observable<JsonDocument>
  72. ©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a

     document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 60 The  Observable  is  a  stream,  can  be  connected  to  an  Observer.   It  is  the  dual  to  Iterable/Iterator,  in  a  push  model  instead  of  a   pull  model.
  73. ©2014  Couchbase  Inc. Going  From  Sync  To  Async //retrieving  a

     document  and  extracting  data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 61 This  is  from  RxJava
  74. ©2014  Couchbase  Inc. RxJava  101 //retrieving  a  document  and  extracting

     data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 62 One  of  the  simplest  transformation  operators,  T  -­‐>  R   Here  gets  a  JsonDocument  and  extract  the  name  String  value   Resulting  in  an  Observable<String>
  75. ©2014  Couchbase  Inc. RxJava  101 //retrieving  a  document  and  extracting

     data  for  output   bucket.async()              .get("key1")              .map(doc  -­‐>  doc.content().getString("name"))              .subscribe(name  -­‐>  System.out.println("Hello  "  +  name)) 63 Then  we  subscribe,  connecting  an  Observer  to  the  stream   "For  each  name  in  the  stream,  print  out  "Hello  name"  "
  76. ©2014  Couchbase  Inc. RxJava  101 Observer  has  3  "hooks":  

    ▪ onNext   – each  time  a  new  item  is  emitted  in  the  stream   ▪ onError   – terminating  event  in  case  of  Exception   ▪ onCompleted   – terminating  event  when  the  stream  is  finished   – an  infinite  stream  could  exist,  not  calling  this 64
  77. ©2014  Couchbase  Inc. RxJava  101 Key  Strength  Is  In  Composition

      RxJava  offers  tons  of  operators  in  Observable  to:   ▪ combine  (merge,  zip,  ...)   ▪ transform  (map,  flatMap,  reduce,  ...)   ▪ filter  (take,  skip,  filter,  ...)   ▪ manage  errors  (retry,  onErrorReturn,  ...) 65
  78. Going  Further

  79. ©2014  Couchbase  Inc. Going  Further ▪ Couchbase  Official  Documentation  

    http://docs.couchbase.com/developer/java-­‐2.0/overview.html   – Section  on  RxJava   ▪ ReactiveX   http://reactivex.io   – RxJava  is  now  part  of  ReactiveX,  which  offers  Rx  bindings  in   other  languages   ▪ RxJava  has  a  great  documentation,  check  it  out!   https://github.com/ReactiveX/RxJava/wiki 67
  80. The  End

  81. Thank  You! Q&A