Web5 2013 : Introduction to NoSQL with Couchbase

Generic introduction to NoSQL, Couchbase presentation and Demonstration.

See : http://conf2013.web-5.org/

Transcript

1. Monday, June 17, 13
2. Introducing   Couchbase  Server  2.0 Tugdual  Grall Technical  Evangelist Monday, June 17, 13
3. About  me... • Tugdual  “Tug”  Grall ­ Couchbase • Technical  Evangelist ­ eXo • CTO ­ Oracle • Developer/Product  Manager ­ Mainly  Java/SOA ­ Developer  in  consulDng  firms • Web ­  @tgrall ­      hHp://blog.grallandco.com ­      tgrall ­ NantesJUG  co-­‐founder ­ Pet  Project  : ­ hHp://www.resultri.com Monday, June 17, 13
4. RDBMS  are  not  Enough? Monday, June 17, 13
5. Growth  is  the  New  Reality • Instagram  gained  nearly  1  million  users  overnight  when  then  expanded  to  Android Monday, June 17, 13
6. Draw  Something  Viral  Growth 19 17 15 13 11 9 7 5 3 3/1 28 26 24 22 20 18 16 14 12 10 8 2/6 Draw  Something  by  OMGPOP Daily  Ac)ve  Users  (millions) 21 2 4 6 8 10 12 14 16 Monday, June 17, 13
7. RDBMS  is  good  for  many  thing,  but  hard  to  scale RDBMS  Scales  Up Get  a  bigger,  more  complex  server Users ApplicaNon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost ApplicaDon  Performance   RelaNonal  Database Web/App  Server  Tier System  Cost ApplicaDon  Performance   Won’t  scale   beyond  this   point How  do  you  take  this  growth? Monday, June 17, 13
8. Web/App  Server  Tier Memcached  Tier MySQL  Tier Scaling  out  RDBMS • Run  Many  SQL  Servers • Data  could  be  shared ­ Done  by  the  applicaDon  code • Caching  for  faster  response  Nme Monday, June 17, 13
9. Scaling  out  flaWens  the  cost  and  performance  curves NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  Ner Users NoSQL  Distributed  Data  Store Web/App  Server  Tier ApplicaNon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost ApplicaDon  Performance   ApplicaDon  Performance   System  Cost NoSQL  Technology  Scales  Out Monday, June 17, 13
10. Building  new  database  to  answer  the  following  requirementsNo  schema  required  before   inserDng  dataNo  schema  change  required  to  change  data  formatAuto-­‐sharding  without   applicaDon  parDcipaDonDistributed  queriesIntegrated  main  memory  cachingData   synchronizaDon  (  mulD-­‐datacenter) Dynamo October  2007 Cassandra August  2008 Bigtable November  2006 Voldemort February  2009 Very  few  organizaNons  want  to  (fewer  can)  build  and  maintain  database  so[ware  technology. But  every  organizaNon  building  interacNve  web  applicaNons  needs  this  technology. A  New  Technology? Monday, June 17, 13
11. Lack  of  flexibility/ rigid  schemas Inability  to  scale  out  data Performance  challenges Cost All  of  these Other 49% 35% 29% 16% 12% 11% What  Is  Biggest  Data  Management  Problem  Driving   Use  of  NoSQL  in  Coming  Year? Source:  Couchbase  Survey,  December  2011,  n  =  1351. Monday, June 17, 13
12. NoSQL  Catalog Key-­‐Value Memcached Membase Redis Data  Structure Document Column Graph MongoDB Couchbase Cassandra Cache (memory  only) Database (memory/disk) Neo4j HBase InfiniteGraph Coherence Monday, June 17, 13
13. Couchbase  Server NoSQL  Document  Database Monday, June 17, 13
14. Couchbase  Open  Source  Project • Leading  NoSQL  database  project  focused  on   distributed  database  technology  and   surrounding  ecosystem • Supports  both  key-­‐value  and  document-­‐ oriented  use  cases • All  components  are  available  under  the  Apache   2.0  Public  License • Obtained  as  packaged  soiware  in  both   enterprise  and  community  ediDons. Couchbase Open Source Project Monday, June 17, 13
15. Easy  Scalability Consistent  High   Performance Always  On   24x365 Grow  cluster  without  applicaDon  changes,   without  downDme  with  a  single  click Consistent  sub-­‐millisecond   read  and  write  response  Dmes   with  consistent  high  throughput No  downDme  for  soiware  upgrades,   hardware  maintenance,  etc. Flexible  Data  Model JSON  document  model  with  no  fixed   schema. Couchbase  Server Monday, June 17, 13
16. Flexible  Data  Model • No  need  to  worry  about  the  database  when  changing  your  applicaDon • Records  can  have  different  structures,  there  is  no  fixed  schema • Allows  painless  data  model  changes  for  rapid  applicaDon  development  {        “ID”:  1,        “FIRST”:  “DipN”,        “LAST”:  “Borkar”,        “ZIP”:  “94040”,        “CITY”:  “MV”,        “STATE”:  “CA” } JSON JSON JSON JSON Monday, June 17, 13
17. AddiDonal  Couchbase  Server  Features Built-­‐in  clustering  –  All  nodes  equal Data  replicaDon  with  auto-­‐failover Zero-­‐downDme  maintenance   Built-­‐in  managed  cached Append-­‐only  storage  layer Online  compacDon Monitoring  and  admin  API  &  UI SDK  for  a  variety  of  languages Monday, June 17, 13
18. Market  AdopDon  –  Customers Internet  Companies Enterprises Monday, June 17, 13
19. Heartbeat Process  monitor Global  singleton  supervisor ConfiguraDon  manager on  each  node Rebalance  orchestrator Node  health  monitor one  per  cluster vBucket  state  and  replicaDon  manager hFp REST  management  API/Web  UI HTTP 8091 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199 Erlang/OTP storage  interface Couchbase  EP  Engine 11210 Memcapable    2.0 Moxi 11211 Memcapable    1.0 Memcached New  Persistence  Layer 8092 Query  API Query  Engine Data  Manager Cluster  Manager Couchbase  Server  Architecture Monday, June 17, 13
20. Heartbeat Process  monitor Global  singleton  supervisor ConfiguraDon  manager on  each  node Rebalance  orchestrator Node  health  monitor one  per  cluster vBucket  state  and  replicaDon  manager hFp REST  management  API/Web  UI HTTP 8091 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199 Erlang/OTP storage  interface Couchbase  EP  Engine 11210 Memcapable    2.0 Moxi 11211 Memcapable    1.0 Memcached New  Persistence  Layer 8092 Query  API Query  Engine Couchbase  Server  Architecture Monday, June 17, 13
21. Couchbase  OperaDons Monday, June 17, 13
22. 3 3 2 Single  node  -­‐  Couchbase  Write  OperaDon Managed  Cache Disk  Queue Disk ReplicaDon  Queue App  Server Couchbase  Server  Node Doc  1 Doc  1 Doc  1 To  other  node Monday, June 17, 13
23. COUCHBASE  SERVER    CLUSTER Basic  OperaDon • Docs  distributed  evenly  across  servers   • Each  server  stores  both  acPve  and  replica  docs Only  one  server  acDve  at  a  Dme • Client  library  provides  app  with  simple  interface  to   database • Cluster  map  provides  map   to  which  server  doc  is  on App  never  needs  to  know • App  reads,  writes,  updates  docs • MulPple  app  servers  can  access  same  document  at   same  Pme User  Configured  Replica  Count  =  1 READ/WRITE/UPDATE ACTIVE SERVER  1 ACTIVE Doc  4 Doc  7 Doc Doc Doc SERVER  2 Doc  8 ACTIVE Doc  1 Doc  2 Doc Doc Doc REPLICA Doc  4 Doc  1 Doc  8 Doc Doc Doc REPLICA Doc  6 Doc  3 Doc  2 Doc Doc Doc REPLICA Doc  7 Doc  9 Doc  5 Doc Doc Doc SERVER  3 Doc  6 Doc  5 Doc  2 Doc Doc Doc Doc  9 APP  SERVER  1 COUCHBASE  Client  Library CLUSTER  MAP COUCHBASE  Client  Library CLUSTER  MAP APP  SERVER  2 Monday, June 17, 13
24. Client  SDK www.couchbase.com/develop Clojure Python Ruby libcouchbase Go Monday, June 17, 13
25. NPM { "name": "my-node-application", "version": "1.0.0", "private": true, "dependencies": { "express": "3.x", "couchbase": "0.0.12", "ejs": ">= 0.0.1" } } Monday, June 17, 13
26. Connect  to  the  cluster var driver = require('couchbase'); dbConfiguration = { "hosts": ["localhost:8091"], "bucket": "ideas" }; driver.connect(dbConfiguration, function(err, cb) { if (err) { throw (err) } // your application code here } Monday, June 17, 13
27. Insert  Data var meetup = {"type" : "meetup", "language" : "javascript"}; cb.set("web5",meetup, function(err, meta) {}); var tmp = {"message" : "hello world!"}; cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {}); Monday, June 17, 13
28. Insert  /  Delete  Data var meetup = {"type" : "meetup", "language" : "javascript"}; cb.set("barcelonajs",meetup, function(err, meta) {}); var tmp = {"message" : "hello world!"}; cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {}); cb.set("todelete", tmp, function(err, meta) {}); cb.remove("todelete", function(err, meta) {}); Monday, June 17, 13
29. Retrieve  the  Data cb.get("product:45", function(errs, doc, metas) { console.log("=== get the document ==="); console.log( doc ); }); var keys = new Array(); keys.push("product:1"); keys.push("product:45"); keys.push("product:65"); keys.push("product:80"); cb.get(keys, null, function(errs, docs, metas) { console.log("\n=== get List of documents ==="); console.log( docs ); }); Monday, June 17, 13
30. Retrieve  the  Data What  if  I  want  all  products  or  meetups? key : barcelonajs { "type": "meetup", "language": "javascript" } key : product:10 { "type": "product", "name": "Product with id 10" } Monday, June 17, 13
31. Calling  a  view  from  your  app var queryParams = { stale: false, key : "meetup" }; cb.view("my_views", "by_type", queryParams, function(err, view) { var keys = new Array(); for (var i = 0; i < view.length; i++) { keys.push(view[i].id); } cb.get(keys, null, function(errs, docs, metas) { console.log(docs); }); }); Monday, June 17, 13
32. Demo  Time Monday, June 17, 13
33. www.couchbase.com/download Couchbase  Server   Monday, June 17, 13
34. Q  &  A Monday, June 17, 13
35. Thank  you! tug@couchbase.com @tgrall Get  Couchbase  Server  at   hHp://www.couchbase.com/download Monday, June 17, 13
36. Monday, June 17, 13