Finding the right NoSQL DB for the job - The path to a non-RDBMS solution at Traackr

Finding the right NoSQL DB for the job - The path to a non-RDBMS solution at Traackr

A walkthrough of Traackr's experience in choosing a NoSQL solution and how we ended up up switching from HBase to MongoDB. This deck goes through some in depth technical aspects, like schema design and our use of secondary indexes.

Ba71ad856b0e8d5f224b8c927c85a7c4?s=128

George P. Stathis

April 23, 2012
Tweet

Transcript

  1. Finding  the  right  NoSQL  DB  for   the  job  

    The  path  to  a  non-­‐RDBMS  solu>on  at  
  2. Who  we  are   •  A  search   engine  

    •  A  people   search  engine   •  An  influencer   search  engine   •  Subscrip>on-­‐ based  
  3. George  Stathis   VP  Engineering   14+  years  of  experience

      building  full-­‐stack  web   soKware  systems  with  a  past   focus  on  e-­‐commerce  and   publishing.  Currently   responsible  for  building   engineering  capability  to   enable  Traackr's  growth  goals.  
  4. What’s  this  talk  about?   •  Why  we  picked  a

     NoSQL  database   •  How  we  picked  a  NoSQL  database   •  My  NoSQL  does  not  do  the  job!  What  now?!   •  Nirvana  =  the  right  tool  for  the  job  
  5. Why  did  we  pick  a  NoSQL  DB?  

  6. There  are  some  misconcep>ons   around  NoSQL  only  being  appropriate

      when  one  needs  to  achieve   “Web  Scale”    
  7. I  need  web  scale!   hXp://www.youtube.com/watch?v=b2F-­‐DItXtZs  

  8. Traackr  picked  NoSQL;  are  we     “Web  Scale”?  

  9. •  In  terms  of  users/traffic?   Do  we  fit  the

     “Web  scale”  profile?  
  10. Source:  compete.com  

  11. Source:  compete.com  

  12. Source:  compete.com  

  13. Source:  compete.com  

  14. Source:  highscalability.com  

  15. None
  16. •  In  terms  of  users/traffic?   •  In  terms  of

     the  amount  of  data?   Do  we  fit  the  “Web  scale”  profile?  
  17. PRIMARY>  use  traackr   switched  to  db  traackr   PRIMARY>

     db.stats()   {    "db"  :  "traackr",    "collec>ons"  :  12,    "objects"  :  68226121,    "avgObjSize"  :  2972.0800625760330,    "dataSize"  :  202773493971,    "storageSize"  :  221491429671,    "numExtents"  :  199,    "indexes"  :  33,    "indexSize"  :  27472394891,    "fileSize"  :  266623699968,    "nsSizeMB"  :  16,    "ok"  :  1   }   That’s  a  quarter  of  a   terabyte  …  
  18. Wait!  What?  My   Synology  NAS  at  home   can

     hold  2TB!  
  19. No  need  for  us  to  track  the  en>re  web  

    Web  Content   Influencer   Content   Not  at  scale  :-­‐)  
  20. •  In  terms  of  users/traffic?   •  In  terms  of

     the  amount  of  data?   Do  we  fit  the  “Web  scale”  profile?  
  21. Alternate  view  of  “Web  Scale”   Web  data  is:  

    Heterogeneous   Unstructured  (text)  
  22. Source:  hXp://www.opte.org/   Visualiza>on  of  the  Internet,  Nov.  23rd  2003

     
  23. Data  sources  are   isolated  islands  of  rich   data

     with  lose  links  to   one  another    
  24. How  do  we  build  a  database  that   models  all

     possible  en>>es  found  on   the  web?  
  25. Modeling  the  web:  the  RDBMS  way  

  26. Source:  socialbuXerflyclt.com  

  27. or  

  28. None
  29. {        "realName":  "David  Chancogne",      

     ">tle":  "CTO",        "descrip>on":  "Web.  Geek.\r\nTraackr:  hXp://traackr.com\r\nPropz:  hXp://propz.me",        "primaryAffilia>on":  "Traackr",        "email":  "dchancogne@traackr.com",        "loca>on":  "Cambridge,  MA,  United  States",        "siteReferences":  [              {                    "siteUrl":  "hXp://twiXer.com/dchancogne",                    "metrics":  [                          {                                "value":  216,                                "name":  "twiXer_followers_count"                          },                          {                                "value":  2107,                                "name":  "twiXer_statuses_count"                          }                    ]              },              {                    "siteUrl":  "hXp://traackr.com/blog/author/david",                    "metrics":  [                          {                                "value":  21,                                "name":  "google_inbound_links"                          }                    ]              }        ]   }   Influencer  data  as  JSON  
  30. “In  the  old  world  of  data  analysis  you  knew  

    exactly  which  ques>ons  you  wanted  to  ask,   which  drove  a  very  predictable  collec>on  and   storage  model.  In  the  new  world  of  data  analysis   your  ques>ons  are  going  to  evolve  and  change   over  >me  and  as  such  you  need  to  be  able  to   collect,  store  and  analyze  data  without  being   constrained  by  resources.”   —  Werner  Vogels,  CTO/VP  Amazon.com  
  31. NoSQL  =  schema  flexibility  

  32. •  In  terms  of  users/traffic?   •  In  terms  of

     the  amount  of  data?   Do  we  fit  the  “Web  scale”  profile?  
  33. •  In  terms  of  users/traffic?   •  In  terms  of

     the  amount  of  data?   •  In  terms  of  the  variety  of  the  data   Do  we  fit  the  “Web  scale”  profile?   ✓  
  34. Traackr’s  Datastore  Requirements   •  Schema  flexibility   •  Good

     at  storing  lots  of  variable  length  text   •  Batch  processing  op>ons   ✓  
  35. Requirement:  text  storage   Variable  text  length:   <  big

     variance  <   140   character   tweets   mul>-­‐page   blog  posts  
  36. Requirement:  text  storage   RDBMS’  answer  to  variable  text  length:

      Plan  ahead  for  largest  value   CLOB/BLOB  
  37. Requirement:  text  storage   Issues  with  CLOB/BLOG  for  us:  

    No  clue  what  largest  value  is   CLOB/BLOB  for  tweets  =  wasted  space  
  38. Requirement:  text  storage   NoSQL  solu>ons  are  great  for  text:

      No  length  requirements  (automated   chunking)   Limited  space  overhead  
  39. Traackr’s  Datastore  Requirements   •  Schema  flexibility   •  Good

     at  storing  lots  of  variable  length  text   •  Batch  processing  op>ons   ✓   ✓  
  40. Requirement:  batch  processing   Some  NoSQL   solu>ons  come  

    with  MapReduce   Source:  hXp://code.google.com/  
  41. Requirement:  batch  processing   MapReduce  +  RDBMS:   Possible  but

     proprietary  solu>ons   Usually  involves  expor>ng  data  from   RDBMS  into  a  NoSQL  system  anyway.   Defeats  data  locality  benefit  of  MR  
  42. Traackr’s  Datastore  Requirements   •  Schema  flexibility   •  Good

     at  storing  lots  of  variable  length  text   •  Batch  processing  op>ons   ✓   ✓   A  NoSQL  op>on  is  the  right  fit   ✓  
  43. How  did  we  pick  a  NoSQL  DB?  

  44. Bewildering  number  of  op>ons   Key/Value  Databases   •  Distributed

     hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms    
  45. Bewildering  number  of  op>ons   Key/Value  Databases   •  Distributed

     hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms    
  46. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Graph  Databases:  while  we  can  model   our  domain  as  a  graph  we  don’t  want  to   pigeonhole  ourselves  into  this  structure.   We’d  rather  use  these  tools  for   specialized  data  analysis  but  not  as  the   main  data  store.  
  47. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Memcache:  memory-­‐based,   we  need  true  persistence  
  48. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Amazon  SimpleDB:  not  willing  to   store  our  data  in  a  proprietary   datastore.  
  49. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Not  willing  to  store  our  data  in  a   proprietary  datastore.   Redis  and  LinkedIn’s  Project   Voldermort:  no  query  filters,   beXer  used  as  queues  or   distributed  caches  
  50. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     CouchDB:  no  ad-­‐hoc  queries;   maturity  in  early  2010  made  us   shy  away  although  we  did  try   early  prototypes.  
  51. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Cassandra:  in  early  2010,   maturity  ques>ons,  no  secondary   indexes  and  no  batch  processing   op>ons  (came  later  on).  
  52. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     MongoDB:  in  early  2010,  maturity   ques>ons,  adop>on  ques>ons   and  no  batch  processing  op>ons.  
  53. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Riak:  very  close  but  in  early  2010,   we  had  adop>on  ques>ons.  
  54. Trimming  op>ons   Key/Value  Databases   •  Distributed  hashtables  

    •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     HBase:  came  across  as  the  most  mature   at  the  >me,  with  several  deployments,  a   healthy  community,  "out-­‐of-­‐the  box"   secondary  indexes  through  a  contrib  and   support  for  batch  processing  using   Hadoop/MR  .  
  55. Climbing  the  learning  curve  

  56. When  Big-­‐Data  =  Big  Architectures   Source:  hXp://www.larsgeorge.com/2009/10/hbase-­‐architecture-­‐101-­‐storage.html   Must

     have  a  Hadoop  HDFS   cluster  of  at  least  2x  replica>on   factor  nodes   Must  have  an  odd   number  of     Zookeeper  quorum   nodes   Then  you  can  run  your  Hbase   nodes  but  it’s  recommended  to   co-­‐locate  regionservers  with   hadoop  datanodes  so  you  have   to  manage  resources.   Master/slave  architecture   means  a  single  point  of  failure,   so  you  need  to  protect  your   master.   And  then  we  also  have  to   manage  the  MapReduce   processes  and  resources  in  the   Hadoop  layer.  
  57. Source:  socialbuXerflyclt.com  

  58. Jokes  aside,  no  one  said  open  source   was  easy

     to  use  
  59. To  be  expected   •  Hadoop/Hbase  are   designed  to

     move   mountains   •  If  you  want  to  move  big   stuff,  be  prepared  to   some>mes  use  big   equipment  
  60. What  it  means  to  a  startup   Development  capacity  before

      Development  capacity  aKer   Congrats,  you   are  now  a   sysadmin…  
  61. Whatever,  we  can  do  it!   Source:  hXp://knowyourmeme.com/memes/honey-­‐badger  

  62. Mapping  an  A-­‐List  to  a  column  store   Name  

    Ranks   References  to  influencer  records  
  63. Mapping  an  A-­‐List  to  a  column  store   Unique  

    key     “aXributes”   column  family   for  general   aXributes   “influencerId”  column  family   for  influencer  ranks  and  foreign  keys  
  64. Mapping  an  A-­‐List  to  a  column  store   Qualifiers  (basically

     aXribute  names)  
  65. Mapping  an  A-­‐List  to  a  column  store   “name”  aXribute

      Influencer  ranks   can  be  aXribute   names  as  well  
  66. Mapping  an  A-­‐List  to  a  column  store   Alist  name

      value   Influencer  id  values  assigned  to  each  rank  (basically   foreign  keys  to  an  influencer  table)  
  67. Mapping  an  A-­‐List  to  a  column  store   Can  get

     preXy  long  so  needs  indexing  and  pagina>on  
  68. Problem:  no  out-­‐of-­‐the-­‐box  row-­‐based   indexing  and  pagina>on  

  69. Whatever,  it’s  open-­‐source!   Source:  hXp://knowyourmeme.com/memes/honey-­‐badger  

  70. Jumping  right  into  the  code  

  71. MapReduce  for  batch  scoring   •  Need  to  re-­‐score  our

     influencer   database  once  a  week   •  M/R  cranked  through  it  in  15  mins  
  72. Source:  hXp://www.charliesheentshirts.info/  

  73. a  few  months  later…  

  74. Need  to  upgrade  to  Hbase  0.90   •  Making  sure

     to  remain  on  recent  code  base   •  Performance  improvements   •  Mostly  to  get  the  latest  bug  fixes   No  thanks!  
  75. Looks  like  something  is  missing  

  76. None
  77. Our  DB  indexes  depend  on  this!  

  78. Let’s  get  this  straight   •  Hbase  no  longer  comes

     with  secondary   indexing  out-­‐of-­‐the-­‐box   •  It’s  been  moved  out  of  the  trunk  to  GitHub   •  Where  only  one  other  company  besides  us   seems  to  care  about  it  
  79. Only  one  other   maintainer   besides  us  

  80. What  it  means  to  a  startup   Development  capacity  

    Congrats,  you  are   now  an  hbase   maintainer…  
  81. Source:  socialbuXerflyclt.com  

  82. Whatever,  we’ll  roll  our  own  indexing!   Source:  hXp://knowyourmeme.com/memes/honey-­‐badger  

  83. Homegrown  Hbase  Indexes   Rows  have  id  prefixes  that  can

     be   efficiently  scanned  using  STARTROW   and  STOPROW  filters   Row  ids  for  Posts  
  84. Homegrown  Hbase  Indexes   Find  posts  for   influencer_id_1234  

      Row  ids  for  Posts  
  85. Homegrown  Hbase  Indexes   Find  posts  for   influencer_id_5678  

    Row  ids  for  Posts  
  86. Homegrown  Hbase  Indexes   •  No  longer  depending  on  

    unmaintained  code   •  Work  with  out-­‐of-­‐the-­‐box  Hbase   installa>on  
  87. What  it  means  to  a  startup   Development  capacity  

    You  are  back  but   you  s>ll  need  to   maintain  indexing   logic  
  88. Source:  hXp://www.charliesheentshirts.info/   Applica>on  layer  indexes  are  slow  and  

    briXle.  The  DB  should  be  doing  this,  not   us.   Sort  of…  
  89. a  few  months  later…  

  90. Cracks  in  the  data  model   huffingtonpost.com   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under  
  91. Cracks  in  the  data  model   huffingtonpost.com   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under   Denormalized/duplicated   for  fast  run>me  access   and  storage  of  influencer-­‐ to-­‐site  rela>onship   proper>es  
  92. Cracks  in  the  data  model   huffingtonpost.com   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under   Content  aXribu>on  logic  could  some>mes   mis-­‐aXribute  posts  because  of  the   duplicated  data.  
  93. Cracks  in  the  data  model   huffingtonpost.com   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under   Exacerbated  when  we  started  tracking   people’s  content  on  a  daily  basis  in   mid-­‐2011  
  94. Fixing  the  cracks  in  the  data  model   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under  
  95. Fixing  the  cracks  in  the  data  model   huffingtonpost.com  

    hXp://www.huffingtonpost.com/arianna-­‐huffington/post_1.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_2.html   hXp://www.huffingtonpost.com/arianna-­‐huffington/post_3.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post1.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post2.html   hXp://www.huffingtonpost.com/shaun-­‐donovan/post3.html   writes  for   authored  by   published  under   writes  for   authored  by   published  under   Normalize  the  sites  
  96. Fixing  the  cracks  in  the  data  model   •  Normaliza>on

     requires  stronger   secondary  indexing   •  Our  applica>on  layer  indexing  would   need  revisi>ng…again!  
  97. What  it  means  to  a  startup   Development  capacity  

    Psych!  You  are  back   to  wri>ng  indexing   code.  
  98. Source:  socialbuXerflyclt.com  

  99. Whatever,  we’ll  change  our  NoSQL!   Source:  hXp://knowyourmeme.com/memes/honey-­‐badger  

  100. Traackr’s  Datastore  Requirements   (Revisited)   •  Schema  flexibility  

    •  Good  at  storing  lots  of  variable  length  text   •  Batch  processing  op>ons  (maybe)   •  Out-­‐of-­‐the-­‐box  SECONDARY  INDEX  support!   •  Simple  to  use  and  administer  
  101. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms    
  102. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Nope!  
  103. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Graph  Databases:  we  looked  at   Neo4J  a  bit  closer  but  passed  again   for  the  same  reasons  as  before.  
  104. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Memcache:  s>ll  no  
  105. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Amazon  SimpleDB:  s>ll  no.  
  106. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Not  willing  to  store  our  data  in  a   proprietary  datastore.   Redis  and  LinkedIn’s  Project   Voldermort:  s>ll  no  
  107. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     CouchDB:  more  mature  but  s>ll   no  ad-­‐hoc  queries.  
  108. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Cassandra:  matured  quite  a  bit,  added   secondary  indexes  and  batch  processing   op>ons  but  more  restric>ve  in  its’  use  than   other  solu>ons.  AKer  the  Hbase  lesson,   simplicity  of  use  was  now  more  important.  
  109. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     Riak:  strong  contender  s>ll  but   adop>on  ques>ons  remained.  
  110. NoSQL  picking  –  Round  2   Key/Value  Databases   • 

    Distributed  hashtables   •  Designed  for  high  load   •  In-­‐memory  or  on-­‐disk   •  Eventually  consistent   Column  Databases   •  Spread  sheet  like   •  Key  is  a  row  id   •  AXributes  are  columns   •  Columns  can  be  grouped   into  families   Document  Databases   •  Like  Key/Value   •  Value  =  Document   •  Document  =  JSON/BSON   •  JSON  =  Flexible  Schema   Graph  Databases   •  Graph  Theory  G=(E,V)   •  Great  for  modeling   networks   •  Great  for  graph-­‐based   query  algorithms     MongoDB:  matured  by  leaps  and  bounds,  increased   adop>on,  support  from  10gen,  advanced  indexing   out-­‐of-­‐the-­‐box  as  well  as  some  batch  processing   op>ons,  breeze  to  use,  well  documented  and  fit  into   our  exis>ng  code  base  very  nicely.  
  111. Immediate  Benefits   •  No  more  maintaining  custom  applica>on-­‐layer  

    secondary  indexing  code  
  112. What  it  means  to  a  startup   Development  capacity  

    Yay!  I’m  back!  
  113. Immediate  Benefits   •  No  more  maintaining  custom  applica>on-­‐layer  

    secondary  indexing  code   •  Single  binary  installa>on  greatly  simplifies   administra>on  
  114. What  it  means  to  a  startup   Development  capacity  

    Honestly,  I  thought   I’d  never  see  you   guys  again!  
  115. Immediate  Benefits   •  No  more  maintaining  custom  applica>on-­‐layer  

    secondary  indexing  code   •  Single  binary  installa>on  greatly  simplifies   administra>on   •  Our  NoSQL  could  now  support  our  domain   model  
  116. many-­‐to-­‐many   rela>onship  

  117. Modeling  an  influencer   Embedded  list  of   references  to

     sites   augmented  with   influencer-­‐specific   site  aXributes  (e.g.   percent  contribu>on   to  content)     {        ”_id":  "770cf5c54492344ad5e45„791ae5d52”,        "realName":  "David  Chancogne",        ">tle":  "CTO",        "descrip>on":  "Web.  Geek.\r\nTraackr:  hXp://traackr.com\r\nPropz:  hXp://propz.me",        "primaryAffilia>on":  "Traackr",        "email":  "dchancogne@traackr.com",        "loca>on":  "Cambridge,  MA,  United  States",        "siteReferences":  [              {                    "siteId":  "b31236da306270dc2b5db34e943af88d",                    "contribu>on":  0.25                },              {                    "siteId":  "602dc370945d3b3480fff4f2a541227c",                    "contribu>on":  1.0                }        ]   }  
  118. Modeling  an  influencer   siteId  indexed  for   “find  influencers

      connected  to  site  X”   > db.influencers.ensureIndex({siteReferences.siteId: 1});! > db.influencers.find({siteReferences.siteId: "602dc370945d3b3480fff4f2a541227c"});! {        ”_id":  "770cf5c54492344ad5e45„791ae5d52”,        "realName":  "David  Chancogne",        ">tle":  "CTO",        "descrip>on":  "Web.  Geek.\r\nTraackr:  hXp://traackr.com\r\nPropz:  hXp://propz.me",        "primaryAffilia>on":  "Traackr",        "email":  "dchancogne@traackr.com",        "loca>on":  "Cambridge,  MA,  United  States",        "siteReferences":  [              {                    "siteId":  "b31236da306270dc2b5db34e943af88d",                    "contribu>on":  0.25                },              {                    "siteId":  "602dc370945d3b3480fff4f2a541227c",                    "contribu>on":  1.0                }        ]   }  
  119. Embedded  list  of   influencer  references   augmented  with  

    “usernames”  (useful   for  content   aXribu>on)   {        ”_id":  "0001e86f73cc3975a29e6a98a41a4280”,        ”url":  "hXp://traackr.com/blog/",        "metrics":  [              {                  "name":  "google_inbound_links",                  "value":  5432              }          ],        "authors":  [              {                  "username":  "dchancogne",                  "influencerId":  "770cf5c54492344ad5e45„791ae5d52"              },            {                  "username":  ”gstathis",                  "influencerId":  "0001e86f73cc3975a29e6a98a41a4280"              }          ]   }   Modeling  a  site  
  120. Modeling  a  site   Indexed  for  “find   sites  associated

     to   influencer  X”   > db.sites.ensureIndex({authors.influencerId: 1});! > db.sites.find({authors.influencerId: "0001e86f73cc3975a29e6a98a41a4280"});! {        ”_id":  "0001e86f73cc3975a29e6a98a41a4280”,        ”url":  "hXp://traackr.com/blog/",        "metrics":  [              {                  "name":  "google_inbound_links",                  "value":  5432              }          ],        "authors":  [              {                  "username":  "dchancogne",                  "influencerId":  "770cf5c54492344ad5e45„791ae5d52"              },            {                  "username":  ”gstathis",                  "influencerId":  "0001e86f73cc3975a29e6a98a41a4280"              }          ]   }  
  121. Other  index  uses   Support  for  alternate  site  URLs  (a.k.a.

     URL  aliases):   {! "_id": "0001e86f73cc3975a29e6a98a41a4280",! "url_hash_list": [! {! "url": "http://traackr.com/blog",! "hash": "770cf5c54492344ad5e45fb791ae5d52"! },! {! "url": "http://blog.traackr.com/",! "hash": "0001e86f73cc3975a29e6a98a41a4280"! }! ]! }!   Indexed  for  “find   sites  associated  to   influencer  X”   Index  on  MD5  hash   of  URL  
  122. Other  Benefits   •  Ad  hoc  queries  and  reports  became

     easier  to  write  with  JavaScript:   no  need  for  a  Java  developer  to  write  map  reduce  code  to  extract   the  data  in  a  usable  form  like  it  was  needed  with  Hbase.  
  123. Ad  hoc  report  example   // File Name: retweetTotal.js! //

    Purpose: report the count of twitter URLs for which we have! // computed the the number of total retweets! print( "NUMBER OF TWITTER URLS where retweetTotal IS SET:" );! print( db.sites.find( { platformName: "twitter.com", ! retweetTotal: { $exists: true } } ).count() );! ! •  Easy  to  execute  JS  report  script  remotely:   ! > mongo <hostname>:<port>/traackr --quiet retweetTotal.js! ! •  Run  as  a  cron  job,  pipe  the  output  to  a  file  and  email  it  out   •  Also,  more  complex  MR-­‐based  reports  are  easily  accessible  to   someone  with  some  JavaScript  knowledge  
  124. Other  Benefits  (cont.)   •  Ad  hoc  queries  and  reports

     became  easier  to  write  with  JavaScript:   no  need  for  a  Java  developer  to  write  map  reduce  code  to  extract   the  data  in  a  usable  form  like  it  was  needed  with  Hbase.   •  Simpler  backups:  Hbase  mostly  relied  on  HDFS  redundancy;  intra-­‐ cluster  replica>on  is  available  but  experimental  and  a  lot  more   involved  to  setup.    
  125. Same  binary  can  be  deployed  several   >mes  for  replica>on

     &  backups  
  126. Same  binary  can  be  deployed  several   >mes  for  replica>on

     &  backups   Different  Availability   Zones  for  beXer  SPOF   tolerance  
  127. Same  binary  can  be  deployed  several   >mes  for  replica>on

     &  backups   priority  0  for  backup   server  so  that  it  never   gets  elected  as   primary  
  128. Same  binary  can  be  deployed  several   >mes  for  replica>on

     &  backups   Using  xfs_freeze   before  taking  backups  
  129. Same  binary  can  be  deployed  several   >mes  for  replica>on

     &  backups   EBS  snapshots  as   backups  are  portable   to  new  instances  (e.g.   QA)  
  130. Other  Benefits  (cont.)   •  Ad  hoc  queries  and  reports

     became  easier  to  write  with  JavaScript:   no  need  for  a  Java  developer  to  write  map  reduce  code  to  extract   the  data  in  a  usable  form  like  it  was  needed  with  Hbase.   •  Simpler  backups:  Hbase  mostly  relied  on  HDFS  redundancy;  intra-­‐ cluster  replica>on  is  available  but  experimental  and  a  lot  more   involved  to  setup.   •  Great  documenta>on   •  Great  adop>on  and  community  
  131. Mongo  cursors  for  batch  scoring   •  Mongo  is  fast

     enough  for  our  data  size  to   be  able  to  serially  score  the  DB  faster   than  the  MapReduce  jobs  did  in  parallel.   •  When  we  grow  larger,  MapReduce  is  s>ll   available  as  an  op>on  
  132. looks  like  we  found  the  right  fit!  

  133. We  have  more  of  this   Development  capacity  

  134. And  less  of  this   Source:  socialbuXerflyclt.com  

  135. Source:  hXp://www.charliesheentshirts.info/   for  now…  

  136. Addi>onal  takeaways   •  Fearless  refactoring   •  Ease  of

     use  and  administra>on  cannot  be   overstated  for  a  small  startup  
  137. Q&A