Automated  tes+ng  with     MongoDB  and  OffScale   Get  your  database  under  control   Omer  Gertel   CTO  @  OffScale   

Managing   databases  is   complicated   

Many  people   are  involved   Developers   QA   DBAs   IT   

Production Dev QA Integration MulAple   environments   

MulAple   copies  and   versions   

No content

We  Help  Companies…     Iterate  Faster  &   Release  Be-er  Products       By…     Shortening  product  cycles   with  DB  management   automa=on   

Production Dev QA Integration Copy  data   Write  code   Write  unit  tests   Clone  Schema   Copy  data   Test  the  applicaAon Clone  schema   Copy  data   Run  integraAon   

Copy  data   Write  code   Write  unit  tests   Clone  Schema   Copy  data   Test  the  applicaAon Clone  schema   Copy  data   Run  integraAon   DataGrove   Git  for  databases Production Dev QA Staging 

For  developers,  by  developers   $ datagrove tag -n=new_tag $ datagrove list $ datagrove load -n=old_tag $ datagrove unload •  Works  automagically   •  Easy  to  integrate  with  exisAng  tools   •  Your  personal  database  Ame  machine  for  FREE   

No content

Automated  TesAng   • IntegraAon  tests   • Build  servers  and  conAnuous  integraAon   • MigraAons   – Some  MongoDB  specific  issues   

Automated  Tests   •  Isolated   •  Repeatable   •  Researchable   •  Fast   

Automated  Tests   •  For  each  test:   – Populate  database   – Run  test   – Drop  database   

What  happens  if  a  test  breaks?   •  Does  it  affect  other  tests?   •  Does  it  stop  the  build  cycle?   •  How  do  you  discover  what  went  wrong?   •  Is  the  bug  related  to  the  data?   •  How  can  you  reproduce  the  bug?   •  What  happens  when  you  push  a  fix?   

Automated  Tests   •  For  each  test:   – Load  dataset   – Run  test   – If  test  fails:   •  Tag  dataset  at  end  of  run   

Schema  MigraAons   

Schema  MigraAons   

•  Migrate  ALL  the  objects   – Test  before  you  run   •  Support  older  structures  in  code   – Test  code  against  old  objects   – Have  datasets  ready  tagged  by  version   Schema  MigraAons   Object  Structure  &  Index  Changes   

       Make  MongoDB  a  first   class  ciAzen  of  the   development  cycle   Version  management  that  doesn’t   suck   

ExisAng  MongoDB  Support   For  single  node  environments   $ datagrove load - n=tag_name Create  virtual  mount  point:     /media/offscale   (1-­‐10  sec)   mongod  -­‐-­‐dpath=  /media/offscale   /media/offscale   Mongod   path-­‐to-­‐old-­‐data   path-­‐to-­‐store-­‐ changes   (opAonal)   

ExisAng  MongoDB  Support   For  single  node  environments   $ datagrove tag - n=tag_name Tag  point  in  Ame  in  virtual  drive   (1-­‐10  sec)   {fsync:1,lock:1}     (using  Java  driver)   /media/offscale   Mongod   path-­‐to-­‐old-­‐data   path-­‐to-­‐store-­‐ changes   (opAonal)   db.$cmd.sys.unlock.findOne()   

Replica  Set   Primary   Slave   Slave   Replica  Set   Primary   Slave   Slave   Upcoming  MongoDB  Cluster  Support    Conceptual  architecture  for  taking  a  snapshot   Replica  Set   Primary   Slave   Slave   One  node  managed  under   OffScale,  per  replica  set,   allows  for  backup,  recovery   and  creaAng  new  nodes   ConfiguraAon   cluster   Cfg  1   Cfg  2   Cfg  3   One  or  more  nodes   managed  by  OffScale  for   backup  and  recovery  of   configuraAon  servers   $ datagrove tag-n=ta Issue  datagrove  tag  for   configuraAon  server   Pause  shard   balancing   Resume  shard   balancing   Issue  datagrove  tag,   one  shard  at  a  Ame   

•  Data  is  a  first  class   ciAzen  in  your   environment   •  TesAng  is  important  to   keep  your  code  working   •  Iterate  faster  &  release   beher  products   Automated  tesAng  with     OffScale  &  MongoDB   

Now  available  at   hhp://off-­‐scale.com/   Get  your  database  under  control