Softshake: Prod to be polyglot

This presentation was delivered during Softshake 2013. Learn why application developers should learn various programming languages and persistent stores

Transcript

1. Monday, October 28, 13
2. Polyglot  Applica-on   Development Tugdual  “Tug”  Grall Technical  Evangelist @tgrall tug@couchbase.com Monday, October 28, 13
3. Modern  Applica-ons • Modern  Applica-ons  Must ­ Manage  any  type  of  data ­ Scale ­ Be  fault  tolerant ­ Adapt  to  change Monday, October 28, 13
4. Standard  Architecture • Mono-­‐Technology ­ Same  “language/architecture” ­ Single  Database  :  RDBMS • One  size  fits  all  approach JSP/JSF Services Messaging JTA DAO/JPA Monday, October 28, 13
5. Is  it  good  for  me? • Many  files ­ XML,  Sources,  .. • Does  Hibernate/JPA  is  good  for  all? ­ Impedance  mismatch • Need  to  scale ­ from  0  to  50millions  users  in  6  weeks  (real  life) ­ can  I  do  that  easily  with  such  architecture? • How  to  add  new  features  easily? ­ code,  schema  change,  ... ­ and  all  this  for  yesterday! • Lot  of  code Monday, October 28, 13
6. One  size  does  NOT  fit  all! Monday, October 28, 13
7. Look  at  our  Data.... Monday, October 28, 13
8. 0 0.50 1.00 1.50 2.00 2011 2006 2000 Source:  IDC  2011  Digital  Universe  Study  (h<p://www.emc.com/collateral/demos/microsites/emc-­‐digital-­‐universe-­‐2011/index.htm) Trillions  of  Gigabytes  (Ze<abytes) Big  Data High  Data  Variety  and  Velocity Unstructured  and  Semi-­‐ Structured  Data Structured  Data Text,  Log  Files,  Click   Streams,  Blogs,   Tweets,  Audio,   Video,  etc. More  Flexible  Data  Model  Required Monday, October 28, 13
9. 50  Million  Users  in  50  Days 19 17 15 13 11 9 7 5 3 3/1 28 26 24 22 20 18 16 14 12 10 8 2/6 Draw  Something  by  OMGPOP Daily  Ac)ve  Users  (millions) 21 2 4 6 8 10 12 14 16 Monday, October 28, 13
10. RDBMS  is  good  for  many  thing,  but  hard  to  scale RDBMS  Scales  Up Get  a  bigger,  more  complex  server Users ApplicaIon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost ApplicaTon  Performance   Rela:onal  Database Web/App  Server  Tier System  Cost ApplicaTon  Performance   Won’t  scale   beyond  this   point How  do  you  take  this  growth? Monday, October 28, 13
11. Scaling  out  fla?ens  the  cost  and  performance  curves NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  Ier Users NoSQL  Distributed  Data  Store Web/App  Server  Tier ApplicaIon  Scales  Out Just  add  more  commodity  web  servers Users System  Cost ApplicaTon  Performance   ApplicaTon  Performance   System  Cost NoSQL  Technology  Scales  Out Monday, October 28, 13
12. Cloudera Hortonworks Mapr OperaTonal  vs.  AnalyTc  Databases Couchbase MongoDB Cassandra Hbase AnalyFc Databases Get  insights  from   data Real-­‐Fme,   InteracFve  Databases Fast  access   to  data NoSQL Monday, October 28, 13
13. Polyglot  Persistence Document  &  K/V •  Products •  User  Profiles •  Game  AcTons •  Sessions •  Shopping  Cart RDBMS •  Financial  Data •  ReporTng Big  Data/Analysis •  Log  Capture •  RecommendaTons •  AdCampaign Indexing •Full  Text  Search •Ad  hoc  queries Monday, October 28, 13
14. Developer  New  Skills • Understand  the  pros/cons  of  each  solu-ons ­ Programming  API ­ How  to  access  the  data?  (API,  Query  Languages) • Learn  how  to  design  data-­‐model ­ De-­‐normalizing  and  duplicate  data  is  not  a  problem • Integrate  each  solu-on  to  your  applica-on • Note:  This  is  something  you  already  do  ! ­ OperaTonal  RDBMS  and  Data  Warehouse Monday, October 28, 13
15. Data  Service  Layer Data  Service • Wrap  datastore  in  Services ­ loosely  coupled Monday, October 28, 13
16. Use  Case  :  Content  Driven  Site Monday, October 28, 13
17. Use  Case:  Mobile  Services PIM  Database -­‐  Legacy  Applica:on -­‐  Product  Informa:on NoSQL -­‐  Product  Data -­‐  Addi:onal  Metada Web/App  Server  Tier Web/App  Server  Tier Monday, October 28, 13
18. Now  what? Monday, October 28, 13
19. Why  mul-ple  languages? • Some-mes  we  have  no  choice ­ HTML/CSS/JavaScript  +  Server  Side ­ Mobile  NaTve  ApplicaTons • Java/.Net  have  not  evolved  that  must  these  past  years ­ Java  EE  5.0  is  sTll  mainstream  (2006) • Many  languages  and  frameworks ­ Scala,  Clojure,  Groovy,  Dart,  Go,  Ruby,  Python,  F#,  Erlang ­ Node.js,  Rails,  Play!,  Grails,  ... Monday, October 28, 13
20. Examples • Real-me  interac-on  with  WebSockets  : ­ Wait  for  Java  EE  7  to  be  out? ­ Hack  your  app  server  and  applicaTon  with  ConTnuaTon  ? ­ Use  Node.js  and  Socket.io  ? • Data  Collec-on  and  Treatment? ­ Find  a  library? ­ Create  your  library? ­ Wait  for  Java  8  and  Lambdas? ­ Use  Scala?  ...  or  other Monday, October 28, 13
21. Examples • Simple  CRUD  Applica-on ­ Use  the  JavaEE  stack  with  JPA  and  JSF? ­ Use  Spring  ? ­ Use  tools  like  Play!,  Grails,  Rails  ? Monday, October 28, 13
22. How  to  chose? • Maturity/Supportability   • Features  Set • Learning  Curve • Produc-vity Monday, October 28, 13
23. Polyglot  Programming  in  Ac-on • Build  a  distributed  database ­ Manage  distributed  processes ­ Save  and  retrieve  data  on  disk ­ Cache  the  data  on  RAM ­ Build  a  query  engine   Monday, October 28, 13
24. Polyglot  Programming  in  Ac-on • Manage  Distributed  Process ­ Erland  and  OTP  (Open  Telecom  Plaform)  has  been  built  for  that ­ Taking  this  will  bootstrap  a  project  quickly  instead  of  implemenTng  in   another  language • Data  access  and  Caching ­ C/C++  is  the  best/fastest  way  to  interact  with  I/O ­ Leverage  exisTng  Caching  layer,  for  example  Memcached  wrigen  in  C • Query  &  Indexing ­ Easy  to  code  for  developer  :  Let’s  use  JavaScript Monday, October 28, 13
25. Couchbase  Server  Architecture ReplicaTon,  Rebalance,     Shard  State  Manager REST  management   API/Web  UI 8091 Admin  Console Erlang  /OTP 11210  /  11211 Data  access  ports Object-­‐managed Cache Storage  Engine 8092 Query  API Query  Engine h\p Data  Manager Cluster  Manager Cluster  Manager WriOen  in  Erlang Data  Manager WriOen  in  C/C++ Monday, October 28, 13
26. Use  Case:  Second  Screen  Gaming Java/Play  Applica:on Python  scrips Administra-on Node.js  +  Socket  IO Monday, October 28, 13
27. Conclusion • Use  the  good  tool  for  the  good  task ­ Do  not  try  to  “hack”  something  when  you  have  a  shorter  soluTon ­ Think  about  your  users  first...  so  deliver  and  get  feedback • Learning  is  part  of  our  job  ! ­ And  this  will  help  you  and  your  project • What  about  maintenance? ­ Do  you  think  it  is  easier  to  maintain  complex  code? Monday, October 28, 13
28. Q&A @tgrall tug@couchbase.com Monday, October 28, 13