$30 off During Our Annual Pro Sale. View Details »

Qype API - Lessons learned

Qype API - Lessons learned

For Hamburg OnRuby and Hamburg Mobile DevCamp. A retrospective on building the Qype service API and lessons learned.

Matthias Käppler

August 04, 2012
Tweet

More Decks by Matthias Käppler

Other Decks in Technology

Transcript

  1. Qype  API:  Lessons  Learned   A  retrospec3ve  on  REST-­‐ful  API

     design     July  2012   MaDhias  Käppler   Qype  GmbH  
  2. A  brief  history...   •  Qype’s  API  is  deployed  as

     a  RoR  2.3  app   •  Online  since  end  of  2008   •  Primary  purpose  was  serving  the  iOS  app   •  Strictly  REST-­‐ful,  small  feature  set     GET http://api.qype.com/v1/places/1
  3. Architecture   DB   SOLR   API  app   Model

      Mappings   Controllers   Website  app   AR  Models   Search   Logic   Search   Logic   SOLR  Ruby  bindings   Serializers  
  4. Today   •  Load  ~60M  req/month   •  ~2000  registered

     consumers   •  99%  of  traffic  caused  by  0.01%  of  consumers   •  Consumers  vastly  differ  in  requirements   •  REST  paradigm  broken  up  in  many  places  
  5. Today  (cont.)   •  Over  the  years,  complexity  grew  with

     the   requirements   – More  feature  rich  mobile  apps   – MORE  mobile  apps   – More  diverse  API  consumers   •  The  result?  
  6. Feature  creep  

  7. Prolifera3on  

  8. What  you  end  up  with  

  9. So  what’s  the  point?  

  10. API  design  is  hard.  

  11. Service  APIs  vs.  Websites   •  Mistakes  in  API  design

     are  not  forgiven!   – Client  lock-­‐in   •  Clients  can  differ  vastly   – Mobile  app  vs.  Web  mash-­‐up   •  Authen3ca3on  and  access  control   – An  API  is  meant  to  access  your  core  data  by  design   •  Performance  maDers  (even  more)  
  12. How  can  we  fix  all  that?  

  13. Mistakes  in  API  design  are  not   forgiven.    

    No,  but...  
  14. Versioning.  

  15. Versioning   •  Granularity   – service  vs  endpoint  versioning  

    •  Architecture/deployment   – One  app  per  version?   •  E.g.  qype-­‐api-­‐v2   – One  engine  per  version?   •  E.g.  app/engines/v2   – One  class  per  version?   •  E.g.  app/controllers/resource_controller_v2.rb  
  16. Versioning  (cont.)   •  Tes3ng?   – i.e.  to  which  version

     does  a  test  apply?   •  Code  sharing  vs.  code  duplica3on?   – i.e.  not  all  routes  may  change  with  an  increment   •  Also  affects  caching  etc.   •  Gems:   – hDps://github.com/bploetz/versionist   – hDps://github.com/filtersquad/rocket_pants  
  17. Clients  can  differ  vastly.  

  18. Build  abstrac3ons  for  things  that   change.  

  19. Modularity.  

  20. A  modular  architecture   Core  API   Apps  AAL  

    Partner  AAL   Standard  API   consumers   Partner  1   Partner  N   App  1   App  N  
  21. (Modularity.)     REST.  

  22. Seriously.  

  23. Why  REST?   •  Keeps  the  interface  lean  and  mean

      •  Shared  understanding  of  ac3ons   •  Makes  a  system  connected   •  Makes  responses  cacheable   •  It’s  SIMPLE!  
  24. But:  simple  can  be  bad!   •  Our  apps  are

     not  simple   •  Hence,  a  simple  API  is  not  always  suitable   •  REST  paradigm  can  be  sokened  on  an  AAL   – Batch  requests   – {  “requests”:  [      {  “method”:  “get”,  “uri”:  “hDp://...”  }      ...   ]  }  
  25. (Modularity.)     Resource  mapping.  

  26. Resource  mapping   •  Qype  API  employs  custom  resource  mapper

      •  Goal:   – Map  AR  models  to  intermediate  format,  then   serialize   – Review.first.to_api(:hash)   •  Gems:   – hDps://github.com/rails/jbuilder/   – hDps://github.com/nesquena/rabl  
  27. (Modularity.)     Serializa3on.  

  28. json  >>  xml.  

  29. Authen3ca3on  &  access  control.  

  30. OAuth?         Leave  the  complexity  to  established

      Web  technology.  
  31. Performance.  

  32. Caching,  caching,  caching.  

  33. Don’ts  and  Do’s.  

  34. Don’t:   GET  hDp://yourapi.com?consumer=x   if consumer == x
 ...


    end
  35. Don’t:   GET  hDp://yourapi.com/resource?a=1   “resource”: {
 “a+1”: 2 }

  36. Don’t:   POST  hDp://yourapi.com/resource.json   <error>field  missing</error>  

  37. Do:         Be  consistent  with  API  status

     codes.  
  38. Do:         Expect  bad  request  parameters.  

  39. Do:         Support  par3al  updates  (PATCH).  

  40. Do:         PREPARE  FOR  CHANGE!  

  41.       Outlook.  

  42. Acceptance  tes3ng   •  Crea3ng  an  acceptance  test  suite  for

     a  service   API  is  not  simple   •  Recommended  for  deep  end-­‐2-­‐end  tests   •  Test  set-­‐up  /  tear-­‐down   •  RSpec  +  consumer  subjects  
  43. Documenta3on   •  Bad:  put  away  in  a  wiki  

    •  Good:  close  to  the  code   •  BeDer:  is  part  of  the  code   – Leverage  REST  representa3ons   – Self-­‐documen3ng  API  code  
  44.       Thanks!