Qype API - Lessons learned

For Hamburg OnRuby and Hamburg Mobile DevCamp. A retrospective on building the Qype service API and lessons learned.

Transcript

1. Qype  API:  Lessons  Learned   A  retrospec3ve  on  REST-­‐ful  API  design     July  2012   MaDhias  Käppler   Qype  GmbH  
2. A  brief  history...   •  Qype’s  API  is  deployed  as  a  RoR  2.3  app   •  Online  since  end  of  2008   •  Primary  purpose  was  serving  the  iOS  app   •  Strictly  REST-­‐ful,  small  feature  set     GET http://api.qype.com/v1/places/1
3. Architecture   DB   SOLR   API  app   Model   Mappings   Controllers   Website  app   AR  Models   Search   Logic   Search   Logic   SOLR  Ruby  bindings   Serializers  
4. Today   •  Load  ~60M  req/month   •  ~2000  registered  consumers   •  99%  of  traffic  caused  by  0.01%  of  consumers   •  Consumers  vastly  differ  in  requirements   •  REST  paradigm  broken  up  in  many  places  
5. Today  (cont.)   •  Over  the  years,  complexity  grew  with  the   requirements   – More  feature  rich  mobile  apps   – MORE  mobile  apps   – More  diverse  API  consumers   •  The  result?  
6. Feature  creep  
7. Prolifera3on  
8. What  you  end  up  with  
9. So  what’s  the  point?  
10. API  design  is  hard.  
11. Service  APIs  vs.  Websites   •  Mistakes  in  API  design  are  not  forgiven!   – Client  lock-­‐in   •  Clients  can  differ  vastly   – Mobile  app  vs.  Web  mash-­‐up   •  Authen3ca3on  and  access  control   – An  API  is  meant  to  access  your  core  data  by  design   •  Performance  maDers  (even  more)  
12. How  can  we  fix  all  that?  
13. Mistakes  in  API  design  are  not   forgiven.     No,  but...  
14. Versioning.  
15. Versioning   •  Granularity   – service  vs  endpoint  versioning   •  Architecture/deployment   – One  app  per  version?   •  E.g.  qype-­‐api-­‐v2   – One  engine  per  version?   •  E.g.  app/engines/v2   – One  class  per  version?   •  E.g.  app/controllers/resource_controller_v2.rb  
16. Versioning  (cont.)   •  Tes3ng?   – i.e.  to  which  version  does  a  test  apply?   •  Code  sharing  vs.  code  duplica3on?   – i.e.  not  all  routes  may  change  with  an  increment   •  Also  affects  caching  etc.   •  Gems:   – hDps://github.com/bploetz/versionist   – hDps://github.com/filtersquad/rocket_pants  
17. Clients  can  differ  vastly.  
18. Build  abstrac3ons  for  things  that   change.  
19. Modularity.  
20. A  modular  architecture   Core  API   Apps  AAL   Partner  AAL   Standard  API   consumers   Partner  1   Partner  N   App  1   App  N  
21. (Modularity.)     REST.  
22. Seriously.  
23. Why  REST?   •  Keeps  the  interface  lean  and  mean   •  Shared  understanding  of  ac3ons   •  Makes  a  system  connected   •  Makes  responses  cacheable   •  It’s  SIMPLE!  
24. But:  simple  can  be  bad!   •  Our  apps  are  not  simple   •  Hence,  a  simple  API  is  not  always  suitable   •  REST  paradigm  can  be  sokened  on  an  AAL   – Batch  requests   – {  “requests”:  [      {  “method”:  “get”,  “uri”:  “hDp://...”  }      ...   ]  }  
25. (Modularity.)     Resource  mapping.  
26. Resource  mapping   •  Qype  API  employs  custom  resource  mapper   •  Goal:   – Map  AR  models  to  intermediate  format,  then   serialize   – Review.first.to_api(:hash)   •  Gems:   – hDps://github.com/rails/jbuilder/   – hDps://github.com/nesquena/rabl  
27. (Modularity.)     Serializa3on.  
28. json  >>  xml.  
29. Authen3ca3on  &  access  control.  
30. OAuth?         Leave  the  complexity  to  established   Web  technology.  
31. Performance.  
32. Caching,  caching,  caching.  
33. Don’ts  and  Do’s.  
34. Don’t:   GET  hDp://yourapi.com?consumer=x   if consumer == x
 ...
 end
35. Don’t:   GET  hDp://yourapi.com/resource?a=1   “resource”: {
 “a+1”: 2 }
36. Don’t:   POST  hDp://yourapi.com/resource.json   <error>field  missing</error>  
37. Do:         Be  consistent  with  API  status  codes.  
38. Do:         Expect  bad  request  parameters.  
39. Do:         Support  par3al  updates  (PATCH).  
40. Do:         PREPARE  FOR  CHANGE!  
41.       Outlook.  
42. Acceptance  tes3ng   •  Crea3ng  an  acceptance  test  suite  for  a  service   API  is  not  simple   •  Recommended  for  deep  end-­‐2-­‐end  tests   •  Test  set-­‐up  /  tear-­‐down   •  RSpec  +  consumer  subjects  
43. Documenta3on   •  Bad:  put  away  in  a  wiki   •  Good:  close  to  the  code   •  BeDer:  is  part  of  the  code   – Leverage  REST  representa3ons   – Self-­‐documen3ng  API  code  
44.       Thanks!