Qype API - Lessons learned

For Hamburg OnRuby and Hamburg Mobile DevCamp. A retrospective on building the Qype service API and lessons learned.

Transcript

1. Qype  API:  Lessons  Learned  
   A  retrospec3ve  on  REST-­‐ful  API  design  
    
   July  2012  
   MaDhias  Käppler  
   Qype  GmbH  
   

   View Slide

2. A  brief  history...  
   •  Qype’s  API  is  deployed  as  a  RoR  2.3  app  
   •  Online  since  end  of  2008  
   •  Primary  purpose  was  serving  the  iOS  app  
   •  Strictly  REST-­‐ful,  small  feature  set  
    
   GET http://api.qype.com/v1/places/1
   

   View Slide

3. Architecture  
   DB  
   SOLR  
   API  app  
   Model  
   Mappings  
   Controllers  
   Website  app  
   AR  Models  
   Search  
   Logic  
   Search  
   Logic  
   SOLR  Ruby  bindings  
   Serializers  
   

   View Slide

4. Today  
   •  Load  ~60M  req/month  
   •  ~2000  registered  consumers  
   •  99%  of  traffic  caused  by  0.01%  of  consumers  
   •  Consumers  vastly  differ  in  requirements  
   •  REST  paradigm  broken  up  in  many  places  
   

   View Slide

5. Today  (cont.)  
   •  Over  the  years,  complexity  grew  with  the  
   requirements  
   – More  feature  rich  mobile  apps  
   – MORE  mobile  apps  
   – More  diverse  API  consumers  
   •  The  result?  
   

   View Slide

6. Feature  creep  
   

   View Slide

7. Prolifera3on  
   

   View Slide

8. What  you  end  up  with  
   

   View Slide

9. So  what’s  the  point?  
   

   View Slide

10. API  design  is  hard.  
    

    View Slide

11. Service  APIs  vs.  Websites  
    •  Mistakes  in  API  design  are  not  forgiven!  
    – Client  lock-­‐in  
    •  Clients  can  differ  vastly  
    – Mobile  app  vs.  Web  mash-­‐up  
    •  Authen3ca3on  and  access  control  
    – An  API  is  meant  to  access  your  core  data  by  design  
    •  Performance  maDers  (even  more)  
    

    View Slide

12. How  can  we  fix  all  that?  
    

    View Slide

13. Mistakes  in  API  design  are  not  
    forgiven.  
     
    No,  but...  
    

    View Slide

14. Versioning.  
    

    View Slide

15. Versioning  
    •  Granularity  
    – service  vs  endpoint  versioning  
    •  Architecture/deployment  
    – One  app  per  version?  
    •  E.g.  qype-­‐api-­‐v2  
    – One  engine  per  version?  
    •  E.g.  app/engines/v2  
    – One  class  per  version?  
    •  E.g.  app/controllers/resource_controller_v2.rb  
    

    View Slide

16. Versioning  (cont.)  
    •  Tes3ng?  
    – i.e.  to  which  version  does  a  test  apply?  
    •  Code  sharing  vs.  code  duplica3on?  
    – i.e.  not  all  routes  may  change  with  an  increment  
    •  Also  affects  caching  etc.  
    •  Gems:  
    – hDps://github.com/bploetz/versionist  
    – hDps://github.com/filtersquad/rocket_pants  
    

    View Slide

17. Clients  can  differ  vastly.  
    

    View Slide

18. Build  abstrac3ons  for  things  that  
    change.  
    

    View Slide

19. Modularity.  
    

    View Slide

20. A  modular  architecture  
    Core  API  
    Apps  AAL   Partner  AAL  
    Standard  API  
    consumers  
    Partner  1   Partner  N  
    App  1   App  N  
    

    View Slide

21. (Modularity.)  
     
    REST.  
    

    View Slide

22. Seriously.  
    

    View Slide

23. Why  REST?  
    •  Keeps  the  interface  lean  and  mean  
    •  Shared  understanding  of  ac3ons  
    •  Makes  a  system  connected  
    •  Makes  responses  cacheable  
    •  It’s  SIMPLE!  
    

    View Slide

24. But:  simple  can  be  bad!  
    •  Our  apps  are  not  simple  
    •  Hence,  a  simple  API  is  not  always  suitable  
    •  REST  paradigm  can  be  sokened  on  an  AAL  
    – Batch  requests  
    – {  “requests”:  [  
       {  “method”:  “get”,  “uri”:  “hDp://...”  }  
       ...  
    ]  }  
    

    View Slide

25. (Modularity.)  
     
    Resource  mapping.  
    

    View Slide

26. Resource  mapping  
    •  Qype  API  employs  custom  resource  mapper  
    •  Goal:  
    – Map  AR  models  to  intermediate  format,  then  
    serialize  
    – Review.first.to_api(:hash)  
    •  Gems:  
    – hDps://github.com/rails/jbuilder/  
    – hDps://github.com/nesquena/rabl  
    

    View Slide

27. (Modularity.)  
     
    Serializa3on.  
    

    View Slide

28. json  >>  xml.  
    

    View Slide

29. Authen3ca3on  &  access  control.  
    

    View Slide

30. OAuth?  
     
     
     
    Leave  the  complexity  to  established  
    Web  technology.  
    

    View Slide

31. Performance.  
    

    View Slide

32. Caching,  caching,  caching.  
    

    View Slide

33. Don’ts  and  Do’s.  
    

    View Slide

34. Don’t:  
    GET  hDp://yourapi.com?consumer=x  
    if consumer == x

    ...

    end
    

    View Slide

35. Don’t:  
    GET  hDp://yourapi.com/resource?a=1  
    “resource”: {

    “a+1”: 2
    }
    

    View Slide

36. Don’t:  
    POST  hDp://yourapi.com/resource.json  
    field  missing  
    

    View Slide

37. Do:  
     
     
     
    Be  consistent  with  API  status  codes.  
    

    View Slide

38. Do:  
     
     
     
    Expect  bad  request  parameters.  
    

    View Slide

39. Do:  
     
     
     
    Support  par3al  updates  (PATCH).  
    

    View Slide

40. Do:  
     
     
     
    PREPARE  FOR  CHANGE!  
    

    View Slide

41.  
     
     
    Outlook.  
    

    View Slide

42. Acceptance  tes3ng  
    •  Crea3ng  an  acceptance  test  suite  for  a  service  
    API  is  not  simple  
    •  Recommended  for  deep  end-­‐2-­‐end  tests  
    •  Test  set-­‐up  /  tear-­‐down  
    •  RSpec  +  consumer  subjects  
    

    View Slide

43. Documenta3on  
    •  Bad:  put  away  in  a  wiki  
    •  Good:  close  to  the  code  
    •  BeDer:  is  part  of  the  code  
    – Leverage  REST  representa3ons  
    – Self-­‐documen3ng  API  code  
    

    View Slide

44.  
     
     
    Thanks!  
    

    View Slide