Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Qype API - Lessons learned

Qype API - Lessons learned

For Hamburg OnRuby and Hamburg Mobile DevCamp. A retrospective on building the Qype service API and lessons learned.

Matthias Käppler

August 04, 2012
Tweet

More Decks by Matthias Käppler

Other Decks in Technology

Transcript

  1. Qype  API:  Lessons  Learned  
    A  retrospec3ve  on  REST-­‐ful  API  design  
     
    July  2012  
    MaDhias  Käppler  
    Qype  GmbH  

    View Slide

  2. A  brief  history...  
    •  Qype’s  API  is  deployed  as  a  RoR  2.3  app  
    •  Online  since  end  of  2008  
    •  Primary  purpose  was  serving  the  iOS  app  
    •  Strictly  REST-­‐ful,  small  feature  set  
     
    GET http://api.qype.com/v1/places/1

    View Slide

  3. Architecture  
    DB  
    SOLR  
    API  app  
    Model  
    Mappings  
    Controllers  
    Website  app  
    AR  Models  
    Search  
    Logic  
    Search  
    Logic  
    SOLR  Ruby  bindings  
    Serializers  

    View Slide

  4. Today  
    •  Load  ~60M  req/month  
    •  ~2000  registered  consumers  
    •  99%  of  traffic  caused  by  0.01%  of  consumers  
    •  Consumers  vastly  differ  in  requirements  
    •  REST  paradigm  broken  up  in  many  places  

    View Slide

  5. Today  (cont.)  
    •  Over  the  years,  complexity  grew  with  the  
    requirements  
    – More  feature  rich  mobile  apps  
    – MORE  mobile  apps  
    – More  diverse  API  consumers  
    •  The  result?  

    View Slide

  6. Feature  creep  

    View Slide

  7. Prolifera3on  

    View Slide

  8. What  you  end  up  with  

    View Slide

  9. So  what’s  the  point?  

    View Slide

  10. API  design  is  hard.  

    View Slide

  11. Service  APIs  vs.  Websites  
    •  Mistakes  in  API  design  are  not  forgiven!  
    – Client  lock-­‐in  
    •  Clients  can  differ  vastly  
    – Mobile  app  vs.  Web  mash-­‐up  
    •  Authen3ca3on  and  access  control  
    – An  API  is  meant  to  access  your  core  data  by  design  
    •  Performance  maDers  (even  more)  

    View Slide

  12. How  can  we  fix  all  that?  

    View Slide

  13. Mistakes  in  API  design  are  not  
    forgiven.  
     
    No,  but...  

    View Slide

  14. Versioning.  

    View Slide

  15. Versioning  
    •  Granularity  
    – service  vs  endpoint  versioning  
    •  Architecture/deployment  
    – One  app  per  version?  
    •  E.g.  qype-­‐api-­‐v2  
    – One  engine  per  version?  
    •  E.g.  app/engines/v2  
    – One  class  per  version?  
    •  E.g.  app/controllers/resource_controller_v2.rb  

    View Slide

  16. Versioning  (cont.)  
    •  Tes3ng?  
    – i.e.  to  which  version  does  a  test  apply?  
    •  Code  sharing  vs.  code  duplica3on?  
    – i.e.  not  all  routes  may  change  with  an  increment  
    •  Also  affects  caching  etc.  
    •  Gems:  
    – hDps://github.com/bploetz/versionist  
    – hDps://github.com/filtersquad/rocket_pants  

    View Slide

  17. Clients  can  differ  vastly.  

    View Slide

  18. Build  abstrac3ons  for  things  that  
    change.  

    View Slide

  19. Modularity.  

    View Slide

  20. A  modular  architecture  
    Core  API  
    Apps  AAL   Partner  AAL  
    Standard  API  
    consumers  
    Partner  1   Partner  N  
    App  1   App  N  

    View Slide

  21. (Modularity.)  
     
    REST.  

    View Slide

  22. Seriously.  

    View Slide

  23. Why  REST?  
    •  Keeps  the  interface  lean  and  mean  
    •  Shared  understanding  of  ac3ons  
    •  Makes  a  system  connected  
    •  Makes  responses  cacheable  
    •  It’s  SIMPLE!  

    View Slide

  24. But:  simple  can  be  bad!  
    •  Our  apps  are  not  simple  
    •  Hence,  a  simple  API  is  not  always  suitable  
    •  REST  paradigm  can  be  sokened  on  an  AAL  
    – Batch  requests  
    – {  “requests”:  [  
       {  “method”:  “get”,  “uri”:  “hDp://...”  }  
       ...  
    ]  }  

    View Slide

  25. (Modularity.)  
     
    Resource  mapping.  

    View Slide

  26. Resource  mapping  
    •  Qype  API  employs  custom  resource  mapper  
    •  Goal:  
    – Map  AR  models  to  intermediate  format,  then  
    serialize  
    – Review.first.to_api(:hash)  
    •  Gems:  
    – hDps://github.com/rails/jbuilder/  
    – hDps://github.com/nesquena/rabl  

    View Slide

  27. (Modularity.)  
     
    Serializa3on.  

    View Slide

  28. json  >>  xml.  

    View Slide

  29. Authen3ca3on  &  access  control.  

    View Slide

  30. OAuth?  
     
     
     
    Leave  the  complexity  to  established  
    Web  technology.  

    View Slide

  31. Performance.  

    View Slide

  32. Caching,  caching,  caching.  

    View Slide

  33. Don’ts  and  Do’s.  

    View Slide

  34. Don’t:  
    GET  hDp://yourapi.com?consumer=x  
    if consumer == x

    ...

    end

    View Slide

  35. Don’t:  
    GET  hDp://yourapi.com/resource?a=1  
    “resource”: {

    “a+1”: 2
    }

    View Slide

  36. Don’t:  
    POST  hDp://yourapi.com/resource.json  
    field  missing  

    View Slide

  37. Do:  
     
     
     
    Be  consistent  with  API  status  codes.  

    View Slide

  38. Do:  
     
     
     
    Expect  bad  request  parameters.  

    View Slide

  39. Do:  
     
     
     
    Support  par3al  updates  (PATCH).  

    View Slide

  40. Do:  
     
     
     
    PREPARE  FOR  CHANGE!  

    View Slide

  41.  
     
     
    Outlook.  

    View Slide

  42. Acceptance  tes3ng  
    •  Crea3ng  an  acceptance  test  suite  for  a  service  
    API  is  not  simple  
    •  Recommended  for  deep  end-­‐2-­‐end  tests  
    •  Test  set-­‐up  /  tear-­‐down  
    •  RSpec  +  consumer  subjects  

    View Slide

  43. Documenta3on  
    •  Bad:  put  away  in  a  wiki  
    •  Good:  close  to  the  code  
    •  BeDer:  is  part  of  the  code  
    – Leverage  REST  representa3ons  
    – Self-­‐documen3ng  API  code  

    View Slide

  44.  
     
     
    Thanks!  

    View Slide