Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Reliability Guide

API Reliability Guide

Insights in to creating a robust, flexible and reliable API.

Nick DeNardis

June 11, 2012

More Decks by Nick DeNardis

Other Decks in Technology


  1. API Reliability Guide @nickdenardis / #psuweb12 http://www.flickr.com/photos/erreeffe/3769670873/

  2. Nick DeNardis Associate Director of Web Communications at Wayne State

    University http://wayne.edu/ Host of EDU Checkup http://educheckup.com/ Curator of EDU Snippits http://edusnippits.com/ Writer for .eduGuru http://doteduguru.com/
  3. Disclaimer Some assembly required.

  4. API Application Programming Interface

  5. Not just for robots http://www.flickr.com/photos/stevent/3241986538/

  6. Value Chain http://apigee.com/

  7. The API’s job is to make the developer as successful

    as possible http://knowyourmeme.com/memes/i-hate-sandcastles-success-kid
  8. It makes the user experience http://www.flickr.com/photos/daychokesnight/2149714792/ can break

  9. Too slow... http://www.flickr.com/photos/toolmantim/6170448143/

  10. Too complicated... http://www.flickr.com/photos/toolmantim/6170448143/

  11. Adds a layer http://www.flickr.com/photos/jabb/6715983809/

  12. Complications with mobile “always on”

  13. Mobile isn’t going anywhere 1.45 Million devices per day 371,000

    births per day http://www.lukew.com/ff/entry.asp?1506
  14. Mobile Data Traffic Expected To Rise 40- Fold Over Next

    Five Years http://techcrunch.com/2010/03/30/mobile-data-traffic-rise-40-fold/
  15. Mobile users use more bandwidth

  16. 250 kb - Avg page weight 2.5 pages - Avg

    number per visit 625 kb - Bandwidth per visit Desktop 50 kb - Avg page weight 25 pages - Avg number per visit 1.25 mb - Bandwidth per visit Mobile 0 325 650 975 1300 Data Desktop Mobile
  17. 0 10 20 30 40 2009 2010 2011 2012 2013

    2014 2015 2016 2017 Mobile Desktop Millions of visitors http://wayne.edu/
  18. The mobile Web is slow And it’s mostly our fault

  19. Time Cell Latency Initial HTML Javascript Images CSS You can’t

    blame the network for everything Time Cell Latency New Content Images Second Request First Request
  20. Do less better

  21. One size != fit all Think versioning from the start

    https://api.twitter.com/1/ https://us2.api.mailchimp.com/1.3/ https://api.foursquare.com/v2/ https://api.instagram.com/v1/ https://www.salesforce.com/services/Soap/c/18.0 https://api.wayne.edu/v1/
  22. SOAP The request: GET /StockPrice HTTP/1.1 Host: example.org Content-Type: application/soap+xml;

    charset=utf-8 Content-Length: nnn <?xml version="1.0"?> <env:Envelope xmlns:env="http://www.w3.org/ 2003/05/soap-envelope" xmlns:s="http://www.example.org/stock-service"> <env:Body> <s:GetStockQuote> <s:TickerSymbol>IBM</s:TickerSymbol> </s:GetStockQuote> </env:Body> </env:Envelope> The response: HTTP/1.1 200 OK Content-Type: application/soap+xml; charset=utf-8 Content-Length: nnn <?xml version="1.0"?> <env:Envelope xmlns:env="http://www.w3.org/ 2003/05/soap-envelope" xmlns:s="http://www.example.org/stock-service"> <env:Body> <s:GetStockQuoteResponse> <s:StockPrice>45.25</s:StockPrice> </s:GetStockQuoteResponse> </env:Body> </env:Envelope> The request: GET /StockPrice/IBM HTTP/1.1 Host: example.org Accept: text/xml Accept-Charset: utf-8 The response: HTTP/1.1 200 OK Content-Type: text/xml; charset=utf-8 Content-Length: nnn <?xml version="1.0"?> <s:Quote xmlns:s="http://example.org/stock- service"> <s:TickerSymbol>IBM</s:TickerSymbol> <s:StockPrice>45.25</s:StockPrice> </s:Quote> REST 4 kb vs 2 kb Round Trip
  23. Stick to REST

  24. XML sucks http://www.flickr.com/photos/philmanker/3654636770/

  25. Your best friend JSON Easy to encode: $arr = array('a'

    => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5); json_encode($arr); Easy to decode: $json = '{"a":1,"b":2,"c":3,"d":4,"e":5}'; json_decode($json); Javascript: var myObject = eval('(' + json + ')'); Douglas Crockford
  26. Existing Resources http://doteduguru.com/id7800-results-higher-ed-cms-usage-survey-2011.html

  27. Flattened highed = More interfaces CRM CMS iModules Events Course

    Schedule Parking Flickr YouTube Faculty Profiles Library Catalog
  28. Your data is everywhere CMS Events LDAP Banner Gather &

    Clean Shadow storage API Webserver Website Mobile Website Mobile App Digital Signage Third Party
  29. API is the glue CMS Events LDAP Banner Gather &

    Clean Shadow storage API Webserver Website Mobile Website Mobile App Digital Signage Third Party
  30. Rolling your own API Trust no one.

  31. Making the glue Server space you control api.domain.edu or domain.edu/api

    Ingredients: Ability: • Database • Cronjobs • Server logs • Analytics • PHP accelerator • Xdebug
  32. Getting data closer

  33. Documentation Design http://www.flickr.com/photos/edublogger/6950969837/

  34. None
  35. Affordance The physical design will communicate how it is suppose

    to be used http://www.flickr.com/photos/hawkexpress/269032594/
  36. /getNews /getNewsReleases /findPeople /getPerson /eventsList /eventInfo /eventRSVP /classSearch /semesterList /parkingLotList

  37. Create. Read. Update. Delete. http://www.flickr.com/photos/fss/2181882493/

  38. Read first. http://www.flickr.com/photos/hackaday/4425372655/

  39. GET /academic/colleges/listing GET /academic/colleges/info/150 REST Category Interface (Class) Function Item

  40. GET /academic/colleges/listing GET /academic/colleges/info GET /academic/majors/listing GET /academic/majors/info GET /academic/classes/listing

    GET /academic/classes/info GET /parking/availability/listing GET /parking/availability/info
  41. The devil is in the details Simple URL Response code

    Total count Data container Keep it lightweight 91 KB
  42. Use only what you need Filters Less data 41 KB

  43. Writing data

  44. POST /admissions/rfi/add POST /admissions/visit/add POST /admissions/application/add POST /academic/colleges/add POST /academic/colleges/edit

    POST /academic/majors/add POST /academic/majors/edit Post data: id=3047
  45. Soft Delete (keep all the data!)

  46. POST /academic/colleges/remove POST /academic/majors/remove POST /academic/classes/remove POST /parking/availability/remove Post data:

  47. Authentication /api/user/auth

  48. Auth workflow Request API Key Rate Limit Check Credentials Username/Pass

    Session ID Session ID (Do work) Return
  49. Rate Limiting 100 per minute per Key/IP

  50. Cache. Cache. Cache.

  51. Professional cache http://www.flickr.com/photos/carlos/2417032795/

  52. APC <?php $bar = 'BAR'; apc_store('foo', $bar); var_dump(apc_fetch('foo')); ?>

  53. Static Cache http://www.flickr.com/photos/basic_sounds/5779597720/

  54. Static files <?php if ((is_file($_SERVER['SCRIPT_FILENAME'].'.json')) && (time()-filemtime($_SERVER['SCRIPT_FILENAME'].'.json') < 3600)) readfile($_SERVER['SCRIPT_FILENAME'].'.json');

    exit; } // (the php script itself goes here) echo $response; $fp = fopen($_SERVER['SCRIPT_FILENAME'].'.json', 'w'); fwrite($fp, $response); fclose($fp); ?>
  55. if (typeof(localStorage) == 'undefined' ) { alert('Your browser does not

    support HTML5 localStorage. Try upgrading.'); } else { try { localStorage.setItem("name", "Hello World!"); //saves to the database, } catch (e) { if (e == QUOTA_EXCEEDED_ERR) { alert('Quota exceeded!'); //data wasn't successfully saved due to quota exceed so throw an error } } document.write(localStorage.getItem("name")); //Hello World! localStorage.removeItem("name"); //deletes the matching item from the database } http://paperkilledrock.com/2010/05/html5-localstorage-part-one/ HTML5 localStorage
  56. JSONP function handle_data(data) { // `data` is now the object

    representation of the JSON data } --- http://some.tld/web/service?callback=handle_data: --- handle_data({"data_1": "hello world", "data_2": ["the","sun","is","shining"]});
  57. Expires header <?php header('Expires: '.gmdate('D, d M Y H:i:s \G\M\T',

    time() + 3600)); ?> Semesters Subjects Degrees Map Categories Map Locations 1 Month Courses Media Experts 1 Week Course Availability Parking Availability No Cache Events News 1 Day
  58. Our Stats Feb 2011 - In production 6.1 million requests

    48% from mobile 21% iOS 23% Android 66% Web 95% GET’s 16 ms average response time
  59. Examples

  60. Mobile news

  61. None
  62. Google API maps/locations/listing events/event/listing directory/people/listing academic/courses/listing

  63. maps/category/listing maps/location/info events/event/listing

  64. faculty/profile/info go/url/info

  65. Hackathon Community through data http://www.flickr.com/photos/hackny/5684887983/

  66. Resources • http://apigee.com/ • http://rubyonrails.org/ • http://cakephp.org/ • https://groups.google.com/group/api-craft/ •

    http://37signals.com/svn/posts/3018-api-design-for-humans • http://broadcast.oreilly.com/2011/06/the-good-the-bad-the-ugly-of-rest- apis.html • http://sixrevisions.com/html/introduction-web-storage/ • http://webcomm.fiu.edu/2011/11/json-as-an-api-tool-and-why-its-awesome/ • http://doteduguru.com/id4579-results-higher-ed-cms-usage.html
  67. Hackathons • http://dschool.stanford.edu/blog/2012/01/27/hack-d-kicks-off-more-than-a- dozen-projects-underway/ • http://civic.mit.edu/blog/schock/occupydata-hackathon-2-roundup • http://newmed.media.mit.edu/health-and-wellness-innovation-2012 • http://nyuad.nyu.edu/hackathon/about/

    • http://startup.berkeley.edu/hackathon/ • http://www.njit.edu/hackathon/ • http://www.lib.umich.edu/art-architecture-engineering-library/ announcements/48-hour-mobile-app-hackathon
  68. Questions? Don’t be shy.

  69. Thank You @nickdenardis [email protected] http://nickdenardis.com/ https://speakerdeck.com/u/nickdenardis