Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Reliability Guide

API Reliability Guide

Insights in to creating a robust, flexible and reliable API.

Nick DeNardis

June 11, 2012
Tweet

More Decks by Nick DeNardis

Other Decks in Technology

Transcript

  1. API Reliability Guide
    @nickdenardis / #psuweb12
    http://www.flickr.com/photos/erreeffe/3769670873/

    View full-size slide

  2. Nick DeNardis
    Associate Director of Web Communications
    at Wayne State University
    http://wayne.edu/
    Host of EDU Checkup
    http://educheckup.com/
    Curator of EDU Snippits
    http://edusnippits.com/
    Writer for .eduGuru
    http://doteduguru.com/

    View full-size slide

  3. Disclaimer
    Some assembly required.

    View full-size slide

  4. API
    Application Programming Interface

    View full-size slide

  5. Not just for robots
    http://www.flickr.com/photos/stevent/3241986538/

    View full-size slide

  6. Value Chain
    http://apigee.com/

    View full-size slide

  7. The API’s job is to
    make the
    developer as
    successful as
    possible
    http://knowyourmeme.com/memes/i-hate-sandcastles-success-kid

    View full-size slide

  8. It makes the user experience
    http://www.flickr.com/photos/daychokesnight/2149714792/
    can break

    View full-size slide

  9. Too slow...
    http://www.flickr.com/photos/toolmantim/6170448143/

    View full-size slide

  10. Too complicated...
    http://www.flickr.com/photos/toolmantim/6170448143/

    View full-size slide

  11. Adds a layer
    http://www.flickr.com/photos/jabb/6715983809/

    View full-size slide

  12. Complications with
    mobile
    “always on”

    View full-size slide

  13. Mobile isn’t
    going anywhere
    1.45 Million devices per day
    371,000 births per day
    http://www.lukew.com/ff/entry.asp?1506

    View full-size slide

  14. Mobile Data Traffic Expected To Rise 40-
    Fold Over Next Five Years
    http://techcrunch.com/2010/03/30/mobile-data-traffic-rise-40-fold/

    View full-size slide

  15. Mobile users use
    more bandwidth

    View full-size slide

  16. 250 kb - Avg page weight
    2.5 pages - Avg number per visit
    625 kb - Bandwidth per visit
    Desktop
    50 kb - Avg page weight
    25 pages - Avg number per visit
    1.25 mb - Bandwidth per visit
    Mobile
    0
    325
    650
    975
    1300
    Data
    Desktop Mobile

    View full-size slide

  17. 0
    10
    20
    30
    40
    2009 2010 2011 2012 2013 2014 2015 2016 2017
    Mobile Desktop
    Millions of
    visitors
    http://wayne.edu/

    View full-size slide

  18. The mobile Web is slow
    And it’s mostly our fault

    View full-size slide

  19. Time
    Cell Latency
    Initial HTML
    Javascript
    Images
    CSS
    You can’t blame the network for everything
    Time
    Cell Latency
    New Content
    Images
    Second Request
    First Request

    View full-size slide

  20. Do less better

    View full-size slide

  21. One size != fit all
    Think versioning from the start
    https://api.twitter.com/1/
    https://us2.api.mailchimp.com/1.3/
    https://api.foursquare.com/v2/
    https://api.instagram.com/v1/
    https://www.salesforce.com/services/Soap/c/18.0
    https://api.wayne.edu/v1/

    View full-size slide

  22. SOAP
    The request:
    GET /StockPrice HTTP/1.1
    Host: example.org
    Content-Type: application/soap+xml; charset=utf-8
    Content-Length: nnn

    xmlns:s="http://www.example.org/stock-service">


    IBM



    The response:
    HTTP/1.1 200 OK
    Content-Type: application/soap+xml; charset=utf-8
    Content-Length: nnn

    xmlns:s="http://www.example.org/stock-service">


    45.25



    The request:
    GET /StockPrice/IBM HTTP/1.1
    Host: example.org
    Accept: text/xml
    Accept-Charset: utf-8
    The response:
    HTTP/1.1 200 OK
    Content-Type: text/xml; charset=utf-8
    Content-Length: nnn


    IBM
    45.25

    REST
    4 kb vs 2 kb
    Round Trip

    View full-size slide

  23. Stick to REST

    View full-size slide

  24. XML sucks
    http://www.flickr.com/photos/philmanker/3654636770/

    View full-size slide

  25. Your best friend
    JSON
    Easy to encode:
    $arr = array('a' => 1, 'b' => 2, 'c' => 3, 'd' => 4, 'e' => 5);
    json_encode($arr);
    Easy to decode:
    $json = '{"a":1,"b":2,"c":3,"d":4,"e":5}';
    json_decode($json);
    Javascript:
    var myObject = eval('(' + json + ')');
    Douglas Crockford

    View full-size slide

  26. Existing Resources
    http://doteduguru.com/id7800-results-higher-ed-cms-usage-survey-2011.html

    View full-size slide

  27. Flattened highed =
    More interfaces
    CRM
    CMS
    iModules
    Events
    Course Schedule
    Parking
    Flickr
    YouTube
    Faculty Profiles
    Library Catalog

    View full-size slide

  28. Your data is everywhere
    CMS
    Events
    LDAP
    Banner
    Gather &
    Clean
    Shadow
    storage
    API
    Webserver
    Website
    Mobile
    Website
    Mobile
    App
    Digital
    Signage
    Third
    Party

    View full-size slide

  29. API is the glue
    CMS
    Events
    LDAP
    Banner
    Gather &
    Clean
    Shadow
    storage
    API
    Webserver
    Website
    Mobile
    Website
    Mobile
    App
    Digital
    Signage
    Third
    Party

    View full-size slide

  30. Rolling your own
    API
    Trust no one.

    View full-size slide

  31. Making the glue
    Server space you control
    api.domain.edu or domain.edu/api
    Ingredients:
    Ability:
    • Database
    • Cronjobs
    • Server logs
    • Analytics
    • PHP accelerator
    • Xdebug

    View full-size slide

  32. Getting data closer

    View full-size slide

  33. Documentation
    Design
    http://www.flickr.com/photos/edublogger/6950969837/

    View full-size slide

  34. Affordance
    The physical design will communicate how it
    is suppose to be used
    http://www.flickr.com/photos/hawkexpress/269032594/

    View full-size slide

  35. /getNews
    /getNewsReleases
    /findPeople
    /getPerson
    /eventsList
    /eventInfo
    /eventRSVP
    /classSearch
    /semesterList
    /parkingLotList
    /parkingAvailable

    View full-size slide

  36. Create. Read. Update. Delete.
    http://www.flickr.com/photos/fss/2181882493/

    View full-size slide

  37. Read first.
    http://www.flickr.com/photos/hackaday/4425372655/

    View full-size slide

  38. GET /academic/colleges/listing
    GET /academic/colleges/info/150
    REST
    Category
    Interface (Class)
    Function
    Item

    View full-size slide

  39. GET /academic/colleges/listing
    GET /academic/colleges/info
    GET /academic/majors/listing
    GET /academic/majors/info
    GET /academic/classes/listing
    GET /academic/classes/info
    GET /parking/availability/listing
    GET /parking/availability/info

    View full-size slide

  40. The devil is in the details
    Simple URL
    Response code
    Total count
    Data container
    Keep it
    lightweight
    91 KB

    View full-size slide

  41. Use only what you need
    Filters
    Less data
    41 KB

    View full-size slide

  42. Writing data

    View full-size slide

  43. POST /admissions/rfi/add
    POST /admissions/visit/add
    POST /admissions/application/add
    POST /academic/colleges/add
    POST /academic/colleges/edit
    POST /academic/majors/add
    POST /academic/majors/edit
    Post data:
    id=3047

    View full-size slide

  44. Soft Delete
    (keep all the data!)

    View full-size slide

  45. POST /academic/colleges/remove
    POST /academic/majors/remove
    POST /academic/classes/remove
    POST /parking/availability/remove
    Post data:
    id=3047

    View full-size slide

  46. Authentication
    /api/user/auth

    View full-size slide

  47. Auth workflow
    Request API Key Rate Limit
    Check Credentials
    Username/Pass
    Session ID
    Session ID
    (Do work) Return

    View full-size slide

  48. Rate Limiting
    100 per minute per Key/IP

    View full-size slide

  49. Cache. Cache.
    Cache.

    View full-size slide

  50. Professional cache
    http://www.flickr.com/photos/carlos/2417032795/

    View full-size slide

  51. APC
    $bar = 'BAR';
    apc_store('foo', $bar);
    var_dump(apc_fetch('foo'));
    ?>

    View full-size slide

  52. Static Cache
    http://www.flickr.com/photos/basic_sounds/5779597720/

    View full-size slide

  53. Static files
    if ((is_file($_SERVER['SCRIPT_FILENAME'].'.json'))
    && (time()-filemtime($_SERVER['SCRIPT_FILENAME'].'.json') < 3600))
    readfile($_SERVER['SCRIPT_FILENAME'].'.json');
    exit;
    }
    // (the php script itself goes here)
    echo $response;
    $fp = fopen($_SERVER['SCRIPT_FILENAME'].'.json', 'w');
    fwrite($fp, $response);
    fclose($fp);
    ?>

    View full-size slide

  54. if (typeof(localStorage) == 'undefined' ) {
    alert('Your browser does not support HTML5 localStorage. Try upgrading.');
    } else {
    try {
    localStorage.setItem("name", "Hello World!"); //saves to the database,
    } catch (e) {
    if (e == QUOTA_EXCEEDED_ERR) {
    alert('Quota exceeded!'); //data wasn't successfully saved due to quota exceed
    so throw an error
    }
    }
    document.write(localStorage.getItem("name")); //Hello World!
    localStorage.removeItem("name"); //deletes the matching item from the database
    }
    http://paperkilledrock.com/2010/05/html5-localstorage-part-one/
    HTML5
    localStorage

    View full-size slide

  55. JSONP
    function handle_data(data) {
    // `data` is now the object representation of the JSON data
    }
    ---
    http://some.tld/web/service?callback=handle_data:
    ---
    handle_data({"data_1": "hello world", "data_2":
    ["the","sun","is","shining"]});

    View full-size slide

  56. Expires header
    header('Expires: '.gmdate('D, d M Y H:i:s \G\M\T', time() + 3600));
    ?>
    Semesters
    Subjects
    Degrees
    Map Categories
    Map Locations
    1 Month
    Courses
    Media Experts
    1 Week
    Course Availability
    Parking Availability
    No Cache
    Events
    News
    1 Day

    View full-size slide

  57. Our Stats
    Feb 2011 - In production
    6.1 million requests
    48% from mobile
    21% iOS
    23% Android
    66% Web
    95% GET’s
    16 ms average response time

    View full-size slide

  58. Google API
    maps/locations/listing
    events/event/listing
    directory/people/listing
    academic/courses/listing

    View full-size slide

  59. maps/category/listing
    maps/location/info
    events/event/listing

    View full-size slide

  60. faculty/profile/info
    go/url/info

    View full-size slide

  61. Hackathon
    Community through data
    http://www.flickr.com/photos/hackny/5684887983/

    View full-size slide

  62. Resources
    • http://apigee.com/
    • http://rubyonrails.org/
    • http://cakephp.org/
    • https://groups.google.com/group/api-craft/
    • http://37signals.com/svn/posts/3018-api-design-for-humans
    • http://broadcast.oreilly.com/2011/06/the-good-the-bad-the-ugly-of-rest-
    apis.html
    • http://sixrevisions.com/html/introduction-web-storage/
    • http://webcomm.fiu.edu/2011/11/json-as-an-api-tool-and-why-its-awesome/
    • http://doteduguru.com/id4579-results-higher-ed-cms-usage.html

    View full-size slide

  63. Hackathons
    • http://dschool.stanford.edu/blog/2012/01/27/hack-d-kicks-off-more-than-a-
    dozen-projects-underway/
    • http://civic.mit.edu/blog/schock/occupydata-hackathon-2-roundup
    • http://newmed.media.mit.edu/health-and-wellness-innovation-2012
    • http://nyuad.nyu.edu/hackathon/about/
    • http://startup.berkeley.edu/hackathon/
    • http://www.njit.edu/hackathon/
    • http://www.lib.umich.edu/art-architecture-engineering-library/
    announcements/48-hour-mobile-app-hackathon

    View full-size slide

  64. Questions?
    Don’t be shy.

    View full-size slide

  65. Thank You
    @nickdenardis
    [email protected]
    http://nickdenardis.com/
    https://speakerdeck.com/u/nickdenardis

    View full-size slide