Big Data (Small Print)

By Tyler Singletary @ API Strategy & Practice Conference
San Francisco, October 23-24-25, 2013

Transcript

1. Big Data* *small print 1 Tyler Singletary Director of Platform, Klout @harmophone
2. My Perspective •  Klout uses GNIP,Twitter, Facebook, Foursquare, LinkedIn and about a dozen other APIs. Consume terabytes/day. •  Distribute own data to partners (Salesforce, CrimsonHexagon) & channels (Gnip, Datasift) •  Over 1,700 monthly active developers 2
3. Finding a great API is only the first step •  The data is what you want! •  You get to it easily, it’s well organized! •  This is awesome! I’m going to do X,Y and Z! •  Did you read the TOS? 3
4. Constraints breed Creativity •  Knowing what you can’t do opens doors •  Legalese is a programming language •  Bend rules, but build a rollback •  Respect the data of others as if it were your own 4
5. Did you know? •  Facebook forbids you from distributing FBIDs •  Also from giving their data, or a derivative, to ad networks, etc. •  Twitter restricts you from separating location data from a tweet •  Nearly all require attribution, have cache and storage restrictions
6. A Rant About Scraping •  If a service has data you want, but doesn’t offer it in an API, do you have a right to it? •  Does this serve a long term business relationship? •  Whenever possible: have a relationship •  Robots.txt
7. Ubiquity •  If APIs are going to be the defacto way businesses communicate, then: •  The rules are important. •  They need to apply to innovators as well as established business •  Terms must scale and be programmatically delivered and executed •  When you judge an API you discover, think beyond your own need: •  If I’m taking this data, and 50 other companies are, what benefit do I have? They have? The provider has? •  You serve at the provider’s pleasure. •  How can APIs be industry-wide if industry is afraid of abuse?
8. Distributing Your Data •  What do you want to encourage? •  What do you want discourage? •  Contingencies! •  Be upfront and clear: cucumber stories. •  legalese: thar be dragons! Test coverage. •  What do you give exceptions for? 8
9. DRM •  No, not that DRM. •  Okay, you probably still think it’s that DRM. •  Data Rights Management
10. Robots.json •  A standard way to describe data rights •  Permissible use •  Data storage, caching •  Rate limits •  Copyrights, data stewardship •  License term, financial details
11. Alternatives •  Hypermedia? •  Requiring everyone to read and understand TOS and implement •  Trust? ignorance? •  Headers
12. Thank you! 12 Tyler Singletary Director of Platform, Klout @harmophone {can: republish, must: attribute, retention: unlimited}