API Driven Development

Presented at PHP North East 2013 in Newcastle, UK

Transcript

1. API Driven Development Presented by @alexbilbie Tuesday, 19 March 13
2. Eating your own dog food Tuesday, 19 March 13
3. Eating your own dog food. And everyone else’s. Tuesday, 19 March 13
4. Tweet @alexbilbie Blog alexbilbie.com Tuesday, 19 March 13
5. Developer at @unilincoln Tuesday, 19 March 13
6. Lincoln Tuesday, 19 March 13
7. Businesses rely on data Tuesday, 19 March 13
8. Information is processed data with context Tuesday, 19 March 13
9. Information allows decisions to be made which help a business grow Tuesday, 19 March 13
10. Ad-hoc sharing is prone to errors Tuesday, 19 March 13
11. Ad-hoc sharing has to change with the business (which can lead to more errors) Tuesday, 19 March 13
12. Ad-hoc data sharing is not scaleable Tuesday, 19 March 13
13. Jeff Bezos had ambitions to grow his small online bookshop Tuesday, 19 March 13
14. “All teams will henceforth expose their data and functionality through service interfaces. Tuesday, 19 March 13
15. Teams must communicate with each other through these interfaces. Tuesday, 19 March 13
16. There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. Tuesday, 19 March 13
17. It doesn’t matter what technology they use. Tuesday, 19 March 13
18. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. Tuesday, 19 March 13
19. Anyone who doesn’t do this will be fired...have a nice day” - Jeff Bezos, Amazon CEO Tuesday, 19 March 13
20. Amazon once just sold books. Now they make billions every year selling infrastructure as a service Tuesday, 19 March 13
21. They got there because they learnt how to effectively share data internally and externally Tuesday, 19 March 13
22. A Today’s letter Tuesday, 19 March 13
23. A P Today’s letterS Tuesday, 19 March 13
24. A P I Today’s letterS Tuesday, 19 March 13
25. API = application programming interface Tuesday, 19 March 13
26. Facade pattern Used to present an easier or simpler interface to an underlying implementation concept Tuesday, 19 March 13
27. machine to machine communication Tuesday, 19 March 13
28. What’s the difference between an API and a protocol ? Tuesday, 19 March 13
29. Protocols are about transport Email, voice, smoke signals Tuesday, 19 March 13
30. APIs are about messages ADDUSER:123, `sudo rm -rf /` Tuesday, 19 March 13
31. APIs allow resource sharing APIs allow functionality sharing APIs allow data sharing Tuesday, 19 March 13
32. APIs allow disparate systems to talk to each other Tuesday, 19 March 13
33. Universities are data hoarders Tuesday, 19 March 13
34. ALL THE DATA student records library items employee details research facilities energy usage events financial timetables journals inventory potential students Tuesday, 19 March 13
35. 100+ databases Sybase MySQL SQL Server Excel spreadsheets MS Access MySQL SQL Server SQL Server SQL Server SQL Server SQL Server SQL Server SQL Server SharePoint SQL Serve SQL MySQL MySQL My Excel spreadsheets SQL Server SQL Server ver Tuesday, 19 March 13
36. App Database Database Database Database Database App’s Database Tuesday, 19 March 13
37. @lncd make cool stuff Tuesday, 19 March 13
38. Nucleus2 Tuesday, 19 March 13
39. Nucleus Database Database Database Database Database Applications Database Database Tuesday, 19 March 13
40. = epic win Tuesday, 19 March 13
41. Today’s agenda •What is an API? •How can APIs help me? •How can my APIs help others? •How can others’ APIs help me? •How can others’ APIs help me help others? •How can my APIs help me help others? Tuesday, 19 March 13
42. I’m interested in web APIs Tuesday, 19 March 13
43. How can APIs help me? Tuesday, 19 March 13
44. Example situation: Tuesday, 19 March 13
45. I have a web app that allows businesses to manage employee data Tuesday, 19 March 13
46. My customers ask me for a mobile version, so I make iOS, Android and Blackberry apps Tuesday, 19 March 13
47. Later on I want to fix something. I also want to change the underlying data source. Tuesday, 19 March 13
48. But these changes affect a lot of things... Tuesday, 19 March 13
49. Every time I want to make a change: 10 Want to change something 20 Change it 30 Update web app 40 Update iOS app 50 Update Android app 60 Update Blackberry app 70 goto 10 Tuesday, 19 March 13
50. :-( Tuesday, 19 March 13
51. APIs Tuesday, 19 March 13
52. Web services !== APIs Tuesday, 19 March 13
53. API driven development can help solve problems like this Tuesday, 19 March 13
54. Every time I want to make a change: 10 Want to change something 20 Change it 30 Update web app 40 Update iOS app 50 Update Android app 60 Update Blackberry app 70 goto 10 Tuesday, 19 March 13
55. becomes... Tuesday, 19 March 13
56. Every time I want to make a change: 10 Want to change something 20 Change it 30 Test web app 40 Test iOS app 50 Test Android app 60 Test Blackberry app 70 goto 10 Tuesday, 19 March 13
57. Assuming your API doesn’t change it’s response you can easily change your underlying business processes as you please Tuesday, 19 March 13
58. APIs allow for rapid prototyping Tuesday, 19 March 13
59. Nucleus Applications Common Web Design OAuth 2.0 SSO Tuesday, 19 March 13
60. Google Reader Mail Calendar Maps Wave Translate Chrome Android Glass Tuesday, 19 March 13
61. Eating your own dog food Tuesday, 19 March 13
62. Increased internal use Tuesday, 19 March 13
63. Allowing others to eat your dog food Tuesday, 19 March 13
64. Early public web APIs •Salesforce (February 2000) •Access to data •eBay (November 2000) •Mass uploading of listings •Amazon (~2002) •Access to data Tuesday, 19 March 13
65. But even earlier that that we had SOAP and XML-RPC endpoints Tuesday, 19 March 13
66. In the 2000s RSS came along Tuesday, 19 March 13
67. Followed by ATOM (which then led to ATOM publishing) Tuesday, 19 March 13
68. Twitter boomed because of their open API Tuesday, 19 March 13
69. Why should I let others play in my sandbox? Tuesday, 19 March 13
70. Free bug reports Tuesday, 19 March 13
71. When bugs are fixed everyone benefits Tuesday, 19 March 13
72. Free feature suggestions Tuesday, 19 March 13
73. When new features are implemented everyone benefits Tuesday, 19 March 13
74. Free marketing Tuesday, 19 March 13
75. Makes you appear “open” and people like open Tuesday, 19 March 13
76. API itself can be a product Tuesday, 19 March 13
77. APIs require planning Tuesday, 19 March 13
78. Questions you should be asking before writing a line of code: •What functionality do my clients need? Tuesday, 19 March 13
79. Questions you should be asking before writing a line of code: •What functionality do my clients need? •What are appropriate responses? Tuesday, 19 March 13
80. Questions you should be asking before writing a line of code: •What functionality do my clients need? •What are appropriate responses? •What implications does this have for (any?) existing clients? Tuesday, 19 March 13
81. Questions you should be asking before writing a line of code: •What functionality do my clients need? •What are appropriate responses? •What implications does this have for (any?) existing clients? •Has anyone else made APIs like this? Can I learn anything from them? Tuesday, 19 March 13
82. Good separation of concerns Tuesday, 19 March 13
83. JSON > XML Tuesday, 19 March 13
84. Accept: Tuesday, 19 March 13
85. Content-type: Tuesday, 19 March 13
86. Don’t reinvent the wheel Tuesday, 19 March 13
87. REST Tuesday, 19 March 13
88. Representational State Transfer Tuesday, 19 March 13
89. GET PUT POST DELETE PATCH Tuesday, 19 March 13
90. GET something PUT POST DELETE PATCH Tuesday, 19 March 13
91. GET something PUT something new POST DELETE PATCH Tuesday, 19 March 13
92. GET something PUT something new POST an update to something DELETE PATCH Tuesday, 19 March 13
93. GET something PUT something new POST an update to something DELETE something PATCH Tuesday, 19 March 13
94. GET something PUT something new POST an update to something DELETE something PATCH something Tuesday, 19 March 13
95. PUT vs POST to create something Tuesday, 19 March 13
96. All good PHP frameworks support these concepts Tuesday, 19 March 13
97. APIs should be: Tuesday, 19 March 13
98. Simple Tuesday, 19 March 13
99. Intuitive Tuesday, 19 March 13
100. Stable* Tuesday, 19 March 13
101. Stable* * really fucking stable Tuesday, 19 March 13
102. Documented Tuesday, 19 March 13
103. Swagger http://zircote.com/swagger-php Tuesday, 19 March 13
104. use Swagger\Annotations as SWG; /** * @SWG\Operation( * httpMethod="GET", summary="Find pet by ID", notes="Returns a pet based on ID", * responseClass="Pet", nickname="getPetById" * ) */ function get_pet($id) { ! ... } Tuesday, 19 March 13
105. Tuesday, 19 March 13
106. Swagger JSON can be interpreted by client libraries e.g. Guzzle (PHP) Tuesday, 19 March 13
107. Architecture Tuesday, 19 March 13
108. Web server Database server Traditional stack Mobile App Tuesday, 19 March 13
109. API web server Database server New stack Web App Memcache server Solr server Mobile App Queue server Tuesday, 19 March 13
110. SPEED !!!! Tuesday, 19 March 13
111. Securing your API Tuesday, 19 March 13
112. HTTPS is your friend Tuesday, 19 March 13
113. Rate limiting Tuesday, 19 March 13
114. johndoe Username **************** Password Tuesday, 19 March 13
115. johndoe Username **************** Password Tuesday, 19 March 13
116. 3rd party clients should never ever ever be allowed near your users’ credentials Tuesday, 19 March 13
117. SAML = XML encoded assertions Tuesday, 19 March 13
118. OAuth Tuesday, 19 March 13
119. OAuth 1.0 is dead, long live OAuth 1.0 Tuesday, 19 March 13
120. OAuth 2.0 is finally standardised Tuesday, 19 March 13
121. Redirect then a POST request, all over HTTPS Tuesday, 19 March 13
122. Redirect then a POST request, all over HTTPS Simples! Tuesday, 19 March 13
123. Shameless plug: Tuesday, 19 March 13
124. github.com/lncd/oauth2 { "require": { "lncd/OAuth2": "*" } } Tuesday, 19 March 13
125. github.com/lncd/oauth2 // Include the storage models include 'model_scope.php'; include 'model_session.php'; // Initiate the Request handler $request = new \OAuth2\Util\Request(); // Initiate the auth server with the models $server = new \OAuth2\ResourceServer(new SessionModel, new ScopeModel); Tuesday, 19 March 13
126. $checkToken = function () use ($server) { return function() use ($server) { // Test for token existance and validity try { $server->isValid(); } // The access token is missing or invalid... catch (\OAuth2\Exception\InvalidAccessTokenException $e) { $app = \Slim\Slim::getInstance(); $res = $app->response(); $res['Content-Type'] = 'application/json'; $res->status(403); $res->body(json_encode(array( 'error' => $e->getMessage() ))); } }; }; Tuesday, 19 March 13
127. $app->get('/user/:id', $checkToken(), function ($id) use ($server, $app) { $user_model = new UserModel(); $user = $user_model->getUser($id); // Basic response $response = array( 'error' => null, 'result' => array( 'user_id' => $user['id'], 'firstname' => $user['firstname'], 'lastname' => $user['lastname'] ) ); // If the acess token has the "user.contact" access token include // an email address and phone number if ($server->hasScope('user.contact')) { $response['result']['email'] = $user['email']; $response['result']['phone'] = $user['phone']; } // Respond $res = $app->response(); $res['Content-Type'] = 'application/json'; $res->body(json_encode($response)); }); Tuesday, 19 March 13
128. Authorisation and resource server github.com/lncd/oauth2 OAuth 2 client code (nearly finished) github.com/lncd/oauth2-client Tuesday, 19 March 13
129. Making APIs work for you Tuesday, 19 March 13
130. Reporting Tuesday, 19 March 13
131. Dashboards Tuesday, 19 March 13
132. http://bit.ly/panicboard Tuesday, 19 March 13
133. Tuesday, 19 March 13
134. Useful APIs Tuesday, 19 March 13
135. Open Calais opencalais.com Tuesday, 19 March 13
136. My Society MapIt mapit.mysociety.org Tuesday, 19 March 13
137. Jenkins CI jenkins-ci.org Travis CI travis-ci.org Tuesday, 19 March 13
138. Github developer.github.com (see also: How Github use Github to build Github) bit.ly/github-use-github Tuesday, 19 March 13
139. Google Analytics developers.google.com/analytic Tuesday, 19 March 13
140. Twilio twilio.com Tuesday, 19 March 13
141. Pivotal Tracker pivotaltracker.com/help/api Tuesday, 19 March 13
142. Campfire github.com/37signals/campfire-api Tuesday, 19 March 13
143. Hubot hubot.github.com Tuesday, 19 March 13
144. Yahoo Pipes pipes.yahoo.com YQL developer.yahoo.com/yql Tuesday, 19 March 13
145. Tony Hirst (Yahoo Pipes + Google Spreadsheet guru) ouseful.info Tuesday, 19 March 13
146. Useful data APIs Tuesday, 19 March 13
147. data.gov.uk Tuesday, 19 March 13
148. DBpedia dbpedia.org Tuesday, 19 March 13
149. Linked data + SPARQL? Tuesday, 19 March 13
150. subject, object, predicate alex, knows, phil sturgeon alex, works at, university of lincoln Tuesday, 19 March 13
151. PREFIX abc: <http://example.com/exampleOntology#> SELECT ?capital ?country WHERE { ?x abc:cityname ?capital ; abc:isCapitalOf ?y . ?y abc:countryname ?country ; abc:isInContinent abc:Africa . } What are all the country capitals in Africa? Tuesday, 19 March 13
152. Guardian Datastore guardian.co.uk/data Tuesday, 19 March 13
153. Fin. Tuesday, 19 March 13
154. Any questions? Tweet me @alexbilbie Email me hello@alexbilbie.com Tuesday, 19 March 13