API Driven Development - Speaker Deck

API Driven Development

by Alex Bilbie

Slide 1

Slide 1 text

API Driven Development Presented by @alexbilbie Tuesday, 19 March 13

Slide 2

Slide 2 text

Eating your own dog food Tuesday, 19 March 13

Slide 3

Slide 3 text

Eating your own dog food. And everyone else’s. Tuesday, 19 March 13

Slide 4

Slide 4 text

Tweet @alexbilbie Blog alexbilbie.com Tuesday, 19 March 13

Slide 5

Slide 5 text

Developer at @unilincoln Tuesday, 19 March 13

Slide 6

Slide 6 text

Lincoln Tuesday, 19 March 13

Slide 7

Slide 7 text

Businesses rely on data Tuesday, 19 March 13

Slide 8

Slide 8 text

Information is processed data with context Tuesday, 19 March 13

Slide 9

Slide 9 text

Information allows decisions to be made which help a business grow Tuesday, 19 March 13

Slide 10

Slide 10 text

Ad-hoc sharing is prone to errors Tuesday, 19 March 13

Slide 11

Slide 11 text

Ad-hoc sharing has to change with the business (which can lead to more errors) Tuesday, 19 March 13

Slide 12

Slide 12 text

Ad-hoc data sharing is not scaleable Tuesday, 19 March 13

Slide 13

Slide 13 text

Jeﬀ Bezos had ambitions to grow his small online bookshop Tuesday, 19 March 13

Slide 14

Slide 14 text

“All teams will henceforth expose their data and functionality through service interfaces. Tuesday, 19 March 13

Slide 15

Slide 15 text

Teams must communicate with each other through these interfaces. Tuesday, 19 March 13

Slide 16

Slide 16 text

There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. Tuesday, 19 March 13

Slide 17

Slide 17 text

It doesn’t matter what technology they use. Tuesday, 19 March 13

Slide 18

Slide 18 text

All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. Tuesday, 19 March 13

Slide 19

Slide 19 text

Anyone who doesn’t do this will be ﬁred...have a nice day” - Jeﬀ Bezos, Amazon CEO Tuesday, 19 March 13

Slide 20

Slide 20 text

Amazon once just sold books. Now they make billions every year selling infrastructure as a service Tuesday, 19 March 13

Slide 21

Slide 21 text

They got there because they learnt how to eﬀectively share data internally and externally Tuesday, 19 March 13

Slide 22

Slide 22 text

A Today’s letter Tuesday, 19 March 13

Slide 23

Slide 23 text

A P Today’s letterS Tuesday, 19 March 13

Slide 24

Slide 24 text

A P I Today’s letterS Tuesday, 19 March 13

Slide 25

Slide 25 text

API = application programming interface Tuesday, 19 March 13

Slide 26

Slide 26 text

Facade pattern Used to present an easier or simpler interface to an underlying implementation concept Tuesday, 19 March 13

Slide 27

Slide 27 text

machine to machine communication Tuesday, 19 March 13

Slide 28

Slide 28 text

What’s the diﬀerence between an API and a protocol ? Tuesday, 19 March 13

Slide 29

Slide 29 text

Protocols are about transport Email, voice, smoke signals Tuesday, 19 March 13

Slide 30

Slide 30 text

APIs are about messages ADDUSER:123, `sudo rm -rf /` Tuesday, 19 March 13

Slide 31

Slide 31 text

APIs allow resource sharing APIs allow functionality sharing APIs allow data sharing Tuesday, 19 March 13

Slide 32

Slide 32 text

APIs allow disparate systems to talk to each other Tuesday, 19 March 13

Slide 33

Slide 33 text

Universities are data hoarders Tuesday, 19 March 13

Slide 34

Slide 34 text

ALL THE DATA student records library items employee details research facilities energy usage events ﬁnancial timetables journals inventory potential students Tuesday, 19 March 13

Slide 35

Slide 35 text

100+ databases Sybase MySQL SQL Server Excel spreadsheets MS Access MySQL SQL Server SQL Server SQL Server SQL Server SQL Server SQL Server SQL Server SharePoint SQL Serve SQL MySQL MySQL My Excel spreadsheets SQL Server SQL Server ver Tuesday, 19 March 13

Slide 36

Slide 36 text

App Database Database Database Database Database App’s Database Tuesday, 19 March 13

Slide 37

Slide 37 text

@lncd make cool stuﬀ Tuesday, 19 March 13

Slide 38

Slide 38 text

Nucleus2 Tuesday, 19 March 13

Slide 39

Slide 39 text

Nucleus Database Database Database Database Database Applications Database Database Tuesday, 19 March 13

Slide 40

Slide 40 text

= epic win Tuesday, 19 March 13

Slide 41

Slide 41 text

Today’s agenda •What is an API? •How can APIs help me? •How can my APIs help others? •How can others’ APIs help me? •How can others’ APIs help me help others? •How can my APIs help me help others? Tuesday, 19 March 13

Slide 42

Slide 42 text

I’m interested in web APIs Tuesday, 19 March 13

Slide 43

Slide 43 text

How can APIs help me? Tuesday, 19 March 13

Slide 44

Slide 44 text

Example situation: Tuesday, 19 March 13

Slide 45

Slide 45 text

I have a web app that allows businesses to manage employee data Tuesday, 19 March 13

Slide 46

Slide 46 text

My customers ask me for a mobile version, so I make iOS, Android and Blackberry apps Tuesday, 19 March 13

Slide 47

Slide 47 text

Later on I want to ﬁx something. I also want to change the underlying data source. Tuesday, 19 March 13

Slide 48

Slide 48 text

But these changes aﬀect a lot of things... Tuesday, 19 March 13

Slide 49

Slide 49 text

Every time I want to make a change: 10 Want to change something 20 Change it 30 Update web app 40 Update iOS app 50 Update Android app 60 Update Blackberry app 70 goto 10 Tuesday, 19 March 13

Slide 50

Slide 50 text

:-( Tuesday, 19 March 13

Slide 51

Slide 51 text

APIs Tuesday, 19 March 13

Slide 52

Slide 52 text

Web services !== APIs Tuesday, 19 March 13

Slide 53

Slide 53 text

API driven development can help solve problems like this Tuesday, 19 March 13

Slide 54

Slide 54 text

Every time I want to make a change: 10 Want to change something 20 Change it 30 Update web app 40 Update iOS app 50 Update Android app 60 Update Blackberry app 70 goto 10 Tuesday, 19 March 13

Slide 55

Slide 55 text

becomes... Tuesday, 19 March 13

Slide 56

Slide 56 text

Every time I want to make a change: 10 Want to change something 20 Change it 30 Test web app 40 Test iOS app 50 Test Android app 60 Test Blackberry app 70 goto 10 Tuesday, 19 March 13

Slide 57

Slide 57 text

Assuming your API doesn’t change it’s response you can easily change your underlying business processes as you please Tuesday, 19 March 13

Slide 58

Slide 58 text

APIs allow for rapid prototyping Tuesday, 19 March 13

Slide 59

Slide 59 text

Nucleus Applications Common Web Design OAuth 2.0 SSO Tuesday, 19 March 13

Slide 60

Slide 60 text

Google Reader Mail Calendar Maps Wave Translate Chrome Android Glass Tuesday, 19 March 13

Slide 61

Slide 61 text

Eating your own dog food Tuesday, 19 March 13

Slide 62

Slide 62 text

Increased internal use Tuesday, 19 March 13

Slide 63

Slide 63 text

Allowing others to eat your dog food Tuesday, 19 March 13

Slide 64

Slide 64 text

Early public web APIs •Salesforce (February 2000) •Access to data •eBay (November 2000) •Mass uploading of listings •Amazon (~2002) •Access to data Tuesday, 19 March 13

Slide 65

Slide 65 text

But even earlier that that we had SOAP and XML-RPC endpoints Tuesday, 19 March 13

Slide 66

Slide 66 text

In the 2000s RSS came along Tuesday, 19 March 13

Slide 67

Slide 67 text

Followed by ATOM (which then led to ATOM publishing) Tuesday, 19 March 13

Slide 68

Slide 68 text

Twitter boomed because of their open API Tuesday, 19 March 13

Slide 69

Slide 69 text

Why should I let others play in my sandbox? Tuesday, 19 March 13

Slide 70

Slide 70 text

Free bug reports Tuesday, 19 March 13

Slide 71

Slide 71 text

When bugs are ﬁxed everyone beneﬁts Tuesday, 19 March 13

Slide 72

Slide 72 text

Free feature suggestions Tuesday, 19 March 13

Slide 73

Slide 73 text

When new features are implemented everyone beneﬁts Tuesday, 19 March 13

Slide 74

Slide 74 text

Free marketing Tuesday, 19 March 13

Slide 75

Slide 75 text

Makes you appear “open” and people like open Tuesday, 19 March 13

Slide 76

Slide 76 text

API itself can be a product Tuesday, 19 March 13

Slide 77

Slide 77 text

APIs require planning Tuesday, 19 March 13

Slide 78

Slide 78 text

Questions you should be asking before writing a line of code: •What functionality do my clients need? Tuesday, 19 March 13

Slide 79

Slide 79 text

Questions you should be asking before writing a line of code: •What functionality do my clients need? •What are appropriate responses? Tuesday, 19 March 13

Slide 80

Slide 80 text

Questions you should be asking before writing a line of code: •What functionality do my clients need? •What are appropriate responses? •What implications does this have for (any?) existing clients? Tuesday, 19 March 13

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Good separation of concerns Tuesday, 19 March 13

Slide 83

Slide 83 text

JSON > XML Tuesday, 19 March 13

Slide 84

Slide 84 text

Accept: Tuesday, 19 March 13

Slide 85

Slide 85 text

Content-type: Tuesday, 19 March 13

Slide 86

Slide 86 text

Don’t reinvent the wheel Tuesday, 19 March 13

Slide 87

Slide 87 text

REST Tuesday, 19 March 13

Slide 88

Slide 88 text

Representational State Transfer Tuesday, 19 March 13

Slide 89

Slide 89 text

GET PUT POST DELETE PATCH Tuesday, 19 March 13

Slide 90

Slide 90 text

GET something PUT POST DELETE PATCH Tuesday, 19 March 13

Slide 91

Slide 91 text

GET something PUT something new POST DELETE PATCH Tuesday, 19 March 13

Slide 92

Slide 92 text

GET something PUT something new POST an update to something DELETE PATCH Tuesday, 19 March 13

Slide 93

Slide 93 text

GET something PUT something new POST an update to something DELETE something PATCH Tuesday, 19 March 13

Slide 94

Slide 94 text

GET something PUT something new POST an update to something DELETE something PATCH something Tuesday, 19 March 13

Slide 95

Slide 95 text

PUT vs POST to create something Tuesday, 19 March 13

Slide 96

Slide 96 text

All good PHP frameworks support these concepts Tuesday, 19 March 13

Slide 97

Slide 97 text

APIs should be: Tuesday, 19 March 13

Slide 98

Slide 98 text

Simple Tuesday, 19 March 13

Slide 99

Slide 99 text

Intuitive Tuesday, 19 March 13

Slide 100

Slide 100 text

Stable* Tuesday, 19 March 13

Slide 101

Slide 101 text

Stable* * really fucking stable Tuesday, 19 March 13

Slide 102

Slide 102 text

Documented Tuesday, 19 March 13

Slide 103

Slide 103 text

Swagger http://zircote.com/swagger-php Tuesday, 19 March 13

Slide 104

Slide 104 text

use Swagger\Annotations as SWG; /** * @SWG\Operation( * httpMethod="GET", summary="Find pet by ID", notes="Returns a pet based on ID", * responseClass="Pet", nickname="getPetById" * ) */ function get_pet($id) { ! ... } Tuesday, 19 March 13

Slide 105

Slide 105 text

Tuesday, 19 March 13

Slide 106

Slide 106 text

Swagger JSON can be interpreted by client libraries e.g. Guzzle (PHP) Tuesday, 19 March 13

Slide 107

Slide 107 text

Architecture Tuesday, 19 March 13

Slide 108

Slide 108 text

Web server Database server Traditional stack Mobile App Tuesday, 19 March 13

Slide 109

Slide 109 text

API web server Database server New stack Web App Memcache server Solr server Mobile App Queue server Tuesday, 19 March 13

Slide 110

Slide 110 text

SPEED !!!! Tuesday, 19 March 13

Slide 111

Slide 111 text

Securing your API Tuesday, 19 March 13

Slide 112

Slide 112 text

HTTPS is your friend Tuesday, 19 March 13

Slide 113

Slide 113 text

Rate limiting Tuesday, 19 March 13

Slide 114

Slide 114 text

johndoe Username **************** Password Tuesday, 19 March 13

Slide 115

Slide 115 text

johndoe Username **************** Password Tuesday, 19 March 13

Slide 116

Slide 116 text

3rd party clients should never ever ever be allowed near your users’ credentials Tuesday, 19 March 13

Slide 117

Slide 117 text

SAML = XML encoded assertions Tuesday, 19 March 13

Slide 118

Slide 118 text

OAuth Tuesday, 19 March 13

Slide 119

Slide 119 text

OAuth 1.0 is dead, long live OAuth 1.0 Tuesday, 19 March 13

Slide 120

Slide 120 text

OAuth 2.0 is ﬁnally standardised Tuesday, 19 March 13

Slide 121

Slide 121 text

Redirect then a POST request, all over HTTPS Tuesday, 19 March 13

Slide 122

Slide 122 text

Redirect then a POST request, all over HTTPS Simples! Tuesday, 19 March 13

Slide 123

Slide 123 text

Shameless plug: Tuesday, 19 March 13

Slide 124

Slide 124 text

github.com/lncd/oauth2 { "require": { "lncd/OAuth2": "*" } } Tuesday, 19 March 13

Slide 125

Slide 125 text

github.com/lncd/oauth2 // Include the storage models include 'model_scope.php'; include 'model_session.php'; // Initiate the Request handler $request = new \OAuth2\Util\Request(); // Initiate the auth server with the models $server = new \OAuth2\ResourceServer(new SessionModel, new ScopeModel); Tuesday, 19 March 13

Slide 126

Slide 126 text

$checkToken = function () use ($server) { return function() use ($server) { // Test for token existance and validity try { $server->isValid(); } // The access token is missing or invalid... catch (\OAuth2\Exception\InvalidAccessTokenException $e) { $app = \Slim\Slim::getInstance(); $res = $app->response(); $res['Content-Type'] = 'application/json'; $res->status(403); $res->body(json_encode(array( 'error' => $e->getMessage() ))); } }; }; Tuesday, 19 March 13

Slide 127

Slide 127 text

$app->get('/user/:id', $checkToken(), function ($id) use ($server, $app) { $user_model = new UserModel(); $user = $user_model->getUser($id); // Basic response $response = array( 'error' => null, 'result' => array( 'user_id' => $user['id'], 'firstname' => $user['firstname'], 'lastname' => $user['lastname'] ) ); // If the acess token has the "user.contact" access token include // an email address and phone number if ($server->hasScope('user.contact')) { $response['result']['email'] = $user['email']; $response['result']['phone'] = $user['phone']; } // Respond $res = $app->response(); $res['Content-Type'] = 'application/json'; $res->body(json_encode($response)); }); Tuesday, 19 March 13

Slide 128

Slide 128 text

Authorisation and resource server github.com/lncd/oauth2 OAuth 2 client code (nearly ﬁnished) github.com/lncd/oauth2-client Tuesday, 19 March 13

Slide 129

Slide 129 text

Making APIs work for you Tuesday, 19 March 13

Slide 130

Slide 130 text

Reporting Tuesday, 19 March 13

Slide 131

Slide 131 text

Dashboards Tuesday, 19 March 13

Slide 132

Slide 132 text

http://bit.ly/panicboard Tuesday, 19 March 13

Slide 133

Slide 133 text

Tuesday, 19 March 13

Slide 134

Slide 134 text

Useful APIs Tuesday, 19 March 13

Slide 135

Slide 135 text

Open Calais opencalais.com Tuesday, 19 March 13

Slide 136

Slide 136 text

My Society MapIt mapit.mysociety.org Tuesday, 19 March 13

Slide 137

Slide 137 text

Jenkins CI jenkins-ci.org Travis CI travis-ci.org Tuesday, 19 March 13

Slide 138

Slide 138 text

Github developer.github.com (see also: How Github use Github to build Github) bit.ly/github-use-github Tuesday, 19 March 13

Slide 139

Slide 139 text

Google Analytics developers.google.com/analytic Tuesday, 19 March 13

Slide 140

Slide 140 text

Twilio twilio.com Tuesday, 19 March 13

Slide 141

Slide 141 text

Pivotal Tracker pivotaltracker.com/help/api Tuesday, 19 March 13

Slide 142

Slide 142 text

Campﬁre github.com/37signals/campﬁre-api Tuesday, 19 March 13

Slide 143

Slide 143 text

Hubot hubot.github.com Tuesday, 19 March 13

Slide 144

Slide 144 text

Yahoo Pipes pipes.yahoo.com YQL developer.yahoo.com/yql Tuesday, 19 March 13

Slide 145

Slide 145 text

Tony Hirst (Yahoo Pipes + Google Spreadsheet guru) ouseful.info Tuesday, 19 March 13

Slide 146

Slide 146 text

Useful data APIs Tuesday, 19 March 13

Slide 147

Slide 147 text

data.gov.uk Tuesday, 19 March 13

Slide 148

Slide 148 text

DBpedia dbpedia.org Tuesday, 19 March 13

Slide 149

Slide 149 text

Linked data + SPARQL? Tuesday, 19 March 13

Slide 150

Slide 150 text

subject, object, predicate alex, knows, phil sturgeon alex, works at, university of lincoln Tuesday, 19 March 13

Slide 151

Slide 151 text

PREFIX abc: SELECT ?capital ?country WHERE { ?x abc:cityname ?capital ; abc:isCapitalOf ?y . ?y abc:countryname ?country ; abc:isInContinent abc:Africa . } What are all the country capitals in Africa? Tuesday, 19 March 13

Slide 152

Slide 152 text

Guardian Datastore guardian.co.uk/data Tuesday, 19 March 13

Slide 153

Slide 153 text

Fin. Tuesday, 19 March 13

Slide 154

Slide 154 text

Any questions? Tweet me @alexbilbie Email me [email protected] Tuesday, 19 March 13