OSCON 2017 - Contract-First API Development Using the OpenAPI Specification (fka Swagger)

Slide 1

Slide 1 text

Contract-first API development using the OpenAPI Specification (fka Swagger)

Slide 2

Slide 2 text

Ian Zelikman @izcoder Dave Forgac @tylerdave

Slide 3

Slide 3 text

Questions

Slide 4

Slide 4 text

BetterAPIs.com https://github.com/tylerdave/OpenAPI-Tutorial/

Slide 5

Slide 5 text

Updates Get repo updates git pull Reprovision VM vagrant suspend vagrant reload --provision Log in vagrant ssh

Slide 6

Slide 6 text

Backup Plan http://editor.swagger.io/

Slide 7

Slide 7 text

Background

Slide 8

Slide 8 text

REST

Slide 9

Slide 9 text

REST Standard

Slide 10

Slide 10 text

REST Standard Architectural Style

Slide 11

Slide 11 text

REST Standard Architectural Style HTTP w/ Constraints

Slide 12

Slide 12 text

REST Standard Architectural Style HTTP w/ Constraints REST-inspired HTTP APIs

Slide 13

Slide 13 text

API Contract

Slide 14

Slide 14 text

API Contract Client ↔ Provider

Slide 15

Slide 15 text

API Contract Client ↔ Provider Interface Specification

Slide 16

Slide 16 text

API Contract Client ↔ Provider Interface Specification SLA, ToS, Limits, Pricing, etc.

Slide 17

Slide 17 text

JSON

Slide 18

Slide 18 text

JSON JavaScript Object Notation

Slide 19

Slide 19 text

JSON JavaScript Object Notation { "things": [ "foo", "bar" ], "message": "Hello, World!" }

Slide 20

Slide 20 text

JSON Schema

Slide 21

Slide 21 text

JSON Schema { "title": "Example Schema", "type": "object", "properties": { "displayName": { "type": "string" }, "age": { "description": "Age in years", "type": "integer", "minimum": 0 } }, "required": ["firstName", "lastName"] }

Slide 22

Slide 22 text

YAML

Slide 23

Slide 23 text

YAML Serialization format

Slide 24

Slide 24 text

YAML Serialization format (More) human-readable

Slide 25

Slide 25 text

YAML Serialization format (More) human-readable Superset of JSON

Slide 26

Slide 26 text

YAML Serialization format (More) human-readable Superset of JSON Language Support

Slide 27

Slide 27 text

YAML Basics - Lists --- colors: - red - green - blue ...

Slide 28

Slide 28 text

YAML Basics - Dictionaries --- session: title: Contract-First API Development type: tutorial ...

Slide 29

Slide 29 text

YAML Basics - Spanning --- description: | This is a long description using a pipe which will preserve newlines. description2: > This is a long desciption using > which will ignore new lines. ...

Slide 30

Slide 30 text

YAML Basics - Nesting --- session: name: Contract-First API Development type: tutorial topics: - apis - openapi specification - swagger languages: ['java', 'nodejs', 'python'] description: > A really useful tutorial during which you'll learn about API specifications and stuff. ...

Slide 31

Slide 31 text

JSON "Talk": { "type": "object", "properties": { "id": { "type": "integer" }, "title": { "minLength": 1, "type": "string", "maxLength": 144 } } } YAML Talk: type: object properties: id: type: integer title: type: string minLength: 1 maxLength: 144 Compare

Slide 32

Slide 32 text

API Definitions

Slide 33

Slide 33 text

API Definitions WSDL / WADL

Slide 34

Slide 34 text

API Definitions WSDL / WADL Swagger -> OpenAPI Spec

Slide 35

Slide 35 text

API Definitions WSDL / WADL Swagger -> OpenAPI Spec API Blueprint

Slide 36

Slide 36 text

API Definitions WSDL / WADL Swagger -> OpenAPI Spec API Blueprint RAML

Slide 37

Slide 37 text

OpenAPI Spec

Slide 38

Slide 38 text

OpenAPI Spec Structure

Slide 39

Slide 39 text

OpenAPI Spec Structure History

Slide 40

Slide 40 text

OpenAPI Spec Structure History Future

Slide 41

Slide 41 text

OpenAPI 3.0

Slide 42

Slide 42 text

OpenAPI 3.0 Coming Soon

Slide 43

Slide 43 text

OpenAPI 3.0 Coming Soon Tooling to Follow

Slide 44

Slide 44 text

This Tutorial

Slide 45

Slide 45 text

Goals

Slide 46

Slide 46 text

Goals OpenAPI Spec

Slide 47

Slide 47 text

Goals OpenAPI Spec Testing

Slide 48

Slide 48 text

Goals OpenAPI Spec Testing Mock

Slide 49

Slide 49 text

Goals OpenAPI Spec Testing Mock Basic Implementation

Slide 50

Slide 50 text

Goals OpenAPI Spec Testing Mock Basic Implementation Documentation

Slide 51

Slide 51 text

Repo Layout ├── implementation │ └── ... ├── lessons │ ├── lesson-1.01 │ │ ├── default_broken.yaml │ │ └── solution.yaml │ ├── lesson-1.02 │ │ ├── example.json │ ... ... │ ├── lesson-2.01 │ │ └── README.md │ └── ... ├── presentation │ └── ... └── work

Slide 52

Slide 52 text

Synced Folder Repo dir on host mapped to /home/ubuntu/tutorial-repo/ on VM

Slide 53

Slide 53 text

Lessons Instructions Work in work directory Via editor on host machine Or via editor in VM terminal Save betterapis.yml Run validator within VM: swagger validate tutorial-repo/work/betterapis.yml

Slide 54

Slide 54 text

Lesson Solutions Done? Compare with contents of lessons/lesson-x.xx/solution.xxx Stuck? Copy lessons/lesson-x.xx/solution.xxx to work

Slide 55

Slide 55 text

Tutorial

Slide 56

Slide 56 text

Lesson 1.01: Setup Goals Explore the environment. Look at some Open API example specs and exercise the tools we will use.

Slide 57

Slide 57 text

Lesson 1.01: Setup Tooling Swagger editor: http://localhost:8000/ Validator: swagger validate tutorial-repo/lessons/lesson-1.02/solution.yaml

Slide 58

Slide 58 text

Lesson 1.01: Setup Exercise Instructions Load several examples from the swagger editor, review them. Import the broken examples from lesson-1.01 directory. Try fixing the errors.

Slide 59

Slide 59 text

Solution 1.01 Notes Got familiar with basic OpenAPI Spec structure

Slide 60

Slide 60 text

Lesson 1.02: Hello, World! Goals Building a first (basic) spec.

Slide 61

Slide 61 text

YAML Example

Slide 62

Slide 62 text

JSON Example

Slide 63

Slide 63 text

Lesson 1.02: Hello, World! Exercise Instructions Build an API for a conference called betterapis Include metadata as shown in the example Paths are empty for now

Slide 64

Slide 64 text

Solution 1.02 Notes This solution might be a bit different than yours in regards to the metadata. Valid Spec!

Slide 65

Slide 65 text

Lesson 1.03: Pets Goals Get familiar with defining paths.

Slide 66

Slide 66 text

Lesson 1.03: Pets Basic Path /pets: get: summary: Get a list of pets description: Retrieve a list of pets responses: 201: description: OK

Slide 67

Slide 67 text

Lesson 1.03: Pets Exercise Instructions Add two paths to the API: /talks /speakers . Both paths only support GET and only return status code 200.

Slide 68

Slide 68 text

Solution 1.03 Notes We defined the very basic fields and objects needed for a valid path.

Slide 69

Slide 69 text

Lesson 1.04: Registration Goals Learn to define complex operations on the API.

Slide 70

Slide 70 text

Lesson 1.04: Registration Paths, Actions paths: /pets: post: summary: Add pet to DB description: Results in new pet information added to the DB parameters: - name: pet in: body description: Pet details schema: required: [name, status] properties: name: type: string description: The pet name

Slide 71

Slide 71 text

Lesson 1.04: Registration Paths, Actions (contd.) responses: 201: description: Created new pet in the database schema: required: [pet-id] properties: pet-id: type: number description: Unique Id for the pet in the system

Slide 72

Slide 72 text

Lesson 1.04: Registration Exercise Instructions Add actions to support speaker registration and talk submission You are free to define the speaker and talk objects as you like as long as you define a unique id in both and exercise defining more than one basic type for the object properties.

Slide 73

Slide 73 text

Solution 1.04 Notes Additional properties: readOnly , format , pattern

Slide 74

Slide 74 text

Lesson 1.05: The Minimalist API Goals Reusing definitions. Learn more in depth about action objects and request parameters.

Slide 75

Slide 75 text

Lesson 1.05: The Minimalist API Path Parameter /pets/{pet-id}: parameters: pet-id: name: pet-id in: path description: Pet identifier type: number required: true

Slide 76

Slide 76 text

Lesson 1.05: The Minimalist API Parameter Reuse /pets/{pet-id}: parameters: - $ref: '#/parameters/pet-id' ... parameters: pet-id: name: pet-id in: path description: Pet identifier type: number required: true

Slide 77

Slide 77 text

Lesson 1.05: The Minimalist API Definitions Reuse ... schema: $ref: '#/definitions/Pet' ... definitions: Pet: type: object required: [name, status] properties: ... Pets: type: array items: $ref: "#/definitions/Pet"

Slide 78

Slide 78 text

Lesson 1.05: The Minimalist API Exercise Instructions 1. Refactor your API to use Talk and Speaker objects. Define Talks and Speakers objects based on the previous and update the responses from /speakers and /talks paths. 2. Add a two new paths /speakers/{speaker-id} and /talks/{talk-id} . Define all the CRUD operations for them and use parameter definition outside of the action for path parameter.

Slide 79

Slide 79 text

Solution 1.05 Notes Time saving with definition. More readable. Example response in the solution

Slide 80

Slide 80 text

Lesson 1.06: Responses Goals Learn more about parameter definition via pagination Learn How to define reusable responses Default responses

Slide 81

Slide 81 text

Lesson 1.06: Responses Pagination parameters: - $ref: '#/parameters/page-size' - $ref: '#/parameters/page-number' ... parameters: page-size: name: page-size in: query description: Number of items type: integer format: int32 minimum: 1 maximum: 100 multipleOf: 10 default: 10

Slide 82

Slide 82 text

Lesson 1.06: Responses Response Definition responses: ServerErrorResponse: description: Server error during request. schema: $ref: "#/definitions/Error" definitions: Error: properties: code: type: integer message: type: string

Slide 83

Slide 83 text

Lesson 1.06: Responses Default Response /pets/{pet-id}/ delete: responses: ... default: $ref: '#/responses/UnknownResponse' responses: UnknownResponse: description: This response is not yet documented by this API.

Slide 84

Slide 84 text

Lesson 1.06: Responses Exercise Instructions Add pagination to the /talks and /speakers paths. Pagination should be included by at least two parameters: page-size , page-number . Add the following responses to all paths: 400, 500, default.

Slide 85

Slide 85 text

Solution 1.06 Notes Functionality complete API

Slide 86

Slide 86 text

Lesson 1.07: Secure Your APIs Goals Learn the different security schemas supported. Global vs. local security via file upload definition example.

Slide 87

Slide 87 text

Lesson 1.07: Secure Your APIs Basic Auth securityDefinitions: type: basic

Slide 88

Slide 88 text

Lesson 1.07: Secure Your APIs API Key securityDefinitions: "type": "apiKey", "name": "api_key", "in": "header"

Slide 89

Slide 89 text

Lesson 1.07: Secure Your APIs OAuth2 securityDefinitions: OauthSecurity: type: oauth2 flow: accessCode authorizationUrl: 'https://oauth.swagger.io.com/authorization' tokenUrl: 'https://oauth.swagger.io/token' scopes: admin: Admin scope user: User scope security: - OauthSecurity: - user

Slide 90

Slide 90 text

Lesson 1.07: Secure Your APIs File Upload paths: /pets/{pet-id}/picture: parameters: - $ref: '#/parameters/pet-id' post: description: Admin operation to upload a pet picture operationId: UploadPicture security: - OauthSecurity: - admin consumes: - multipart/form-data parameters: - name: picture in: formData

Slide 91

Slide 91 text

Lesson 1.07: Secure Your APIs Exercise Instructions Define a security scheme for your API. Use Oauth2. Add a new path to be able to upload speaker resume and secure it using admin role.

Slide 92

Slide 92 text

Solution 1.07 Notes Security representation in the editor Header in responses

Slide 93

Slide 93 text

Lesson 1.08: Doc the Docs Goals Learn additional points on spec documentation

Slide 94

Slide 94 text

Lesson 1.08: Doc the Docs OperationId /pets: get: operationId: GetPets

Slide 95

Slide 95 text

Description GFM /pets: get: description: ## Retrieve multiple pet objects. For example: - pet1 - pet2

Slide 96

Slide 96 text

Lesson 1.08: Doc the Docs Tags paths: /pets: get: tags: - pet ... tags: name: pet description: Pet operations

Slide 97

Slide 97 text

Lesson 1.08: Doc the Docs Exercise Instructions Update for API with more information on the operations description, using GFM. Add tags and operationIds to all your operations

Slide 98

Slide 98 text

Solution 1.08 Notes Tags in the editor

Slide 99

Slide 99 text

Lesson 1.09: Can We Split This? Goals Learn how to support not having all the API in one flat file

Slide 100

Slide 100 text

Lesson 1.09: Can We Split This? Reference External Files /pets: get: summary: Get a list of pets description: Retrieve a list of pets operationId: GetPets parameters: - $ref: 'parameters.yaml#/page-size' - $ref: 'parameters.yaml#/page-number'

Slide 101

Slide 101 text

Lesson 1.09: Can We Split This? parameters.yaml Parameters: page-size: name: page-size in: query description: Number of items type: integer format: int32 minimum: 1 maximum: 100 multipleOf: 10 default: 10

Slide 102

Slide 102 text

Lesson 1.09: Can We Split This? Serving External Files

Slide 103

Slide 103 text

Lesson 1.09: Can We Split This? Exercise Instructions Split your API spec. The proposed scheme is to have separate file for definitions, parameters and responses. You can consider other split strategies.

Slide 104

Slide 104 text

Solution 1.09 Notes A better-organized specification

Slide 105

Slide 105 text

Part 1 Recap 1.01: Setup 1.02: Hello, World! 1.03: Pets 1.04: Registration 1.05: The Minimalist API 1.06: Responses 1.07: Secure Your APIs 1.08: Doc the Docs 1.09: Can We Split This?

Slide 106

Slide 106 text

Break

Slide 107

Slide 107 text

Contract-first API development using the OpenAPI Specification (fka Swagger) Part 2

Slide 108

Slide 108 text

What do we get?

Slide 109

Slide 109 text

What do we get?

Slide 110

Slide 110 text

Benefits

Slide 111

Slide 111 text

Benefits Documentation

Slide 112

Slide 112 text

Benefits Documentation Mocking

Slide 113

Slide 113 text

Benefits Documentation Mocking Testing

Slide 114

Slide 114 text

Benefits Documentation Mocking Testing Code Generation

Slide 115

Slide 115 text

Benefits Documentation Mocking Testing Code Generation

Slide 116

Slide 116 text

Code Generation

Slide 117

Slide 117 text

Code Generators

Slide 118

Slide 118 text

Code Generators Servers

Slide 119

Slide 119 text

Code Generators Servers Clients

Slide 120

Slide 120 text

Code Generators Servers Clients Documentation

Slide 121

Slide 121 text

Swagger-Codegen

Slide 122

Slide 122 text

Swagger-Codegen Via Swagger editor Calls to https://generator.swagger.io/

Slide 123

Slide 123 text

Swagger-Codegen Via Swagger editor Calls to https://generator.swagger.io/ Via CLI http://swagger.io/swagger-codegen/

Slide 124

Slide 124 text

Integrated Frameworks

Slide 125

Slide 125 text

Integrated Frameworks Swagger Inflector (Java)

Slide 126

Slide 126 text

Integrated Frameworks Swagger Inflector (Java) swagger-node (Node.js)

Slide 127

Slide 127 text

Integrated Frameworks Swagger Inflector (Java) swagger-node (Node.js) Connexion (Python)

Slide 128

Slide 128 text

Lesson 2.01: Code Generation Goals Server/Client Code from Spec

Slide 129

Slide 129 text

Generate Server

Slide 130

Slide 130 text

Generate Client

Slide 131

Slide 131 text

Lesson 2.01: Code Generation Exercise Instructions Generate server & client side code with your favorite option provided by the code generator. (bonus) Update server side code so that the /talks and /speakers paths return empty list on GET. Use the methods provided by the client code in order test the responses from the server.

Slide 132

Slide 132 text

Solution 2.01 Notes Experimented with code generated

Slide 133

Slide 133 text

Connexion

Slide 134

Slide 134 text

Connexion

Slide 135

Slide 135 text

Connexion Python + Flask

Slide 136

Slide 136 text

Connexion Python + Flask Spec As Configuration

Slide 137

Slide 137 text

Connexion Python + Flask Spec As Configuration Routing, Validation, etc.

Slide 138

Slide 138 text

Connexion Python + Flask Spec As Configuration Routing, Validation, etc.

Slide 139

Slide 139 text

Explicit Routing

Slide 140

Slide 140 text

Explicit Routing Explicit Function Name paths: /hello_world: post: operationId: myapp.api.hello_world

Slide 141

Slide 141 text

Explicit Routing Explicit Function Name paths: /hello_world: post: operationId: myapp.api.hello_world Separate Controller Name paths: /hello_world: post: x-swagger-router-controller: myapp.api operationId: hello_world

Slide 142

Slide 142 text

Automatic Routing

Slide 143

Slide 143 text

Automatic Routing from connexion.resolver import RestyResolver app = connexion.FlaskApp(__name__) app.add_api('swagger.yaml', resolver=RestyResolver('api'))

Slide 144

Slide 144 text

Automatic Route Resolution paths: /: get: # Implied operationId: api.get /foo: get: # Implied operationId: api.foo.search post: # Implied operationId: api.foo.post '/foo/{id}': get: # Implied operationId: api.foo.get put: # Implied operationId: api.foo.put copy: # Implied operationId: api.foo.copy delete: # Implied operationId: api.foo.delete

Slide 145

Slide 145 text

Request Validation

Slide 146

Slide 146 text

Request Validation JSON Schema

Slide 147

Slide 147 text

Request Validation JSON Schema Required parameters

Slide 148

Slide 148 text

Request Validation JSON Schema Required parameters Types and Formats

Slide 149

Slide 149 text

Request Validation JSON Schema Required parameters Types and Formats Custom Validators

Slide 150

Slide 150 text

Request Validation JSON Schema Required parameters Types and Formats Custom Validators HTTP 400 w/ Details

Slide 151

Slide 151 text

Response Handling

Slide 152

Slide 152 text

Response Handling Serialization

Slide 153

Slide 153 text

Response Handling Serialization JSON Encoder

Slide 154

Slide 154 text

Response Handling Serialization JSON Encoder Validation Optional

Slide 155

Slide 155 text

Response Handling Serialization JSON Encoder Validation Optional Custom Validators

Slide 156

Slide 156 text

Security

Slide 157

Slide 157 text

Security OAuth 2 via Spec

Slide 158

Slide 158 text

Security OAuth 2 via Spec DIY API Key Basic Auth

Slide 159

Slide 159 text

Other Features

Slide 160

Slide 160 text

Other Features Swagger UI

Slide 161

Slide 161 text

Other Features Swagger UI Swagger JSON

Slide 162

Slide 162 text

Other Features Swagger UI Swagger JSON Flask Integration

Slide 163

Slide 163 text

Other Features Swagger UI Swagger JSON Flask Integration

Slide 164

Slide 164 text

Lesson 2.02: Run the API Goals Python/Flask Connexion

Slide 165

Slide 165 text

Connexion Implementation

Slide 166

Slide 166 text

Lesson 2.02: Run the API Exercise Instructions Run the the betterapis application using the connexion implementation Activate virtualenv: workon tutorial Run app: python -m betterapis Register two speakers and submit a talk for each one. Use HTTP POSTs via Postman, curl, et al. Request speaker list to verify data persisted.

Slide 167

Slide 167 text

Solution 2.02 Notes Populated data

Slide 168

Slide 168 text

Lesson 2.03: Mock Server Goals Run mock server for client to experiment with the API

Slide 169

Slide 169 text

Lesson 2.03: Mock Server examples responses: 200: description: Returns a specific talk schema: $ref: '#/definitions/Pet' examples: application/json: { id: 12345, name: "pythagoras", status: "Adopted" }

Slide 170

Slide 170 text

Lesson 2.03: Mock Server Usage connexion run betterapis.yaml --mock=all -v

Slide 171

Slide 171 text

Lesson 2.03: Mock Server Exercise Instructions Update responses to have examples Run and test your mock server

Slide 172

Slide 172 text

Solution 2.03 Notes Can mock with any spec

Slide 173

Slide 173 text

Lesson 2.04: Test with Dredd Goals Learn how the spec connects tests and implementation

Slide 174

Slide 174 text

Lesson 2.04: Test with Dredd Installation Using npm Provided in the VM

Slide 175

Slide 175 text

Lesson 2.04: Test with Dredd Usage Dredd init ? Location of the API description document tutorial-repo/implementation/betterapis/specs/betterapis.yaml ? Command to start API backend server e.g. (bundle exec rails server) ? URL of tested API endpoint http://127.0.0.1:8080 ? Programming language of hooks python ? Do you want to use Apiary test inspector? No ? Dredd is best served with Continuous Integration. Create CircleCI config for Dredd? No Configuration saved to dredd.yml Run test now, with: $ dredd

Slide 176

Slide 176 text

Lesson 2.04: Test with Dredd How Dredd works In request: x-example or default In response: format matching

Slide 177

Slide 177 text

Lesson 2.04: Test with Dredd Parameter Example pet-id: name: pet-id in: path description: Pet identifier type: number required: true x-example: 42

Slide 178

Slide 178 text

Lesson 2.04: Test with Dredd Exercise Instructions Update all the GET actions so that they can be tested using Dredd. Initialize dredd and run dredd --method GET command in order to verify that tests are passing. (Use ids from data you initialized in the application in previous lesson)

Slide 179

Slide 179 text

Solution 2.04 Notes 4 tests passed and 6 skipped.

Slide 180

Slide 180 text

Lesson 2.05: New Features Goals Development cycle for new feature

Slide 181

Slide 181 text

Lesson 2.05: New Features Flow Update the spec Run tests -> Fail Update the code Run tests -> Success

Slide 182

Slide 182 text

New Feature Demo

Slide 183

Slide 183 text

Lesson 2.05: New Features Exercise Instructions Add a new feature to the application to support reviews. Besides a unique id the review object should reference the talk_id it refers to and details .

Slide 184

Slide 184 text

Solution 2.05 Notes Connecting it all together

Slide 185

Slide 185 text

Lesson 2.06: Documentation Goals Generating automatic documentation for clients

Slide 186

Slide 186 text

Lesson 2.06: Documentation Tooling Connexion: HTML, Console swagger-ui Many others

Slide 187

Slide 187 text

Connexion Docs Demo

Slide 188

Slide 188 text

Lesson 2.06: Documentation Exercise Instructions Register a new speaker and submit a talk using the connexion UI. Use the UI to also update and delete the talk and the speaker.

Slide 189

Slide 189 text

Solution 2.06 Notes Easy to distribute documentation

Slide 190

Slide 190 text

Part 2 Recap 2.01: Code Generation 2.02: Run the API 2.03: Mock Server 2.04: Test with Dredd 2.05: New Features 2.06: Documentation

Slide 191

Slide 191 text

More Resources BetterAPIs.com

Slide 192

Slide 192 text

Thank You!

Slide 193

Slide 193 text

Ian ian.zelikman@gmail.com @izcoder Thank You! Questions? Dave dave@forgac.com @tylerdave