Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Zend Expressive - An Introduction For API Building

Zend Expressive - An Introduction For API Building

Whether it's micro-service architectures, single-page web apps in React or Angular, or simply providing data-and-functionality-as-a-service, API development is firmly a thing today. There's plenty of advice out there about designing APIs - but how do you turn it into working, shippable code?

In this talk, Stuart will introduce you to his framework of choice for API development - Zend Expressive. He'll explain why he's gone with it, how to get started, and how to organise your code to get the best out of it. He'll also cover the essential elements that are missing (authentication, logging, metrics, validation, error handling, rate limiting), and finish by looking at how to go about testing your finished API.

Presented at Aberdeen PHP, 1st August, 2018.

2c1dc90ff7bf69097a151677624777d2?s=128

Stuart Herbert

August 01, 2018
Tweet

Transcript

  1. A presentation by @stuherbert
 for @GanbaroDigital Zend Expressive An Introduction

    For API Building
  2. Industry veteran: architect, engineer, leader, manager, mentor F/OSS contributor since

    1994 Talking and writing about PHP since 2004 Chief Software Archaeologist Building Quality @GanbaroDigital About Stuart
  3. Follow me I do tweet a lot about non-tech stuff

    though :) @stuherbert
  4. @GanbaroDigital ?? ?? What framework(s) do you currently use for

    building APIs?
  5. @GanbaroDigital Zend Expressive is my framework of choice for building

    JSON API services.
  6. @GanbaroDigital I'm here to show you how I currently use

    Zend Expressive for building API services.
  7. @GanbaroDigital This is just how I go about it. I'm

    here to learn from you too.
  8. @GanbaroDigital In This Talk 1. Why Zend Expressive 2. Getting

    Started 3. Organising Your Code 4. Essential Elements 5. Testing Your API
  9. @GanbaroDigital I won't be comparing Zend Expressive against other frameworks.

  10. @GanbaroDigital Why Zend Expressive?

  11. @GanbaroDigital ?? ?? How can a framework help you build

    your API?
  12. @GanbaroDigital https://flic.kr/p/uTwoPu

  13. @GanbaroDigital 1. Framework Isolation

  14. @GanbaroDigital To explain what this is, let's look at what

    happens when your API gets called.
  15. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  16. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  17. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  18. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  19. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  20. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  21. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  22. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  23. @GanbaroDigital Bootstrap Routing Dispatch Response Controller Features

  24. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Controller Features

  25. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Controller Features

  26. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features
  27. @GanbaroDigital I want to be able to upgrade / replace

    the framework without touching anything else.
  28. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features
  29. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features
  30. @GanbaroDigital Bootstrap Routing Dispatch Response Framework v.Next Glue Code Business

    Logic Controller Features
  31. @GanbaroDigital “Framework upgrades are very expensive. By isolating the framework,

    I can keep that cost under control.
  32. @GanbaroDigital This means that the "glue code" can't depend on

    the framework.
  33. @GanbaroDigital Neither can anything in the business model or data

    model.
  34. @GanbaroDigital This has other benefits.

  35. @GanbaroDigital 2. Glue Re-use

  36. @GanbaroDigital “ The more time I can spend on features,

    the more productive I am.
  37. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features
  38. @GanbaroDigital PSR-7 / PSR-15 give me middleware and framework isolation.

  39. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features
  40. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  41. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  42. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  43. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  44. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  45. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  46. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  47. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  48. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  49. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  50. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  51. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  52. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  53. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  54. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  55. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  56. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  57. @GanbaroDigital Each piece of middleware is a standalone piece of

    logic.
  58. @GanbaroDigital It knows nothing about any other middleware. It knows

    nothing about the controller / action at the end of the pipeline.
  59. @GanbaroDigital 3. Change Isolation

  60. @GanbaroDigital “ The #1 cause of new defects in working

    code is change.
  61. @GanbaroDigital By isolating code, I can isolate the impact of

    code changes.
  62. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  63. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  64. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  65. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  66. @GanbaroDigital I would rather repeat isolated code than have brittle

    DRY code.
  67. @GanbaroDigital “ Today's repeated code isn't always tomorrow's repeated code.

  68. @GanbaroDigital 4. Ability To Reason

  69. @GanbaroDigital PHP's core strength is just how damned deterministic its

    execution model is.
  70. @GanbaroDigital Anything that obfuscates it, or introduces non-determinism, increases your

    defect count.
  71. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  72. @GanbaroDigital Everything is traceable.

  73. @GanbaroDigital Everything is testable on its own.

  74. @GanbaroDigital Motivations Recap 1. Framework Isolation 2. Glue Re-use 3.

    Change Isolation 4. Ability To Reason
  75. @GanbaroDigital Zend Expressive satisfies all my motivations.

  76. @GanbaroDigital Your motivations may be different. Nothing wrong with that!

  77. @GanbaroDigital Getting Started

  78. @GanbaroDigital https://unsplash.com/photos/RFId0_7kep4 Guided Installation

  79. @GanbaroDigital composer create-project
 zendframework/zend-expressive-skeleton 
 <your-app-name>

  80. @GanbaroDigital Here's what I use when building JSON APIs.

  81. @GanbaroDigital

  82. @GanbaroDigital The modular structure helps me achieve code isolation.

  83. @GanbaroDigital

  84. @GanbaroDigital I haven't felt the need to try any of

    the others.
  85. @GanbaroDigital Coding against PSR-11 means there's no obvious benefit switching

    between modern DI containers.
  86. @GanbaroDigital

  87. @GanbaroDigital My Zend Expressive API apps spend around 20% in

    the 'framework' column. Reducing that overhead is important to me.
  88. @GanbaroDigital

  89. @GanbaroDigital I don't use any templating at all (yet) in

    my API apps.
  90. @GanbaroDigital I'm interested in creating self-documenting APIs one day. I

    will use Twig when that happens.
  91. @GanbaroDigital

  92. @GanbaroDigital We'll come back to error handling shortly.

  93. @GanbaroDigital Organising Your Code

  94. @GanbaroDigital Zend Expressive uses a different layout than (say) Wordpress.

  95. @GanbaroDigital Folder Structure Created By Installer

  96. @GanbaroDigital config/ Config files - yours & the framework's Folder

    Structure Created By Installer
  97. @GanbaroDigital config/ data/ Config files - yours & the framework's

    Config cache home Folder Structure Created By Installer
  98. @GanbaroDigital config/ data/ public/ Config files - yours & the

    framework's Config cache home Static assets (largely unused in APIs) Folder Structure Created By Installer
  99. @GanbaroDigital In something like Wordpress, everything lives in the 'public'

    folder AKA the DocRoot
  100. @GanbaroDigital config/ data/ public/ src/ Config files - yours &

    the framework's Config cache home Static assets (largely unused in APIs) Your glue code and business logic Folder Structure Created By Installer
  101. @GanbaroDigital config/ data/ public/ src/ test/ Config files - yours

    & the framework's Config cache home Static assets (largely unused in APIs) Your glue code and business logic Unit tests for what's in src/ Folder Structure Created By Installer
  102. @GanbaroDigital config/ data/ public/ src/ test/ vendor/ Config files -

    yours & the framework's Config cache home Static assets (largely unused in APIs) Your glue code and business logic Unit tests for what's in src/ Packages installed via composer Folder Structure Created By Installer
  103. @GanbaroDigital Zend Expressive uses PSR-4 autoloading to load each module

    in your app.
  104. @GanbaroDigital config/ data/ public/ src/ test/ vendor/ Config files -

    yours & the framework's Config cache home Static assets (largely unused in APIs) Your glue code and business logic Unit tests for what's in src/ Packages installed via composer Folder Structure Created By Installer
  105. @GanbaroDigital config/ data/ public/ src/ test/ vendor/ Config files -

    yours & the framework's Config cache home Static assets (largely unused in APIs) Your glue code and business logic Unit tests for what's in src/ Packages installed via composer Folder Structure Created By Installer
  106. @GanbaroDigital src/ Folder Structure Created By Installer

  107. @GanbaroDigital src/ App/ Folder Structure Created By Installer Your module

  108. @GanbaroDigital src/ App/ src/ PSR-4 Namespace starts here Folder Structure

    Created By Installer Your module
  109. @GanbaroDigital { ..., "autoload": "psr-4": { "App\\": "src/App/src" } },

    ... }
  110. @GanbaroDigital src/ App/ src/ <code> PSR-4 Namespace starts here Folder

    Structure Created By Installer Your namespaced code Your module
  111. @GanbaroDigital src/ App/ src/ <code> ConfigProvider.php PSR-4 Namespace starts here

    Folder Structure Created By Installer Your namespaced code DI container / template config Your module
  112. @GanbaroDigital src/ App/ src/ <code> ConfigProvider.php templates/ PSR-4 Namespace starts

    here Folder Structure Created By Installer Your namespaced code DI container / template config Your module Your views
  113. @GanbaroDigital First thing I do is replace 'App' with a

    sensible namespace, e.g. GanbaroDigital\BillingAPI\Core
  114. @GanbaroDigital src/ App/ Modular Folder Structure

  115. @GanbaroDigital src/ App/ Modular Folder Structure

  116. @GanbaroDigital src/ Core/ Modular Folder Structure Dumping ground

  117. @GanbaroDigital Then I add modules to provide functionality to business

    logic.
  118. @GanbaroDigital src/ Core/ Storage/ Health/ Datastore abstraction Modular Folder Structure

    Health check API Dumping ground
  119. @GanbaroDigital And modules for individual business logic areas.

  120. @GanbaroDigital src/ Core/ Storage/ Health/ Invoices/ Payments/ Datastore abstraction Modular

    Folder Structure Health check API Business logic Dumping ground More business logic
  121. @GanbaroDigital I prefer to have each API path route to

    its own module. e.g. /u/ganbarodigital/invoices /u/ganbarodigital/payments
  122. @GanbaroDigital One of the really nice things about Zend Expressive

    is that you can have whatever structure suits you.
  123. @GanbaroDigital Essential Elements

  124. @GanbaroDigital Zend Expressive provides a great platform for building API

    services.
  125. @GanbaroDigital Out of the box, it isn't a 100% ready-to-go

    API framework.
  126. @GanbaroDigital Most of what we need can be handled by

    building some middleware.
  127. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  128. @GanbaroDigital A couple of things need adding as services built

    by the DI container.
  129. @GanbaroDigital Logging

  130. @GanbaroDigital https://speakerdeck.com/stuartherbert/designing-for- disaster-preparing-your-code-for-emergencies

  131. @GanbaroDigital Logging Needs ... 1. A logger in the DI

    container 2. Middleware to log all requests & responses 3. Middleware to add extra metadata
  132. @GanbaroDigital Use: Psr\Log\LoggerInterface as the request key into the DI

    container.
  133. @GanbaroDigital Have the DI container return a Monolog instance. You

    need Monolog's support for customising the logger after it has been created.
  134. @GanbaroDigital i.e. an 'immutable' logger is about as much use

    as a chocolate fireguard here!
  135. @GanbaroDigital Gradually add metadata fields to help you find relevant

    logs. Think of them like database indexes.
  136. @GanbaroDigital Add a 'uid' metadata: unique ID for that API

    request
  137. @GanbaroDigital Logging Needs ... 1. A logger in the DI

    container 2. Middleware to log all requests & responses 3. Middleware to add extra metadata
  138. @GanbaroDigital LogRequestsAndResponses: middleware to add initial metadata to the logger

    & capture inputs and outputs
  139. @GanbaroDigital Add a 'request-id' metadata: trace API calls across your

    architecture.
  140. @GanbaroDigital Logging Needs ... 1. A logger in the DI

    container 2. Middleware to log all requests & responses 3. Middleware to add extra metadata
  141. @GanbaroDigital In the Authentication middleware, add a 'user-id' to the

    logger's metadata.
  142. @GanbaroDigital Error Handling

  143. @GanbaroDigital Error Handling Needs ... 1. Common code for RFC

    7807 responses 2. Common exception for error responses 3. Middleware to handle the common exception 4. Middleware for 404 Not Found handling
  144. @GanbaroDigital RFC 7807 defines a common format for API responses

    when an error has occurred.
  145. @GanbaroDigital Error Handling Needs ... 1. Common code for RFC

    7807 responses 2. Common exception for error responses 3. Middleware to handle the common exception 4. Middleware for 404 Not Found handling
  146. @GanbaroDigital I have my business logic throw an 'ApiError' exception.

    The business logic creates the appropriate Response and stuffs it inside the exception.
  147. @GanbaroDigital https://speakerdeck.com/stuartherbert/
 5-patterns-to-improve-application-robustness

  148. @GanbaroDigital Error Handling Needs ... 1. Common code for RFC

    7807 responses 2. Common exception for error responses 3. Middleware to handle the common exception 4. Middleware for 404 Not Found handling
  149. @GanbaroDigital To catch (and log!) ApiError exceptions, I add a

    'HandleApiErrors' middleware to my app's pipeline.
  150. @GanbaroDigital Error Handling Needs ... 1. Common code for RFC

    7807 responses 2. Common exception for error responses 3. Middleware to handle the common exception 4. Middleware for 404 Not Found handling
  151. @GanbaroDigital I add a 'NotFoundHandler' to generate RFC 7807 responses

    for HTTP 404 errors.
  152. @GanbaroDigital Validation

  153. @GanbaroDigital Validation Needs ... 1. application/vnd+ Content-Type 2. JSON schema

    for input payloads 3. Middleware to validate against JSON schema 4. (optional) JSON schema for output payloads
  154. @GanbaroDigital I don't use application/json as my API Content-Type.

  155. @GanbaroDigital application/vnd+billingapi.invoice.v1

  156. @GanbaroDigital Validation Needs ... 1. application/vnd+ Content-Type 2. JSON schema

    for input payloads 3. Middleware to validate against JSON schema 4. (optional) JSON schema for output payloads
  157. @GanbaroDigital https://speakerdeck.com/stuartherbert/
 json-schema-for-validating-api-requests

  158. @GanbaroDigital Validation Needs ... 1. application/vnd+ Content-Type 2. JSON schema

    for input payloads 3. Middleware to validate against JSON schema 4. (optional) JSON schema for output payloads
  159. @GanbaroDigital Add middleware to look at the incoming Content-Type and

    validate the payload against the matching JSON schema.
  160. @GanbaroDigital Validation Needs ... 1. application/vnd+ Content-Type 2. JSON schema

    for input payloads 3. Middleware to validate against JSON schema 4. (optional) JSON schema for output payloads
  161. @GanbaroDigital The same middleware can also validate all of your

    API responses.
  162. @GanbaroDigital Authentication

  163. @GanbaroDigital Authentication Needs ... 1. An identity provider 2. Middleware

    to handle CORS requests 3. Middleware to authenticate a user 4. Middleware to authorise a user
  164. @GanbaroDigital An identity provider is a user database + UI

    all in one.
  165. @GanbaroDigital A commercial one like auth0.com might be easier than

    building your own.
  166. @GanbaroDigital Authentication Needs ... 1. An identity provider 2. Middleware

    to handle CORS requests 3. Middleware to authenticate a user 4. Middleware to authorise a user
  167. @GanbaroDigital The CORS spec mandates that some CORS-requests are always

    unauthenticated.
  168. @GanbaroDigital If your API can be called directly from JavaScript

    running in browsers, you must handle authentication inside your API app.
  169. @GanbaroDigital The world is moving to JavaScript consuming APIs. And

    better data privacy laws. Add support upfront.
  170. @GanbaroDigital Authentication Needs ... 1. An identity provider 2. Middleware

    to handle CORS requests 3. Middleware to authenticate a user 4. Middleware to authorise a user
  171. @GanbaroDigital Use separate middleware items for authentication and authorisation.

  172. @GanbaroDigital Authentication Needs ... 1. An identity provider 2. Middleware

    to handle CORS requests 3. Middleware to authenticate a user 4. Middleware to authorise a user
  173. @GanbaroDigital Keep authorisation simple. Use OAuth2 scopes, not RBAC.

  174. @GanbaroDigital In practice, RBAC schemes perform very poorly.

  175. @GanbaroDigital Metrics

  176. @GanbaroDigital Metrics Need ... 1. Metrics infrastructure (e.g. Prometheus) 2.

    A statsd client
  177. @GanbaroDigital https://speakerdeck.com/stuartherbert/designing-for- disaster-preparing-your-code-for-emergencies

  178. @GanbaroDigital There's no PSR for a metrics client today.

  179. @GanbaroDigital Add your preferred metrics client to the DI container.

  180. @GanbaroDigital Call the client throughout your code to report timings

    and counters.
  181. @GanbaroDigital Don't use middleware to track your API response times.

    Get these timings from your Nginx logs.
  182. @GanbaroDigital Rate Limiting

  183. @GanbaroDigital Most APIs need to protect themselves from their heaviest

    users.
  184. @GanbaroDigital A common approach is to limit the number of

    requests per user, per time period.
  185. @GanbaroDigital Rate Limiting Needs ... 1. Identity provider to provide

    user IDs 2. Redis to track API usage 3. Middleware to update API usage 4. Cron job to reset API usage 5. Auto-scaling infrastructure
  186. @GanbaroDigital Rate Limiting Needs ... 1. Identity provider to provide

    user IDs 2. Redis to track API usage 3. Middleware to update API usage 4. Cron job to reset API usage 5. Auto-scaling infrastructure
  187. @GanbaroDigital Redis gives us atomic increment / decrement.

  188. @GanbaroDigital Using an external store like Redis makes auto-scaling possible.

  189. @GanbaroDigital Rate Limiting Needs ... 1. Identity provider to provide

    user IDs 2. Redis to track API usage 3. Middleware to update API usage 4. Cron job to reset API usage 5. Auto-scaling infrastructure
  190. @GanbaroDigital Add middleware to decrement per-user rate allowance and return

    errors when the allowance is used up.
  191. @GanbaroDigital The middleware can also add headers to responses to

    tell end-users what their remaining allowance is.
  192. @GanbaroDigital Rate Limiting Needs ... 1. Identity provider to provide

    user IDs 2. Redis to track API usage 3. Middleware to update API usage 4. Cron job to reset API usage 5. Auto-scaling infrastructure
  193. @GanbaroDigital Easiest way to give the end-user their next allowance

    is from a cron job or equivalent.
  194. @GanbaroDigital Don't build cron features into your API service. Use

    the real thing.
  195. @GanbaroDigital Rate Limiting Needs ... 1. Identity provider to provide

    user IDs 2. Redis to track API usage 3. Middleware to update API usage 4. Cron job to reset API usage 5. Auto-scaling infrastructure
  196. @GanbaroDigital In practice, per-user rate limits don't protect you when

    everyone wants to use the API at the same time.
  197. @GanbaroDigital Auto-scaling infrastructure is your friend there.

  198. @GanbaroDigital Testing APIs

  199. @GanbaroDigital Bootstrap Routing Dispatch Response Framework Glue Code Business Logic

    Controller Features Middleware Middleware Middleware Features
  200. @GanbaroDigital Write unit tests for individual components.

  201. @GanbaroDigital Write end-to-end tests for the API as a whole.

  202. @GanbaroDigital https://getpostman.com

  203. Thank You Can We Help You? A presentation by @stuherbert


    for @GanbaroDigital