RuhrJS 2016: The dirty little secrets of building large, highly available, scalable HTTP APIs

When you first start building an API for a new product you mostly focus on getting an MVP ready, with the goal of shipping as soon as possible so you can get feedback from customers. If you are lucky enough, your product will be successful and you will have to start worrying about things like authentication, authorization, documentation, validation, rate limiting, geo-redundancy, and no downtime deployments. In this talk I will go over some real life examples of our experience evolving our APIs at Auth0 and some of the tools we use for that.

Transcript

1. The dirty little secrets of building large, highly available, scalable HTTP APIs by @dschenkelman
2. Topic 1
3. Topic 2
4. Topic 3
5. Why? var express = require('express'); var app = express(); app.get('/', function (req, res) { res.send('Hello World!'); }); var server = app.listen(3000, function () { var host = server.address().address; var port = server.address().port; console.log('Example app listening at http://%s:%s', host, port); });
6. Developers
7. Evolution
8. Pillars
9. Validation
10. enabled: true
11. enabled: 1
12. enabled: 1.0
13. enabled: "false"
15. JSON Schemas "enabled": { "type": "boolean" }
16. Errors for developers { "statusCode": 400, "error": "Bad Request", "message": "Payload validation error: 'Expected type boolean but found type string' on property enabled (<code>true</code> if the rule is enabled, <code>false</code> otherwise).", "errorCode": "invalid_body" }
17. ratify const ZSchemaErrors = require('z-schema-errors'); const errorReporters = ['headers', 'query', 'path', 'payload'].reduce((current, part) => { current[part] = ZSchemaErrors.init({/*...*/}); return current; }, {}); plugins.push({ register: require('ratify'), options: { errorReporters: errorReporters, /*...*/ } });
18. Documentation
19. Old API Explorer
20. Auto-generate docs
21. Swagger
22. Leverage validation "enabled": { "type": "boolean", "description": "<code>true</code> if the connection is enabled, <code>false</code> (default) otherwise" }
23. Customize
24. AuthN & AuthZ
25. Token Exchange API Client API Server 1. Credentials 2. API Token 3. Requests
26. Granular Security
27. Decentralized Issuance
28. Expiration Control
29. Debuggability
30. JSON Web Token JWT
31. Authenticate For all endpoints
32. Authorize Per endpoint
33. Route { method: 'GET', path: '/api/v2/clients/{id}', config: { auth: { strategy: 'jwt', scope: ['read:clients', 'read:client_keys'] }, } }
34. Blacklisting
35. Badgers
36. Stress tests
37. Faith unsigned long income;
38. Engineers Stress
39. What to do…
40. Rate Limiting
41. Token Bucket
42. limitd #port to listen on port: 9001 #db path db: /var/limitd/database #define the bucket types buckets: customers: size: 10 per_second: 10
43. 429 X-RateLimit-Limit Maximum amount X-RateLimit-Remaining How many there are left X-RateLimit-Reset When will bucket be full again
44. patova plugins.push({ register: require('patova'), options: { event: 'onPostAuth', // when to perform the limit check type: 'tenant', // bucket address: env.LIMITD_SERVER, extractKey: function(request, reply, done){ const key = request.auth.credentials.__tenant; done(null, key); } } });
45. Geo Redundancy
46. Data Center Failure
47. Natural Disasters
48. Cloud Provider Failure
49. Our Setup
50. Our Setup App Instances MongoDB Elastic Search App Instances MongoDB Elastic Search MongoDB
51. The switch
52. Changes
53. Experiments
54. feature-change const feature_change = require('feature-change'); var options = { expected: function(cb){ mongo_search(mongo_opts, cb); }, actual: function(cb){ es_search(es_opts, cb); }, logAction: function(current_result, new_result){ // invoked when there is a difference in the results // (useful for logging) } }; feature_change(options, function(err, result){ // this is the original callback you were using for mongo // err and result always come from mongo_search });
55. Iron out differences
56. Wrap up
57. Shipping is important
58. Remember: evolve
59. Validation & Docs • http://json-schema.org • http://swagger.io • https://auth0.com/docs/api/v2 (example of branded auto-generated docs) • https://github.com/mac-/ratify
60. AuthN & AuthZ • https://tools.ietf.org/html/rfc7519 • http://jwt.io/ • https://auth0.com/blog/2014/12/02/using-json- web-tokens-as-api-keys/ • https://auth0.com/blog/2015/03/10/blacklist-json- web-token-api-keys/ • https://github.com/auth0/hapi-auth-jwt
61. Rate limiting • http://tools.ietf.org/html/rfc6585 • https://github.com/limitd • https://github.com/dschenkelman/patova
62. BaaS • http://security.stackexchange.com/a/83382 • http://www.brendangregg.com/FlameGraphs/ cpuflamegraphs.html • https://github.com/thlorenz/v8-perf/issues/4 • https://github.com/auth0/node-baas • http://docs.aws.amazon.com/AutoScaling/latest/ DeveloperGuide/US_SetUpASLBApp.html
63. Geo Redundancy • http://highscalability.com/blog/2014/12/1/auth0- architecture-running-in-multiple-cloud-providers- and-r.html • https://auth0.com/availability-trust • http://docs.mongodb.org/master/tutorial/deploy- geographically-distributed-replica-set/
64. Feature Changes • http://zachholman.com/talk/move-fast-break- nothing/ • https://auth0.com/blog/2015/10/27/feature- changes-at-auth0/ • https://github.com/dschenkelman/feature- change
65. Thanks https://github.com/dschenkelman/api-secrets-talk @dschenkelman npm i dschenkelman