NodeConfBR 2016: The dirty little secrets of building large, highly available, scalable HTTP APIs

When you first start building an API for a new product you mostly focus on getting an MVP ready, with the goal of shipping as soon as possible so you can get feedback from customers. If you are lucky enough, your product will be successful and you will have to start worrying about things like authentication, authorization, documentation, validation, rate limiting, geo-redundancy, and no downtime deployments. In this talk I will go over some real life examples of our experience evolving our APIs at Auth0 and some of the tools we use for that.

Transcript

1. The dirty little secrets of building large, highly available, scalable HTTP APIs by @dschenkelman
2. Christophers talk: Part 2
3. Why? var express = require('express'); var app = express(); app.get('/', function (req, res) { res.send('Hello World!'); }); var server = app.listen(3000, function () { var host = server.address().address; var port = server.address().port; console.log('Example app listening at http://%s:%s', host, port); });
4. Developers
5. Evolution
6. Pillars
7. Validation
8. enabled: true
9. enabled: 1
10. enabled: 1.0
11. enabled: "false"
13. JSON Schemas "enabled": { "type": "boolean" }
14. Errors for developers { "statusCode": 400, "error": "Bad Request", "message": "Payload validation error: 'Expected type boolean but found type string' on property enabled (<code>true</code> if the rule is enabled, <code>false</code> otherwise).", "errorCode": "invalid_body" }
15. ratify const ZSchemaErrors = require('z-schema-errors'); const errorReporters = ['headers', 'query', 'path', 'payload'].reduce((current, part) => { current[part] = ZSchemaErrors.init({/*...*/}); return current; }, {}); plugins.push({ register: require('ratify'), options: { errorReporters: errorReporters, /*...*/ } });
16. Documentation
17. Old API Explorer
18. Auto-generate docs
19. Swagger
20. Leverage validation "enabled": { "type": "boolean", "description": "<code>true</code> if the connection is enabled, <code>false</code> (default) otherwise" }
21. Customize
22. AuthN & AuthZ
23. Token Exchange API Client API Server 1. Credentials 2. API Token 3. Requests
24. Granular Security
25. Decentralized Issuance
26. Expiration Control
27. Debuggability
28. JSON Web Token JWT
29. Authenticate For all endpoints
30. Authorize Per endpoint
31. Route { method: 'GET', path: '/api/v2/clients/{id}', config: { auth: { strategy: 'jwt', scope: ['read:clients', 'read:client_keys'] }, } }
32. Blacklisting
33. Interval
39. 2nd half
40. Stress tests
41. Optimism unsigned long income;
42. Ops Stress
43. What to do…
44. Rate Limiting
45. Token Bucket
46. limitd #port to listen on port: 9001 #db path db: /var/limitd/database #define the bucket types buckets: customers: size: 10 per_second: 10
47. 429 X-RateLimit-Limit Maximum amount X-RateLimit-Remaining How many there are left X-RateLimit-Reset When will bucket be full again
48. patova plugins.push({ register: require('patova'), options: { event: 'onPostAuth', // when to perform the limit check type: 'tenant', // bucket address: env.LIMITD_SERVER, extractKey: function(request, reply, done){ const key = request.auth.credentials.__tenant; done(null, key); } } });
49. Geo Redundancy
50. Data Center Failure
51. Natural Disasters
52. Cloud Provider Failure
53. Our Setup
54. Our Setup App Instances MongoDB Elastic Search App Instances MongoDB Elastic Search MongoDB
55. The switch
56. Changes
57. Experiments
58. feature-change const feature_change = require('feature-change'); var options = { expected: cb => mongo_search(mongo_opts, cb), actual: cb => es_search(es_opts, cb), logAction: (current_result, new_result) => { // invoked when there is a difference in the results // (useful for logging) } }; feature_change(options, (err, result) => { // this is the original callback you were using for mongo // err and result always come from mongo_search });
59. Iron out differences
60. Wrap up
61. Shipping is important
62. Remember: evolve
63. Validation & Docs • http://json-schema.org • http://swagger.io • https://auth0.com/docs/api/v2 (example of branded auto-generated docs) • https://github.com/mac-/ratify
64. AuthN & AuthZ • https://tools.ietf.org/html/rfc7519 • http://jwt.io/ • https://auth0.com/blog/2014/12/02/using-json- web-tokens-as-api-keys/ • https://auth0.com/blog/2015/03/10/blacklist-json- web-token-api-keys/ • https://github.com/auth0/hapi-auth-jwt
65. Rate limiting • http://tools.ietf.org/html/rfc6585 • https://github.com/limitd • https://github.com/dschenkelman/patova
66. BaaS • http://security.stackexchange.com/a/83382 • http://www.brendangregg.com/FlameGraphs/ cpuflamegraphs.html • https://github.com/thlorenz/v8-perf/issues/4 • https://github.com/auth0/node-baas • http://docs.aws.amazon.com/AutoScaling/latest/ DeveloperGuide/US_SetUpASLBApp.html
67. Geo Redundancy • http://highscalability.com/blog/2014/12/1/auth0- architecture-running-in-multiple-cloud-providers- and-r.html • https://auth0.com/availability-trust • http://docs.mongodb.org/master/tutorial/deploy- geographically-distributed-replica-set/
68. Feature Changes • http://zachholman.com/talk/move-fast-break- nothing/ • https://auth0.com/blog/2015/10/27/feature- changes-at-auth0/ • https://github.com/dschenkelman/feature- change
69. Thanks https://github.com/dschenkelman/api-secrets-talk @dschenkelman npm i dschenkelman