Scalable REST API in Node.js

Transcript

1. Scalable REST API in Node.js
2. Who am I - Hi, I’m Mehdi Hasan Khan ! - Currently Software Architect @ ShopUp 
 We’re a JavaScript shop and we are hiring - Writing JavaScript professionally for 9 years - Developer of Avro Keyboard - Dad of a wonderful kid
 
 Twitter: @MehdiHK GitHub: https://github.com/mugli LinkedIn: https://www.linkedin.com/in/mehdihk/
3. What I DON’T mean when I talk about scaling
4. Lies, Damned Lies and Benchmarks
5. In real world your database and other backend services are probably the bottleneck. Not the HTTP Framework.
7. Non-specific scalability promise is a dangerous myth
8. What do I mean when I talk about scaling
9. Growth Team size Code Resource requirement
10. Common API Life Cycle • Build • Design • Develop • Document • Test • Use • Access Control • Data Access • Run • Scale with traffic • Availability • Monitoring
11. This talk focuses on the 1st step
12. Build
13. Design Develop Document Test
14. Design & Document Build
15. Why design before develop?
16. Most badly designed REST APIs aren't bad because they were developed by bad coders. It's because they were not designed in the first place, they were just developed.
17. JUST WRITE IT. require('express')
18. As the flexibility of a system increases, its usability decreases Flexibility-Usability Tradeoff Universal Principles of Design
19. Pragmatic REST 101 https://www.slideshare.net/apigee/rest-design-webinar
20. https://pages.apigee.com/rs/apigee/images/api-design-ebook-2012-03.pdf
23. • Include .json extension or not in endpoints? • CamelCase or snake _case? • Version in the url or header or param? • Verbs are bad? How do I make search endpoints? • Should GET requests have body? • Do you even paginate bro? Cursor or offset? • Formatted output or minified? • How do I avoid over-fetching?
24. If you read enough advice, everything will cancel each other out
25. (╯°□°)╯︵ ┻━┻
26. What everyone is asking for is Consistency
27. Consistency reduces cognitive load
28. “The competent programmer is fully aware of the limited size of his own skull” Edsger Dijkstra
29. Think of API as UI for other devs
31. Think of API as a contract
32. Tell me exactly what’s accepted
33. Tell me exactly what I can expect
34. Promise me you won’t change them without notice
35. The case for schema first API
36. Don’t make me think (╯°□°)╯︵ ┻━┻
37. Tech Debt That Hurts
38. Don’t make me think (╯°□°)╯︵ ┻━┻
39. Better: hapi and joi
40. Better: hapi and joi • Correct status code • Auto-generated error message • Consistent error object • Declarative syntax, no ugly regex hacks, type checks • What’s expected is immediately visible • Makes refactoring easy for other API developers • Can be used to check response object too before they are send to the client! 
 (Not shown in the example)
41. Constraints set us free
42. Bonus: Zero effort living documentation!
43. Documentation by hand gets outdated, API should be the source of truth
44. Living documentation
45. For schema definition, configuration is better than code
46. Using Joi with Express npm install celebrate
47. Beyond Joi
48. https://json-schema.org/
49. Beyond Joi and JavaScript Ecosystem
51. Fastify with JSON Schema
52. Same benefit: Zero effort living documentation
53. ..and, wait for it…
54. You can generate code from the spec too!
55. Reverse: You can generate code from the spec too!
58. Can’t we have both for API?
59. Enter OpenAPI
60. The OpenAPI Specification defines a standard, programming language-agnostic interface description for REST APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic.
62. Sample Spec
63. Quick start: Express & OpenAPI 3 https://developer.ibm.com/recipes/tutorials/builds-apis-with-node-js-express-and- openapi-3/ npm i express-openapi-validator
64. https://openapi.tools/ Benefit of having spec that targets both humans and machines
66. Nope
67. Don’t get overwhelmed by tools
68. Think from the first principles
69. Think of API as UI for other devs
70. Think of API as a contract
71. And then
72. Develop Build
73. Express: Order based routing
74. ⚠ Be aware of route orders
75. It gets worse with middleware ordering
77. Compare that to hapi
78. • hapi searches paths in order from most specific to least specific. • If you have two routes, /filename.jpg and /filename.{ext} a request to /filename.jpg will match the first route, and not the second, whatever their order is in code. • That also mean, a route with the path /{files*} will be the last route tested, and will only match if all other routes fail.
79. • hapi has deterministic routing. Each request can only map to one route, and its routing table will be the same every time you start the server. • As the application size and teams grow, routing conflicts become more of a concern. You want a banana but you get a gorilla holding the banana and the entire jungle. • If you have two routes that conflict, hapi will show an error on startup, providing details on the routes that conflict, making it much easier to debug and fix. This is much better than spending hours debugging this at runtime.
80. Caching
81. hapi provides powerful caching capabilities
82. Test Build
84. Express and supertest
85. You want to test the route • You don’t need to spawn the whole server • You don’t need to listen to a socket • You don’t even need to send a real http call and process it • Because you are not testing if the underlying http stack is working in Node.js, you are testing if you pass some values to your endpoint, if it returns correct result
86. Both fastify and hapi support http injection
87. fastify.inject
88. server.inject in hapi
89. Final thoughts • Express is minimal and optimized for quick learning and prototyping. Unfortunately it became the most popular framework in Node.js ecosystem for the same reason. There are better options though. • Hapi brought a lot of battle tested and production ready practices to Node.js • Fastify had the luxury of coming later and learnt from both of them. • If you are stuck with existing express based api, there are middlewares that tries to make it better.
90. Thank you and keep building awesome stuff!