Making your Node.js API super fast

Transcript

1. Make Your Node.js API Super Fast Tamar Twena-Stern
2. Tamar Twena-Stern twitter @sterntwena linkedin /tamarstern tamar.twena@gmail.com
3. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
4. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
5. General Tips - DB Related • Optimize you DB queries with indexes and efficient operations • Denormalize your data • Scale your DB - Read/ write replica, replica set , sharding
6. How Can I Improve My Node.js Application Related Database Work ?
7. Let’s Start With A Common Stack
8. Which Parameters We Should Look At When We Take A Performance Benchmark ? • Latency - 99% • Average requests per second • Total requests
9. Using Mongoose VS Native Mongo Driver
10. Server With Mongoose import express from 'express'; import mongoose from 'mongoose'; // Create an Express application const app = express(); // Connect to MongoDB using Mongoose mongoose.connect('mongodb://localhost/mydatabase'); const db = mongoose.connection;
11. Server With Mongoose // Define a Mongoose schema for the person object const personSchema = new mongoose.Schema({ name: String, age: Number, gender: String }); const Person = mongoose.model('Person', personSchema);
12. Server With Mongoose app.post('/persons', async (req, res) => { try { // Extract person data from the request body const { name, age, gender } = req.body; // Create a new person document using the Person model const person = new Person({ name, age, gender }); // Save the person to the database await person.save(); res.status(201).json({ message: 'Person saved successfully' }); } catch (error) { console.error('Error saving person:', error); res.status(500).json({ error: 'Error saving person' }); } });
13. Server With Native Mongo Driver import express from 'express'; import { MongoClient } from 'mongodb'; // Connection URL const url = 'mongodb://localhost:27017'; // Database Name const dbName = 'mydatabase'; const client = new MongoClient(url); try { client.connect(); } catch (err) { console.log(err); }
14. Server With Native Mongo Driver app.post('/persons', async (req, res) => { try { // Extract person data from the request body const { name, age, gender } = req.body; // Create a new person document const person = { name: name, age: age, gender: gender }; // Insert the person document const result = await collection.insertOne(person); res.status(201).json({ message: 'Person saved successfully' }); } catch (error) { console.error('Error saving person:', error); res.status(500).json({ error: 'Error saving person' }); } });
15. Results - Mongoose
16. Results - Mongo Driver
17. Performance Comparison - Mongoose VS Native Mongo Driver
18. Performance Comparison - Mongoose VS Native Mongo Driver
19. Why Mongo Driver Improved Performance Over Mongoose By 30% ?
20. Work With DB Connection Pool
21. DB Connection Pool
22. Compare Results When Working With A Connection Pool
23. Results - No Connection Pool
24. Results - With Connection Pool
25. Performance Comparison - Optimizing The Connection Pool
26. Performance Comparison - Connection Pool vs No Connection Pool
27. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
28. Node.js Architecture
29. Use Efficient Serialization - It Is A Synchronous Operation That Blocks The Event Loop
30. Efficient Response Serialization with fast-json-stringify
31. Default Express Serializer And Get API // Middleware to parse JSON request body app.use(express.json()); app.get('/persons', async (req, res) => { try { const gender = req.query.gender; const person = await collection.findOne({ gender: gender }, { limit: 1 }); if (person) { res.json(person); } else { res.status(404).json({ message: 'Person not found' }); } } catch (err) { console.error(err); res.status(500).json({ message: 'Internal server error' }); } });
32. fast-json-serializer Middleware import fastJson from 'fast-json-stringify'; const stringify = fastJson({ title: 'Person', type: 'object', properties: { name: { type: 'string' }, gender: { type: 'string' }, age: { description: 'Age in years', type: 'integer' } } })
33. fast-json-serializer Middleware // Custom JSON serialization middleware const customJsonMiddleware = (req, res, next) => { // Override the res.json() method with custom serialization logic res.json = function(data) { const jsonData = stringify(data); // Custom serialization logic res.setHeader('Content-Type', 'application/json'); res.end(jsonData); }; // Call the next middleware or route handler next(); }; // Use the custom JSON serialization middleware for all routes app.use(customJsonMiddleware);
34. Performance Comparison - express.json vs fast-json-stringify
35. Performance Comparison - express.json vs fast-json-stringify
36. Fast-json-stringify will give the performance advantage You can look for other libraries that will give you the performance edge
37. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
38. Plan : Build perfect systems that will be always stable Reality : Applications crash, random bugs, missing data in production
39. Logging Will Cause Performance Degradation
40. Results - Without Logs
41. Results - With One Log Line
42. Different Log Libraries // Winston import winston from 'winston'; const logger = winston.createLogger({ level: 'info', format: winston.format.json(), transports: [new winston.transports.Console()], }); logger.info("Person object saved successfully"); // pinojs import pino from 'pino'; const logger = pino({}); logger.info("Person object saved successfully");
43. Different Log Libraries // bunyun import bunyan from 'bunyan'; const logger = bunyan.createLogger({ name: 'myapp' }); logger.info("Person object saved successfully"); // log4js import log4js from'log4js'; const logger = log4js.getLogger(); logger.level = 'info'; logger.info("Person object saved successfully");
44. Node.js Log Libraries - Performance Comparison
45. Node.js Log Libraries - Performance Comparison
46. Conclusion - Performance Wise - Prefer Pinojs
47. Log Library To Do The Message Building And Not Your Service logger.info("Person object saved successfully %O", person); logger.info(`Person ${JSON.stringify(person)} saved successfully`);
48. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
49. Web Framework For Writing Node.js Microservices
50. Express Vs Fastify Vs Nest.js Performance benchmark
51. Result - Express
52. Result - Nest.js
53. Results - Fastify
54. Comparing Results - Nest.js vs Express vs Fastify
55. Comparing Results - Nest.js vs Express vs Fastify
56. If you are able to refactor your microservices code or You are starting a new project Choosing a performance efficient web framework will contribute a lot to fast request processing
57. 1. 2. 3.
58. Database Work Performance Efficient Log Library Should We Choose A different web Framework ? Caching Efficient JSON parsing and stringify
59. Authenticating Users With JWT Tokens
60. Adding JWT Authentication To The Server
61. Validating Request With JWT Token import jwt from 'jsonwebtoken'; // Middleware to parse the JWT token from the request app.use((req, res, next) => { const authHeader = req.headers.authorization; if (authHeader && authHeader.startsWith('Bearer ')) { const token = authHeader.split(' ')[1]; try { // Verify and decode the JWT token const decodedToken = jwt.verify(token, secretKey); // Attach the decoded token to the request object req.decodedToken = decodedToken; } catch (error) { return res.status(401).json({ error: 'Invalid token' }); } } next(); });
62. Performance Comparison After Adding JWT Authentication
63. Results - Without Authentication
64. Results - With Authentication
65. To Solve The Performance Problem, Lets Add Caching
66. Results - Without Authentication
67. Results - With Authentication
68. Results - Authentication And Caching
69. Performance Comparison - With Caching
70. Performance Comparison - With Caching
71. Performance Comparison - With Caching
72. Node-Cache - In memory caching import NodeCache from 'node-cache'; const cache = new NodeCache({ stdTTL: 3600 }); const cachedToken = cache.get(token); cache.set(token, decodedToken);
73. Prefer Using Redis - Distributed Cache import { createClient } from 'redis'; const client = await createClient().connect(); await client.set('key', 'value'); const value = await client.get('key'); await client.disconnect();
74. Cache Computationally Intensive Operations I/O - DB Queries I/O - HTTP Requests From Third Party Encoding Encryption / Decryption
75. Cache best practices • Use read-through pattern — first check cache, fallback to DB/upstream API. • Cache based on request parameters to avoid stale data. • Set proper cache TTL. • Watch cache hit ratio and fine tune expiry times accordingly. • Avoid caching user-specific or sensitive data.
76. Summary • Optimize your DB queries • For large scale - work with the DB driver • Have the ability to optimize your connection pool • Use caching when needed • Choose efficient - ◦ Serializer ◦ Log library ◦ Web framework
77. Tamar Twena-Stern twitter @sterntwena linkedin /tamarstern tamar.twena@gmail.com