How to Make a High Quality Node.js Application at Sigma JS Meetup

In this talk we discuss about quality of Node.js application as a collection of seven attributes: portability, reliability, efficiency, usability (human engineering), testability, understandability, and modifiability.

Sigma JavaScript Meetup, Sep 4 2017

Transcript

1. How to Make a High Quality Node.js Application by Nikita Galkin Sep 3, 2017
2. Nikita Galkin Love and Know: ▰ JavaScript, Node.js, OpenSource, TypeScript, Docker, AWS ▰ How to split monolith into microservices Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard. ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Last Talks: ▰ Docker for Node.js developer ▰ 5 production Node.js stories ▰ Testing in Node.js World ▰ TypeScript for Node.js applications ▰ Spec driven development in Microservices Links: Site GitHub Twitter Facebook 2
3. 3 What Is Node.js Today?
6. ▰ Install Node.js through nvm on your computer ▰ Prefer LTS version for production ▰ Use node.green for checking ECMAScript feature support in node.js version ▰ Read and reread Node.js native modules official documentation Node.js 6
7. 7 What is Quality for a Node.js Application?
8. What is Quality? 8 Attributes: ▰ Portability ▰ Reliability ▰ Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability
9. Who Takes Care of Which Attribute? 9 ▰ Understandability – Developer ▰ Modifiability – Developer ▰ Portability – DevOps Engineer ▰ Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer
10. 10 Understandability
11. “Understandability is about a software product that is easy for a maintainer to comprehend. 11 11
12. Node.js Async Programming Paradigms 12 ▰ callbacks ▰ usage async package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project
13. Callback Hell 13 makeRequest('http://yourserver.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) { saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });
14. Promise Chain 14 makeRequest('http://yourserver.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData; sendResponse(model); });
15. async/await Example 15 async function example() { const resp = await makeRequest('http://yourserver.com/path/request'); const processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }
16. Node.js Async Programming Trend 16 ▰ async/await approach ▰ koa@2.x instead of express ▰ Node.js v8 with native async/await support ▰ Use of util.promisify for creating Promise from function with callback
17. 17 Modifiability
18. “Modifiability is about a software product that is easy for a maintainer to change. 18 18
19. How Can You Increase Modifiability? 19 ▰ Keep your readme.md up-to-date ▰ Use eslint rules ▰ Reduce coupling by ▻ Dependency Injection ▻ Settings instead of hardcoded values ▰ Write js doc blocks or use typescript
20. //Without DI const logger = require('./logger'); const db = require('./database'); class UsersRepository { async findUser(id) { let user; try { user = await db.find(id); } catch (e) { logger.error(`User ${id} not found`); } return user; } } Dependency Injection Example 20 //With DI class UsersRepository { constructor(logger, db) { this.logger = logger; this.db = db; } async findUser(id) { let user; try { user = await this.db.find(id); } catch (e) { this.logger.error(`User ${id} not found`); } return user; } }
21. 21 Portability
22. “Portability is about creating a software product that is easily moved to another platform. 22 22
23. How Does Node.js Support Repeatability? 23 ▰ Node.js application has package.json ▰ package-lock.json new implementation of shrinkwrap ▰ This package-lock.json should be committed to the git repo
24. How to Configure Your Node.js Application? 24 What to configure: ▰ environment type ▰ ports ▰ connections ▰ tokens How to configure: ▰ config in package.json ▰ config files ▰ environment variables ▰ service registry
25. How to Configure Your Application? 25 ▰ Use .env.example file for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe
26. dotenv-safe Example 26 # Default log level LOG_LEVEL=info # Graceful shutdown timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Temporary Redis connection string REDIS_URI=redis://127.0.0.1:6379 require('dotenv-safe').load(); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, api: { prefix: process.env.HTTP_API_PREFIX, }, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;
27. Portability Fuckup Example 27 ▰ After git clone <repo> there are a lot changed files ▰ File naming convention is camelCase instead of snake-case ▰ Mac OS and Ubuntu were used during development ▰ Mac OS has case insensitive filesystem ▰ Ubuntu has case sensitive filesystem
28. 28 Reliability
29. “Reliability is about a software product that does what it's supposed to do, dependably. 29 29
30. What Should Your Application Do? 30 It depends, but your Node.js app should: ▰ start ▰ stop ▰ write logs ▰ degrade functionality ▰ restart on issues. pm2 can be used for this.
31. Reliability Fuckup Example 31 ▰ During any deploy part of user’s data was losted ▰ DevOps Engineer restarted DB after each deploy ▰ There were transactions, which weren’t finished during the stop
32. How to Stop Your Node.js Application? 32 1. Add hooks on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work
33. const server = app.listen(settings.port, () => { logger.info(`Server started on port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully, not all connection was closed`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); process.on('SIGTERM', stopHandler); Graceful Shutdown Example 33
34. 34 Efficiency
35. “Efficiency is about a software product that economizes on both running time and space consumption. 35 35
36. Node.js efficiency factors 36 ▰ scalability ▰ performance ▰ memory usage ▰ CPU usage
37. let theThing = null; const replaceThing = () => { const originalThing = theThing; const unused = () => { if (originalThing) console.log("hi"); }; theThing = { longStr: new Array(1000000).join('*'), someMethod: function () { console.log(someMessage); } }; }; setInterval(replaceThing, 1000); Memory Leaks Example in Node.js 37 async function propClientGet (ipcClient, request) { const propClient = createGetRequest(request); ipcClient.write(propClient); return new Promise(resolve => { ipcClient.on('data', resolve); }); };
38. 38 Usability
39. “Human engineering (also known as usability) is about a software product that is easy and comfortable to use. 39 39
40. Node.js Application Usability 40 Node.js Usability factors: ▰ response time ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required)
41. ▰ Use sync operations instead of async ▰ Make too many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end) How to Block Event Loop? 41
42. 42 Testability
43. “Testability is about a software product that is easy to test. 43 43
44. How test your application? 44 ▰ Unit tests (only for complex code) ▰ Integration tests (mocha, jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)
45. 45 THANKS! HAPPY CODING WITH NODE.JS You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or at my site galk.in