How to make high quality Node.js application at JSDevTalk

1. How to Make a High Quality Node.js Application by Nikita Galkin Oct 28, 2018
2. 2 “I picked Java because I felt the most people would be able to understand the code examples if they were written in Java. That was the case in 1997, but how about in 2017? … But I went for the alternative: JavaScript.” 27 March 2018
3. 3 Take notes and todos Ask questions Share ideas Keep calm and validate
4. Nikita Galkin Love and Know: ▰ How to make developers and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 4
5. 5 What is Quality?
6. What is Quality? 6 Attributes: ▰ Portability ▰ Reliability ▰ Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability
7. What is Quality? 7 ▰ Portability – DevOps Engineer ▰ Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer ▰ Understandability – Developer ▰ Modifiability – Developer
8. 8 Understandability
9. “Understandability is about a software product that is easy for a maintainer to comprehend. 9 9
10. Code style 10 ▰ Eslint (tslint) is your Wunderwaffe ▻ tslint -p tsconfig.json --fix ▻ eslint . --fix ▻ WebStorm can set style from eslint file ▰ If you don’t lint your code, then start with eslint helper ▰ More at Code Style and Violence by Anton Nemtsev
11. Node.js Async Programming Paradigms 11 ▰ callbacks ▰ usage async package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project
12. makeRequest('http://server.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) { saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });
13. makeRequest('http://server.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData; sendResponse(model); });
14. async function example() { const url = 'http://server.com/path/request'; const resp = await makeRequest(url); const processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }
15. Specify the types 15 Choose typescript or JSDoc. Benefits: ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example with autotype-checking
16. /** * @typedef {Object} CoverageEntry * @property {string} url * @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
17. 17 Modifiability
18. “Modifiability is about a software product that is easy for a maintainer to change. 18 18
19. How Can You Increase Modifiability? 19 ▰ Be consistent! ▰ Keep your readme.md up-to-date ▰ Use generators ▻ plop ▻ Yeoman.io ▰ Remove unused code and dependencies ▻ depcheck
20. Readme structure 20 ▰ Overview ▰ Development ▻ Tooling ▻ Project structure ▰ Configuration ▰ Run and Deploy ▰ Testing approach
21. Generators 21
22. 22 Portability
23. “Portability is about creating a software product that is easily moved to another platform. 23 23
24. What is Quality? 24 Do you know: ▰ How do you release? ▰ How do you configure? ▰ How do you version? ▰ How do you scale? Your DevOps knows.
25. 12factor.net
26. 26 Build Release Run
27. How to be Repeatable? 27 ▰ Explicitly declare and isolate dependencies ▰ Lock your dependencies package-lock.json or yarn.lock ▰ This lock file should be committed ▰ Use same tool versions on all environment ▰ Use Docker
28. FROM node:10.12-alpine ENV APP_WORKDIR=/usr/src/app RUN apk update && apk upgrade && \ apk add --virtual build-deps git openssh-client py-pip make COPY package.json yarn.lock $APP_WORKDIR WORKDIR $APP_WORKDIR RUN yarn install COPY . $APP_WORKDIR RUN yarn build RUN rm -rf tsconfig.json src RUN apk del build-deps RUN yarn install --production RUN yarn cache clean EXPOSE 3000 CMD ["node", "dist/index.js"]
29. How to Configure Your Application? 29 ▰ Use .env.example file for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe
30. # .env.example # Default log level LOG_LEVEL=info # Graceful shutdown timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Redis connection string REDIS_URI=redis://127.0.0.1:6379
31. require('dotenv-safe').load({ allowEmptyValues: true, path: join(__dirname, '..', '.env'), sample: join(__dirname, '..', '.env.example') }); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;
32. 32 Reliability
33. “Reliability is about a software product that does what it's supposed to do, dependably. 33 33
34. What Should Your Application Do? 34 It depends, but your Node.js app should: ▰ start ▰ stop ▰ write logs, good packages for that: pino or bunyan ▰ handle errors, for example degrade functionality on problem with 3rd parties ▰ restart on issues. pm2 or Docker can be used for this.
35. How to Stop Your Node.js Application? 35 1. Add hooks on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work
36. const server = app.listen(settings.port, () => { logger.info(`Server started on port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); ['SIGTERM','SIGINT','SIGHUP'] .forEach(signal => process.on(signal, stopHandler);
37. Error handling 37 ▰ Operational errors should be handled ▰ Programmer errors cannot be handled, application should be stopped ▰ Always use instance of Error Object at throw, error event, Promise reject or callback ▰ More
38. import './config'; import logger from './logger'; process.on('unhandledRejection', (reason) => { logger.fatal({ error: reason }, 'Unhandled Rejection'); process.exit(1); }); process.on('uncaughtException', (error) => { logger.fatal(error, 'Unhandled Exception'); process.exit(1); }); process.on('warning', (warning) => { logger.warn(warning, 'Warning detected'); });
39. 39 Efficiency
40. “Efficiency is about a software product that economizes on both running time and space consumption. 40 40
41. Node.js efficiency factors 41 ▰ Memory usage ▰ CPU usage ▰ Connections count ▰ Request per second ▰ Response time ▰ etc
42. Node.js efficiency advices 42 ▰ Reuse connections ▰ Use Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data ▰ Use right protocol ▻ REST ▻ Server-Side-Events ▻ WS ▻ TCP
43. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
44. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile") .pipe(res); }); router.get('/users, function(req, res, next) { users.find({}) .pipe(res); });
45. 45 Usability
46. “Usability is about a software product that is easy and comfortable to use. 46 46
47. Node.js Application Usability 47 Node.js Usability factors: ▰ response time ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required) ▰ 3rd parties like DB/cache/etc
49. ▰ Use sync operations instead of async ▰ Make too many CPU usage in one process ▰ Use something like: while (Date.now() < end) ▰ Use long running event handlers without process.nextTick() How to Block Event Loop? DON’T DO IT 49
50. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() => { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
51. 51 Testability
52. “Testability is about a software product that is easy to test. 52 52
53. Test layers 53 ▰ Unit tests ▰ Integration tests (mocha, jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)
54. const app = require('./../src/app'); const request = require('supertest'); describe('GET /ping', function (){ it('should return status 200', function (done) { request(app) .get('/ping') .expect('Content-Type', /json/) .expect(200, {response: 'success'}, done); }); });
55. describe('CLI', function () { let messageCallback, errorCallback, closeCallback; function startCLI(commandOptions) { let childProcess = require('child_process') .spawn('path/to/cli', commandOptions); childProcess.stdout.on('data', messageCallback); childProcess.stderr.on('data', errorCallback); childProcess.on('close', closeCallback); return childProcess; } it('should stopping gracefully', function(done) { closeCallback = (code) => { expect(code).to.equal(0); done(); } errorCallback = done; let cli = startCLI([`-f=example`]); setTimeout(() => cli.kill('SIGTERM'), 1000); }) })
56. 56 What is quality assurance?
57. Expectations Reality VS
58. 58 THANKS! HAPPY CODING WITH NODE.JS You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or at my site galk.in