How to Make a High Quality Node.js Application at Morning Lohika

We will start with the question “What is Quality?”. In the answer, we will find several attributes, which will be chapters in this talk. For every attribute, we will talk, how this quality attribute can be improved in a Node.js application, which tools or process can be used for that.

Transcript

1. How to Make a High Quality Node.js Application by Nikita Galkin Aug 4, 2018
2. Nikita Galkin Love and Know: ▰ How to make developers and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 2
3. 3 What are your expectations?
4. What are they doing?
5. 5 Take notes and todos Ask questions Share ideas Keep calm and validate
6. 6 What is Quality?
7. What is Quality? 7 Attributes: ▰ Portability ▰ Reliability ▰ Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability
8. What is Quality? 8 ▰ Portability – DevOps Engineer ▰ Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer ▰ Understandability – Developer ▰ Modifiability – Developer
9. 9 Understandability
10. “Understandability is about a software product that is easy for a maintainer to comprehend. 10 10
11. Code style 11 ▰ Eslint (tslint) is your Wunderwaffe ▻ tslint -p tsconfig.json --fix ▻ eslint . --fix ▻ WebStorm can set style from eslint file ▰ If you don’t lint your code, then start with eslint helper ▰ More at Code Style and Violence by Anton Nemtsev
12. Node.js Async Programming Paradigms 12 ▰ callbacks ▰ usage async package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project
13. makeRequest('http://server.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) { saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });
14. makeRequest('http://server.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData; sendResponse(model); });
15. async function example() { const resp = await makeRequest('http://server.com/path/request') const processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }
16. Specify the types 16 Choose typescript or JSDoc. Benefits: ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example with autotype-checking
17. /** * @typedef {Object} CoverageEntry * @property {string} url * @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
18. 18 Modifiability
19. “Modifiability is about a software product that is easy for a maintainer to change. 19 19
20. How Can You Increase Modifiability? 20 ▰ Be consistent! ▰ Keep your readme.md up-to-date ▰ Use generators ▻ plop ▻ Yeoman.io ▰ Remove unused code and dependencies ▻ depcheck
21. Readme structure 21 ▰ Overview ▰ Development ▻ Tooling ▻ Project structure ▰ Configuration ▰ Run and Deploy ▰ Testing approach
22. Generators 22
23. 23 Portability
24. “Portability is about creating a software product that is easily moved to another platform. 24 24
25. What is Quality? 25 Do you know: ▰ How do you release? ▰ How do you configure? ▰ How do you version? ▰ How do you scale? Your DevOps knows.
26. 12factor.net
27. How to be Repeatable? 27 ▰ Explicitly declare and isolate dependencies ▰ Lock your dependencies package-lock.json or yarn.lock ▰ This lock file should be committed ▰ Use same tool versions on all environment ▰ Use Docker
28. FROM node:8.11-alpine ENV APP_WORKDIR=/usr/src/app RUN apk update && apk upgrade && \ apk add --virtual build-deps git openssh-client py-pip make COPY package.json yarn.lock $APP_WORKDIR WORKDIR $APP_WORKDIR RUN yarn install COPY . $APP_WORKDIR RUN yarn build RUN rm -rf tsconfig.json src RUN apk del build-deps RUN yarn install --production RUN yarn cache clean EXPOSE 3000 CMD ["node", "dist/index.js"]
29. How to Configure Your Application? 29 ▰ Use .env.example file for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe
30. # .env.example # Default log level LOG_LEVEL=info # Graceful shutdown timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Redis connection string REDIS_URI=redis://127.0.0.1:6379
31. require('dotenv-safe').load({ allowEmptyValues: true, path: join(__dirname, '..', '.env'), sample: join(__dirname, '..', '.env.example') }); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, api: { prefix: process.env.HTTP_API_PREFIX, }, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;
32. 32 Reliability
33. “Reliability is about a software product that does what it's supposed to do, dependably. 33 33
34. What Should Your Application Do? 34 It depends, but your Node.js app should: ▰ start ▰ stop ▰ write logs ▰ degrade functionality on problem with 3rd parties ▰ restart on issues. pm2 or Docker can be used for this.
35. How to Stop Your Node.js Application? 35 1. Add hooks on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work
36. const server = app.listen(settings.port, () => { logger.info(`Server started on port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); ['SIGTERM','SIGINT','SIGHUP'].forEach(signal => process.on(signal, stopHandler);
37. 37 Efficiency
38. “Efficiency is about a software product that economizes on both running time and space consumption. 38 38
39. Node.js efficiency factors 39 ▰ Memory usage ▰ CPU usage ▰ Connections count ▰ Request per second ▰ Response time ▰ etc
40. Node.js efficiency advices 40 ▰ Reuse connections ▰ Use Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data ▰ Use right protocol ▻ REST ▻ Server-Side-Events ▻ WS ▻ TCP
41. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
42. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile").pipe(res); }); router.get('/users, function(req, res, next) { users.find({}).pipe(res); });
43. 43 Usability
44. “Usability is about a software product that is easy and comfortable to use. 44 44
45. Node.js Application Usability 45 Node.js Usability factors: ▰ response time ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required) ▰ 3rd parties like DB/cache/etc
47. ▰ Use sync operations instead of async ▰ Make too many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end) ▰ Use long running event handlers without process.nextTick() How to Block Event Loop? DON’T DO IT 47
48. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() => { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
49. 49 Testability
50. “Testability is about a software product that is easy to test. 50 50
51. Test layers 51 ▰ Unit tests ▰ Integration tests (mocha, jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)
52. const app = require('./../src/app'); const request = require('supertest'); describe('GET /ping', function (){ it('should return status 200', function (done) { request(app) .get('/ping') .expect('Content-Type', /json/) .expect(200, {response: 'success'}, done); }); });
53. describe('CLI', function () { let messageCallback, errorCallback, closeCallback function startCLI(commandOptions) { let childProcess = require('child_process').spawn('path/to/cli', commandOptions) childProcess.stdout.on('data', messageCallback) childProcess.stderr.on('data', errorCallback) childProcess.on('close', closeCallback) return childProcess } it('should stopping gracefully', function(done) { closeCallback = (code) => { expect(code).to.equal(0) done() } errorCallback = done let cli = startCLI([`-f=example`]) setTimeout(() => cli.kill('SIGTERM'), 1000) }) })
54. 54 What is quality assurance?
55. Об ожиданиях 55
56. 56 THANKS! HAPPY CODING WITH NODE.JS