How to Make a High Quality Node.js Application at Morning Lohika

How to Make a High Quality Node.js Application at Morning Lohika

We will start with the question “What is Quality?”. In the answer, we will find several attributes, which will be chapters in this talk. For every attribute, we will talk, how this quality attribute can be improved in a Node.js application, which tools or process can be used for that.

A2443b68836f2d166eae52b940e99bf6?s=128

Nikita Galkin

August 04, 2018
Tweet

Transcript

  1. How to Make a High Quality Node.js Application by Nikita

    Galkin Aug 4, 2018
  2. Nikita Galkin Love and Know: ▰ How to make developers

    and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 2
  3. 3 What are your expectations?

  4. What are they doing?

  5. 5 Take notes and todos Ask questions Share ideas Keep

    calm and validate
  6. 6 What is Quality?

  7. What is Quality? 7 Attributes: ▰ Portability ▰ Reliability ▰

    Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability
  8. What is Quality? 8 ▰ Portability – DevOps Engineer ▰

    Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer ▰ Understandability – Developer ▰ Modifiability – Developer
  9. 9 Understandability

  10. “Understandability is about a software product that is easy for

    a maintainer to comprehend. 10 10
  11. Code style 11 ▰ Eslint (tslint) is your Wunderwaffe ▻

    tslint -p tsconfig.json --fix ▻ eslint . --fix ▻ WebStorm can set style from eslint file ▰ If you don’t lint your code, then start with eslint helper ▰ More at Code Style and Violence by Anton Nemtsev
  12. Node.js Async Programming Paradigms 12 ▰ callbacks ▰ usage async

    package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project
  13. makeRequest('http://server.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) {

    saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });
  14. makeRequest('http://server.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData;

    sendResponse(model); });
  15. async function example() { const resp = await makeRequest('http://server.com/path/request') const

    processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }
  16. Specify the types 16 Choose typescript or JSDoc. Benefits: ▰

    autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example with autotype-checking
  17. /** * @typedef {Object} CoverageEntry * @property {string} url *

    @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
  18. 18 Modifiability

  19. “Modifiability is about a software product that is easy for

    a maintainer to change. 19 19
  20. How Can You Increase Modifiability? 20 ▰ Be consistent! ▰

    Keep your readme.md up-to-date ▰ Use generators ▻ plop ▻ Yeoman.io ▰ Remove unused code and dependencies ▻ depcheck
  21. Readme structure 21 ▰ Overview ▰ Development ▻ Tooling ▻

    Project structure ▰ Configuration ▰ Run and Deploy ▰ Testing approach
  22. Generators 22

  23. 23 Portability

  24. “Portability is about creating a software product that is easily

    moved to another platform. 24 24
  25. What is Quality? 25 Do you know: ▰ How do

    you release? ▰ How do you configure? ▰ How do you version? ▰ How do you scale? Your DevOps knows.
  26. 12factor.net

  27. How to be Repeatable? 27 ▰ Explicitly declare and isolate

    dependencies ▰ Lock your dependencies package-lock.json or yarn.lock ▰ This lock file should be committed ▰ Use same tool versions on all environment ▰ Use Docker
  28. FROM node:8.11-alpine ENV APP_WORKDIR=/usr/src/app RUN apk update && apk upgrade

    && \ apk add --virtual build-deps git openssh-client py-pip make COPY package.json yarn.lock $APP_WORKDIR WORKDIR $APP_WORKDIR RUN yarn install COPY . $APP_WORKDIR RUN yarn build RUN rm -rf tsconfig.json src RUN apk del build-deps RUN yarn install --production RUN yarn cache clean EXPOSE 3000 CMD ["node", "dist/index.js"]
  29. How to Configure Your Application? 29 ▰ Use .env.example file

    for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe
  30. # .env.example # Default log level LOG_LEVEL=info # Graceful shutdown

    timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Redis connection string REDIS_URI=redis://127.0.0.1:6379
  31. require('dotenv-safe').load({ allowEmptyValues: true, path: join(__dirname, '..', '.env'), sample: join(__dirname, '..',

    '.env.example') }); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, api: { prefix: process.env.HTTP_API_PREFIX, }, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;
  32. 32 Reliability

  33. “Reliability is about a software product that does what it's

    supposed to do, dependably. 33 33
  34. What Should Your Application Do? 34 It depends, but your

    Node.js app should: ▰ start ▰ stop ▰ write logs ▰ degrade functionality on problem with 3rd parties ▰ restart on issues. pm2 or Docker can be used for this.
  35. How to Stop Your Node.js Application? 35 1. Add hooks

    on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work
  36. const server = app.listen(settings.port, () => { logger.info(`Server started on

    port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); ['SIGTERM','SIGINT','SIGHUP'].forEach(signal => process.on(signal, stopHandler);
  37. 37 Efficiency

  38. “Efficiency is about a software product that economizes on both

    running time and space consumption. 38 38
  39. Node.js efficiency factors 39 ▰ Memory usage ▰ CPU usage

    ▰ Connections count ▰ Request per second ▰ Response time ▰ etc
  40. Node.js efficiency advices 40 ▰ Reuse connections ▰ Use Promise.all()

    in async functions ▰ Make calculation in DB layer ▰ Use precalculated data ▰ Use right protocol ▻ REST ▻ Server-Side-Events ▻ WS ▻ TCP
  41. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const

    pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
  42. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile").pipe(res); }); router.get('/users, function(req, res,

    next) { users.find({}).pipe(res); });
  43. 43 Usability

  44. “Usability is about a software product that is easy and

    comfortable to use. 44 44
  45. Node.js Application Usability 45 Node.js Usability factors: ▰ response time

    ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required) ▰ 3rd parties like DB/cache/etc
  46. None
  47. ▰ Use sync operations instead of async ▰ Make too

    many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end) ▰ Use long running event handlers without process.nextTick() How to Block Event Loop? DON’T DO IT 47
  48. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() =>

    { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
  49. 49 Testability

  50. “Testability is about a software product that is easy to

    test. 50 50
  51. Test layers 51 ▰ Unit tests ▰ Integration tests (mocha,

    jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)
  52. const app = require('./../src/app'); const request = require('supertest'); describe('GET /ping',

    function (){ it('should return status 200', function (done) { request(app) .get('/ping') .expect('Content-Type', /json/) .expect(200, {response: 'success'}, done); }); });
  53. describe('CLI', function () { let messageCallback, errorCallback, closeCallback function startCLI(commandOptions)

    { let childProcess = require('child_process').spawn('path/to/cli', commandOptions) childProcess.stdout.on('data', messageCallback) childProcess.stderr.on('data', errorCallback) childProcess.on('close', closeCallback) return childProcess } it('should stopping gracefully', function(done) { closeCallback = (code) => { expect(code).to.equal(0) done() } errorCallback = done let cli = startCLI([`-f=example`]) setTimeout(() => cli.kill('SIGTERM'), 1000) }) })
  54. 54 What is quality assurance?

  55. Об ожиданиях 55

  56. 56 THANKS! HAPPY CODING WITH NODE.JS