How to make high quality Node.js application at JSDevTalk

How to make high quality Node.js application at JSDevTalk

A2443b68836f2d166eae52b940e99bf6?s=128

Nikita Galkin

October 28, 2018
Tweet

Transcript

  1. How to Make a High Quality Node.js Application by Nikita

    Galkin Oct 28, 2018
  2. 2 “I picked Java because I felt the most people

    would be able to understand the code examples if they were written in Java. That was the case in 1997, but how about in 2017? … But I went for the alternative: JavaScript.” 27 March 2018
  3. 3 Take notes and todos Ask questions Share ideas Keep

    calm and validate
  4. Nikita Galkin Love and Know: ▰ How to make developers

    and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 4
  5. 5 What is Quality?

  6. What is Quality? 6 Attributes: ▰ Portability ▰ Reliability ▰

    Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability
  7. What is Quality? 7 ▰ Portability – DevOps Engineer ▰

    Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer ▰ Understandability – Developer ▰ Modifiability – Developer
  8. 8 Understandability

  9. “Understandability is about a software product that is easy for

    a maintainer to comprehend. 9 9
  10. Code style 10 ▰ Eslint (tslint) is your Wunderwaffe ▻

    tslint -p tsconfig.json --fix ▻ eslint . --fix ▻ WebStorm can set style from eslint file ▰ If you don’t lint your code, then start with eslint helper ▰ More at Code Style and Violence by Anton Nemtsev
  11. Node.js Async Programming Paradigms 11 ▰ callbacks ▰ usage async

    package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project
  12. makeRequest('http://server.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) {

    saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });
  13. makeRequest('http://server.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData;

    sendResponse(model); });
  14. async function example() { const url = 'http://server.com/path/request'; const resp

    = await makeRequest(url); const processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }
  15. Specify the types 15 Choose typescript or JSDoc. Benefits: ▰

    autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example with autotype-checking
  16. /** * @typedef {Object} CoverageEntry * @property {string} url *

    @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
  17. 17 Modifiability

  18. “Modifiability is about a software product that is easy for

    a maintainer to change. 18 18
  19. How Can You Increase Modifiability? 19 ▰ Be consistent! ▰

    Keep your readme.md up-to-date ▰ Use generators ▻ plop ▻ Yeoman.io ▰ Remove unused code and dependencies ▻ depcheck
  20. Readme structure 20 ▰ Overview ▰ Development ▻ Tooling ▻

    Project structure ▰ Configuration ▰ Run and Deploy ▰ Testing approach
  21. Generators 21

  22. 22 Portability

  23. “Portability is about creating a software product that is easily

    moved to another platform. 23 23
  24. What is Quality? 24 Do you know: ▰ How do

    you release? ▰ How do you configure? ▰ How do you version? ▰ How do you scale? Your DevOps knows.
  25. 12factor.net

  26. 26 Build Release Run

  27. How to be Repeatable? 27 ▰ Explicitly declare and isolate

    dependencies ▰ Lock your dependencies package-lock.json or yarn.lock ▰ This lock file should be committed ▰ Use same tool versions on all environment ▰ Use Docker
  28. FROM node:10.12-alpine ENV APP_WORKDIR=/usr/src/app RUN apk update && apk upgrade

    && \ apk add --virtual build-deps git openssh-client py-pip make COPY package.json yarn.lock $APP_WORKDIR WORKDIR $APP_WORKDIR RUN yarn install COPY . $APP_WORKDIR RUN yarn build RUN rm -rf tsconfig.json src RUN apk del build-deps RUN yarn install --production RUN yarn cache clean EXPOSE 3000 CMD ["node", "dist/index.js"]
  29. How to Configure Your Application? 29 ▰ Use .env.example file

    for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe
  30. # .env.example # Default log level LOG_LEVEL=info # Graceful shutdown

    timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Redis connection string REDIS_URI=redis://127.0.0.1:6379
  31. require('dotenv-safe').load({ allowEmptyValues: true, path: join(__dirname, '..', '.env'), sample: join(__dirname, '..',

    '.env.example') }); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;
  32. 32 Reliability

  33. “Reliability is about a software product that does what it's

    supposed to do, dependably. 33 33
  34. What Should Your Application Do? 34 It depends, but your

    Node.js app should: ▰ start ▰ stop ▰ write logs, good packages for that: pino or bunyan ▰ handle errors, for example degrade functionality on problem with 3rd parties ▰ restart on issues. pm2 or Docker can be used for this.
  35. How to Stop Your Node.js Application? 35 1. Add hooks

    on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work
  36. const server = app.listen(settings.port, () => { logger.info(`Server started on

    port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); ['SIGTERM','SIGINT','SIGHUP'] .forEach(signal => process.on(signal, stopHandler);
  37. Error handling 37 ▰ Operational errors should be handled ▰

    Programmer errors cannot be handled, application should be stopped ▰ Always use instance of Error Object at throw, error event, Promise reject or callback ▰ More
  38. import './config'; import logger from './logger'; process.on('unhandledRejection', (reason) => {

    logger.fatal({ error: reason }, 'Unhandled Rejection'); process.exit(1); }); process.on('uncaughtException', (error) => { logger.fatal(error, 'Unhandled Exception'); process.exit(1); }); process.on('warning', (warning) => { logger.warn(warning, 'Warning detected'); });
  39. 39 Efficiency

  40. “Efficiency is about a software product that economizes on both

    running time and space consumption. 40 40
  41. Node.js efficiency factors 41 ▰ Memory usage ▰ CPU usage

    ▰ Connections count ▰ Request per second ▰ Response time ▰ etc
  42. Node.js efficiency advices 42 ▰ Reuse connections ▰ Use Promise.all()

    in async functions ▰ Make calculation in DB layer ▰ Use precalculated data ▰ Use right protocol ▻ REST ▻ Server-Side-Events ▻ WS ▻ TCP
  43. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const

    pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
  44. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile") .pipe(res); }); router.get('/users, function(req,

    res, next) { users.find({}) .pipe(res); });
  45. 45 Usability

  46. “Usability is about a software product that is easy and

    comfortable to use. 46 46
  47. Node.js Application Usability 47 Node.js Usability factors: ▰ response time

    ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required) ▰ 3rd parties like DB/cache/etc
  48. None
  49. ▰ Use sync operations instead of async ▰ Make too

    many CPU usage in one process ▰ Use something like: while (Date.now() < end) ▰ Use long running event handlers without process.nextTick() How to Block Event Loop? DON’T DO IT 49
  50. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() =>

    { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
  51. 51 Testability

  52. “Testability is about a software product that is easy to

    test. 52 52
  53. Test layers 53 ▰ Unit tests ▰ Integration tests (mocha,

    jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)
  54. const app = require('./../src/app'); const request = require('supertest'); describe('GET /ping',

    function (){ it('should return status 200', function (done) { request(app) .get('/ping') .expect('Content-Type', /json/) .expect(200, {response: 'success'}, done); }); });
  55. describe('CLI', function () { let messageCallback, errorCallback, closeCallback; function startCLI(commandOptions)

    { let childProcess = require('child_process') .spawn('path/to/cli', commandOptions); childProcess.stdout.on('data', messageCallback); childProcess.stderr.on('data', errorCallback); childProcess.on('close', closeCallback); return childProcess; } it('should stopping gracefully', function(done) { closeCallback = (code) => { expect(code).to.equal(0); done(); } errorCallback = done; let cli = startCLI([`-f=example`]); setTimeout(() => cli.kill('SIGTERM'), 1000); }) })
  56. 56 What is quality assurance?

  57. Expectations Reality VS

  58. 58 THANKS! HAPPY CODING WITH NODE.JS You can find me

    on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or at my site galk.in