Tweet
Tweet

Slide 1

Slide 1 text

How to Make a High Quality Node.js Application by Nikita Galkin Oct 28, 2018

Slide 2

Slide 2 text

2 “I picked Java because I felt the most people would be able to understand the code examples if they were written in Java. That was the case in 1997, but how about in 2017? … But I went for the alternative: JavaScript.” 27 March 2018

Slide 3

Slide 3 text

3 Take notes and todos Ask questions Share ideas Keep calm and validate

Slide 4

Slide 4 text

Nikita Galkin Love and Know: ▰ How to make developers and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 4

Slide 5

Slide 5 text

5 What is Quality?

Slide 6

Slide 6 text

What is Quality? 6 Attributes: ▰ Portability ▰ Reliability ▰ Efficiency ▰ Usability ▰ Testability ▰ Understandability ▰ Modifiability

Slide 7

Slide 7 text

What is Quality? 7 ▰ Portability – DevOps Engineer ▰ Reliability – Product Owner ▰ Efficiency – System Architect ▰ Usability – UX/UI Designer ▰ Testability – QA Engineer ▰ Understandability – Developer ▰ Modifiability – Developer

Slide 8

Slide 8 text

8 Understandability

Slide 9

Slide 9 text

“Understandability is about a software product that is easy for a maintainer to comprehend. 9 9

Slide 10

Slide 10 text

Code style 10 ▰ Eslint (tslint) is your Wunderwaffe ▻ tslint -p tsconfig.json --fix ▻ eslint . --fix ▻ WebStorm can set style from eslint file ▰ If you don’t lint your code, then start with eslint helper ▰ More at Code Style and Violence by Anton Nemtsev

Slide 11

Slide 11 text

Node.js Async Programming Paradigms 11 ▰ callbacks ▰ usage async package ▰ promises ▰ generators ▰ async/await Please don’t use several of them in one project

Slide 12

Slide 12 text

makeRequest('http://server.com/path/request', function step1(req, resp) { processData(resp, function step2(err, processedData) { saveData(processedData, function step3(err, model) { loadRelatedData(model, function step4(err, relData) { model.rel = relData; sendResponse(model); }); }); }); });

Slide 13

Slide 13 text

makeRequest('http://server.com/path/request') .then(processData) .then(saveData) .then(loadRelatedData) .then((relData) => { model.rel = relData; sendResponse(model); });

Slide 14

Slide 14 text

async function example() { const url = 'http://server.com/path/request'; const resp = await makeRequest(url); const processedData = await processData(resp); const model = await saveData(processedData); model.rel = await loadRelatedData(model); await sendResponse(model); }

Slide 15

Slide 15 text

Specify the types 15 Choose typescript or JSDoc. Benefits: ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example with autotype-checking

Slide 16

Slide 16 text

/** * @typedef {Object} CoverageEntry * @property {string} url * @property {string} text * @property {!Array} ranges */

Slide 17

Slide 17 text

17 Modifiability

Slide 18

Slide 18 text

“Modifiability is about a software product that is easy for a maintainer to change. 18 18

Slide 19

Slide 19 text

How Can You Increase Modifiability? 19 ▰ Be consistent! ▰ Keep your readme.md up-to-date ▰ Use generators ▻ plop ▻ Yeoman.io ▰ Remove unused code and dependencies ▻ depcheck

Slide 20

Slide 20 text

Readme structure 20 ▰ Overview ▰ Development ▻ Tooling ▻ Project structure ▰ Configuration ▰ Run and Deploy ▰ Testing approach

Slide 21

Slide 21 text

Generators 21

Slide 22

Slide 22 text

22 Portability

Slide 23

Slide 23 text

“Portability is about creating a software product that is easily moved to another platform. 23 23

Slide 24

Slide 24 text

What is Quality? 24 Do you know: ▰ How do you release? ▰ How do you configure? ▰ How do you version? ▰ How do you scale? Your DevOps knows.

Slide 25

Slide 25 text

12factor.net

Slide 26

Slide 26 text

26 Build Release Run

Slide 27

Slide 27 text

How to be Repeatable? 27 ▰ Explicitly declare and isolate dependencies ▰ Lock your dependencies package-lock.json or yarn.lock ▰ This lock file should be committed ▰ Use same tool versions on all environment ▰ Use Docker

Slide 28

Slide 28 text

FROM node:10.12-alpine ENV APP_WORKDIR=/usr/src/app RUN apk update && apk upgrade && \ apk add --virtual build-deps git openssh-client py-pip make COPY package.json yarn.lock $APP_WORKDIR WORKDIR $APP_WORKDIR RUN yarn install COPY . $APP_WORKDIR RUN yarn build RUN rm -rf tsconfig.json src RUN apk del build-deps RUN yarn install --production RUN yarn cache clean EXPOSE 3000 CMD ["node", "dist/index.js"]

Slide 29

Slide 29 text

How to Configure Your Application? 29 ▰ Use .env.example file for description ▰ Use .env file for local development ▰ Use environment variables for any other environment ▰ Don’t use default config values in your code ▰ Break your start process if any value is missed ▰ Use dotenv-safe

Slide 30

Slide 30 text

# .env.example # Default log level LOG_LEVEL=info # Graceful shutdown timeout in milliseconds SHUTDOWN_TIMEOUT=1000 # HTTP Boundary port HTTP_PORT=8000 # Redis connection string REDIS_URI=redis://127.0.0.1:6379

Slide 31

Slide 31 text

require('dotenv-safe').load({ allowEmptyValues: true, path: join(__dirname, '..', '.env'), sample: join(__dirname, '..', '.env.example') }); const logger = require('./logger'); logger.levels('stdout', process.env.LOG_LEVEL); const settings = { shutdownTimeout: process.env.SHUTDOWN_TIMEOUT, http: { port: process.env.HTTP_PORT, }, redis: { uri: process.env.REDIS_URI, }, }; module.exports = settings;

Slide 32

Slide 32 text

32 Reliability

Slide 33

Slide 33 text

“Reliability is about a software product that does what it's supposed to do, dependably. 33 33

Slide 34

Slide 34 text

What Should Your Application Do? 34 It depends, but your Node.js app should: ▰ start ▰ stop ▰ write logs, good packages for that: pino or bunyan ▰ handle errors, for example degrade functionality on problem with 3rd parties ▰ restart on issues. pm2 or Docker can be used for this.

Slide 35

Slide 35 text

How to Stop Your Node.js Application? 35 1. Add hooks on signals 2. Log start of graceful shutdown 3. Close incoming business logic flow 4. Set a forced timeout 5. Notify consumers about shutdown 6. Correctly disconnect from all connections or mark them unref 7. Clean all timeout and intervals 8. Node.js will finish work

Slide 36

Slide 36 text

const server = app.listen(settings.port, () => { logger.info(`Server started on port ${settings.port}`); }); function stopHandler(){ logger.info(`Stopping server on port ${settings.port}`); const timeout = setTimeout(() => { logger.info(`Server on port ${settings.port} stop forcefully`); process.exit(1); }, settings.stopTimeout); server.close(() => { logger.info(`Server on port ${settings.port} stopped`); clearTimeout(timeout); }); }; process.on('exit', (code) => logger.info(`Exit with code: ${code}`)); ['SIGTERM','SIGINT','SIGHUP'] .forEach(signal => process.on(signal, stopHandler);

Slide 37

Slide 37 text

Error handling 37 ▰ Operational errors should be handled ▰ Programmer errors cannot be handled, application should be stopped ▰ Always use instance of Error Object at throw, error event, Promise reject or callback ▰ More

Slide 38

Slide 38 text

import './config'; import logger from './logger'; process.on('unhandledRejection', (reason) => { logger.fatal({ error: reason }, 'Unhandled Rejection'); process.exit(1); }); process.on('uncaughtException', (error) => { logger.fatal(error, 'Unhandled Exception'); process.exit(1); }); process.on('warning', (warning) => { logger.warn(warning, 'Warning detected'); });

Slide 39

Slide 39 text

39 Efficiency

Slide 40

Slide 40 text

“Efficiency is about a software product that economizes on both running time and space consumption. 40 40

Slide 41

Slide 41 text

Node.js efficiency factors 41 ▰ Memory usage ▰ CPU usage ▰ Connections count ▰ Request per second ▰ Response time ▰ etc

Slide 42

Slide 42 text

Node.js efficiency advices 42 ▰ Reuse connections ▰ Use Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data ▰ Use right protocol ▻ REST ▻ Server-Side-Events ▻ WS ▻ TCP

Slide 43

Slide 43 text

const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds

Slide 44

Slide 44 text

router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile") .pipe(res); }); router.get('/users, function(req, res, next) { users.find({}) .pipe(res); });

Slide 45

Slide 45 text

45 Usability

Slide 46

Slide 46 text

“Usability is about a software product that is easy and comfortable to use. 46 46

Slide 47

Slide 47 text

Node.js Application Usability 47 Node.js Usability factors: ▰ response time ▰ response time ▰ and again response time What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ application load (so be ready to scale) ▰ processing type (use streams when they are required) ▰ 3rd parties like DB/cache/etc

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

▰ Use sync operations instead of async ▰ Make too many CPU usage in one process ▰ Use something like: while (Date.now() < end) ▰ Use long running event handlers without process.nextTick() How to Block Event Loop? DON’T DO IT 49

Slide 50

Slide 50 text

let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() => { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);

Slide 51

Slide 51 text

51 Testability

Slide 52

Slide 52 text

“Testability is about a software product that is easy to test. 52 52

Slide 53

Slide 53 text

Test layers 53 ▰ Unit tests ▰ Integration tests (mocha, jest, etc) ▰ REST API tests (abao, dredd) ▰ e2e functional tests (QA specific) ▰ performance tests (QA specific)

Slide 54

Slide 54 text

const app = require('./../src/app'); const request = require('supertest'); describe('GET /ping', function (){ it('should return status 200', function (done) { request(app) .get('/ping') .expect('Content-Type', /json/) .expect(200, {response: 'success'}, done); }); });

Slide 55

Slide 55 text

describe('CLI', function () { let messageCallback, errorCallback, closeCallback; function startCLI(commandOptions) { let childProcess = require('child_process') .spawn('path/to/cli', commandOptions); childProcess.stdout.on('data', messageCallback); childProcess.stderr.on('data', errorCallback); childProcess.on('close', closeCallback); return childProcess; } it('should stopping gracefully', function(done) { closeCallback = (code) => { expect(code).to.equal(0); done(); } errorCallback = done; let cli = startCLI([`-f=example`]); setTimeout(() => cli.kill('SIGTERM'), 1000); }) })

Slide 56

Slide 56 text

56 What is quality assurance?

Slide 57

Slide 57 text

Expectations Reality VS

Slide 58

Slide 58 text

58 THANKS! HAPPY CODING WITH NODE.JS You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or at my site galk.in