Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to make high quality Node.js application at JSDevTalk

How to make high quality Node.js application at JSDevTalk

Nikita Galkin

October 28, 2018
Tweet

More Decks by Nikita Galkin

Other Decks in Programming

Transcript

  1. How to Make a High
    Quality Node.js
    Application
    by Nikita Galkin
    Oct 28, 2018

    View Slide

  2. 2
    “I picked Java because I felt the
    most people would be able to
    understand the code examples if
    they were written in Java. That
    was the case in 1997, but how
    about in 2017?

    But I went for the alternative:
    JavaScript.”
    27 March 2018

    View Slide

  3. 3
    Take notes and todos
    Ask questions
    Share ideas
    Keep calm and validate

    View Slide

  4. Nikita
    Galkin
    Love and Know:
    ▰ How to make developers and business happy
    ▰ Technical and process debt elimination
    Believe that:
    ▰ Any problem must be solved at the right level
    ▰ Software is easy. People are hard
    ▰ A problem should be highlighted, an idea should
    be "sold", a solution should be demonstrated
    Links:
    Site GitHub Twitter Facebook
    4

    View Slide

  5. 5
    What is Quality?

    View Slide

  6. What is Quality?
    6
    Attributes:
    ▰ Portability
    ▰ Reliability
    ▰ Efficiency
    ▰ Usability
    ▰ Testability
    ▰ Understandability
    ▰ Modifiability

    View Slide

  7. What is Quality?
    7
    ▰ Portability – DevOps Engineer
    ▰ Reliability – Product Owner
    ▰ Efficiency – System Architect
    ▰ Usability – UX/UI Designer
    ▰ Testability – QA Engineer
    ▰ Understandability – Developer
    ▰ Modifiability – Developer

    View Slide

  8. 8
    Understandability

    View Slide

  9. “Understandability is about
    a software product that is
    easy for a maintainer to
    comprehend.
    9
    9

    View Slide

  10. Code style
    10
    ▰ Eslint (tslint) is your Wunderwaffe
    ▻ tslint -p tsconfig.json --fix
    ▻ eslint . --fix
    ▻ WebStorm can set style from eslint file
    ▰ If you don’t lint your code, then start with eslint helper
    ▰ More at Code Style and Violence by Anton Nemtsev

    View Slide

  11. Node.js Async Programming Paradigms
    11
    ▰ callbacks
    ▰ usage async package
    ▰ promises
    ▰ generators
    ▰ async/await
    Please don’t use several of them in one project

    View Slide

  12. makeRequest('http://server.com/path/request',
    function step1(req, resp) {
    processData(resp, function step2(err, processedData) {
    saveData(processedData, function step3(err, model) {
    loadRelatedData(model, function step4(err, relData) {
    model.rel = relData;
    sendResponse(model);
    });
    });
    });
    });

    View Slide

  13. makeRequest('http://server.com/path/request')
    .then(processData)
    .then(saveData)
    .then(loadRelatedData)
    .then((relData) => {
    model.rel = relData;
    sendResponse(model);
    });

    View Slide

  14. async function example() {
    const url = 'http://server.com/path/request';
    const resp = await makeRequest(url);
    const processedData = await processData(resp);
    const model = await saveData(processedData);
    model.rel = await loadRelatedData(model);
    await sendResponse(model);
    }

    View Slide

  15. Specify the types
    15
    Choose typescript or JSDoc. Benefits:
    ▰ autocomplete for engineers
    ▰ always actual code documentation
    ▰ puppeteer source-code is awesome JSDoc
    example with autotype-checking

    View Slide

  16. /**
    * @typedef {Object} CoverageEntry
    * @property {string} url
    * @property {string} text
    * @property {!Array} ranges
    */

    View Slide

  17. 17
    Modifiability

    View Slide

  18. “Modifiability is about a
    software product that is
    easy for a maintainer to
    change.
    18
    18

    View Slide

  19. How Can You Increase Modifiability?
    19
    ▰ Be consistent!
    ▰ Keep your readme.md up-to-date
    ▰ Use generators
    ▻ plop
    ▻ Yeoman.io
    ▰ Remove unused code and dependencies
    ▻ depcheck

    View Slide

  20. Readme structure
    20
    ▰ Overview
    ▰ Development
    ▻ Tooling
    ▻ Project structure
    ▰ Configuration
    ▰ Run and Deploy
    ▰ Testing approach

    View Slide

  21. Generators
    21

    View Slide

  22. 22
    Portability

    View Slide

  23. “Portability is about creating a
    software product that is
    easily moved to another
    platform.
    23
    23

    View Slide

  24. What is Quality?
    24
    Do you know:
    ▰ How do you release?
    ▰ How do you configure?
    ▰ How do you version?
    ▰ How do you scale?
    Your DevOps knows.

    View Slide

  25. 12factor.net

    View Slide

  26. 26
    Build
    Release
    Run

    View Slide

  27. How to be Repeatable?
    27
    ▰ Explicitly declare and isolate dependencies
    ▰ Lock your dependencies package-lock.json
    or yarn.lock
    ▰ This lock file should be committed
    ▰ Use same tool versions on all environment
    ▰ Use Docker

    View Slide

  28. FROM node:10.12-alpine
    ENV APP_WORKDIR=/usr/src/app
    RUN apk update && apk upgrade && \
    apk add --virtual build-deps git openssh-client py-pip make
    COPY package.json yarn.lock $APP_WORKDIR
    WORKDIR $APP_WORKDIR
    RUN yarn install
    COPY . $APP_WORKDIR
    RUN yarn build
    RUN rm -rf tsconfig.json src
    RUN apk del build-deps
    RUN yarn install --production
    RUN yarn cache clean
    EXPOSE 3000
    CMD ["node", "dist/index.js"]

    View Slide

  29. How to Configure Your Application?
    29
    ▰ Use .env.example file for description
    ▰ Use .env file for local development
    ▰ Use environment variables for any other
    environment
    ▰ Don’t use default config values in your code
    ▰ Break your start process if any value is missed
    ▰ Use dotenv-safe

    View Slide

  30. # .env.example
    # Default log level
    LOG_LEVEL=info
    # Graceful shutdown timeout in milliseconds
    SHUTDOWN_TIMEOUT=1000
    # HTTP Boundary port
    HTTP_PORT=8000
    # Redis connection string
    REDIS_URI=redis://127.0.0.1:6379

    View Slide

  31. require('dotenv-safe').load({
    allowEmptyValues: true,
    path: join(__dirname, '..', '.env'),
    sample: join(__dirname, '..', '.env.example')
    });
    const logger = require('./logger');
    logger.levels('stdout', process.env.LOG_LEVEL);
    const settings = {
    shutdownTimeout: process.env.SHUTDOWN_TIMEOUT,
    http: {
    port: process.env.HTTP_PORT,
    },
    redis: {
    uri: process.env.REDIS_URI,
    },
    };
    module.exports = settings;

    View Slide

  32. 32
    Reliability

    View Slide

  33. “Reliability is about a
    software product that does
    what it's supposed to do,
    dependably.
    33
    33

    View Slide

  34. What Should Your Application Do?
    34
    It depends, but your Node.js app should:
    ▰ start
    ▰ stop
    ▰ write logs, good packages for that: pino or bunyan
    ▰ handle errors, for example degrade functionality on
    problem with 3rd parties
    ▰ restart on issues. pm2 or Docker can be used for this.

    View Slide

  35. How to Stop Your Node.js Application?
    35
    1. Add hooks on signals
    2. Log start of graceful shutdown
    3. Close incoming business logic flow
    4. Set a forced timeout
    5. Notify consumers about shutdown
    6. Correctly disconnect from all connections or
    mark them unref
    7. Clean all timeout and intervals
    8. Node.js will finish work

    View Slide

  36. const server = app.listen(settings.port, () => {
    logger.info(`Server started on port ${settings.port}`);
    });
    function stopHandler(){
    logger.info(`Stopping server on port ${settings.port}`);
    const timeout = setTimeout(() => {
    logger.info(`Server on port ${settings.port} stop forcefully`);
    process.exit(1);
    }, settings.stopTimeout);
    server.close(() => {
    logger.info(`Server on port ${settings.port} stopped`);
    clearTimeout(timeout);
    });
    };
    process.on('exit', (code) => logger.info(`Exit with code: ${code}`));
    ['SIGTERM','SIGINT','SIGHUP']
    .forEach(signal => process.on(signal, stopHandler);

    View Slide

  37. Error handling
    37
    ▰ Operational errors should be handled
    ▰ Programmer errors cannot be handled,
    application should be stopped
    ▰ Always use instance of Error Object at throw,
    error event, Promise reject or callback
    ▰ More

    View Slide

  38. import './config';
    import logger from './logger';
    process.on('unhandledRejection', (reason) => {
    logger.fatal({ error: reason }, 'Unhandled Rejection');
    process.exit(1);
    });
    process.on('uncaughtException', (error) => {
    logger.fatal(error, 'Unhandled Exception');
    process.exit(1);
    });
    process.on('warning', (warning) => {
    logger.warn(warning, 'Warning detected');
    });

    View Slide

  39. 39
    Efficiency

    View Slide

  40. “Efficiency is about a software
    product that economizes on
    both running time and space
    consumption.
    40
    40

    View Slide

  41. Node.js efficiency factors
    41
    ▰ Memory usage
    ▰ CPU usage
    ▰ Connections count
    ▰ Request per second
    ▰ Response time
    ▰ etc

    View Slide

  42. Node.js efficiency advices
    42
    ▰ Reuse connections
    ▰ Use Promise.all() in async functions
    ▰ Make calculation in DB layer
    ▰ Use precalculated data
    ▰ Use right protocol
    ▻ REST
    ▻ Server-Side-Events
    ▻ WS
    ▻ TCP

    View Slide

  43. const balancesP = getBalances(userAddr, tokenAddresses);
    const metricsP = getTokenMetricsObject();
    const pricesP = coinmarketcap.get();
    const [balances, metrics, prices] = await
    Promise.all([balancesP, metricsP, pricesP]);
    // 1-3 seconds
    const balances = await getBalances(userAddr,
    tokenAddresses);
    const metrics = await getTokenMetricsObject();
    const prices = await coinmarketcap.get();
    // 3-9 seconds

    View Slide

  44. router.get('/someFile',
    function(req, res, next) {
    fs.createReadStream("./toSomeFile")
    .pipe(res);
    });
    router.get('/users,
    function(req, res, next) {
    users.find({})
    .pipe(res);
    });

    View Slide

  45. 45
    Usability

    View Slide

  46. “Usability is about a software
    product that is easy and
    comfortable to use.
    46
    46

    View Slide

  47. Node.js Application Usability
    47
    Node.js Usability factors:
    ▰ response time
    ▰ response time
    ▰ and again response time
    What affects the response time:
    ▰ network latency
    ▰ Node.js event loop delay
    ▰ application load (so be ready to
    scale)
    ▰ processing type (use streams
    when they are required)
    ▰ 3rd parties like DB/cache/etc

    View Slide

  48. View Slide

  49. ▰ Use sync operations instead of async
    ▰ Make too many CPU usage in one process
    ▰ Use something like:
    while (Date.now() < end)
    ▰ Use long running event handlers without
    process.nextTick()
    How to Block Event Loop? DON’T DO IT
    49

    View Slide

  50. let last = process.hrtime();
    const CHECK_DELAY_EVERY_SECOND = 1;
    setInterval(() => {
    const time = process.hrtime(last);
    let delay = (time[0] * 1e3 + time[1] / 1e6);
    delay = delay / CHECK_DELAY_EVERY_SECOND - 1000;
    console.log(delay.toFixed(2));
    last = process.hrtime();
    }, CHECK_DELAY_EVERY_SECOND * 1000);

    View Slide

  51. 51
    Testability

    View Slide

  52. “Testability is about a
    software product that is
    easy to test.
    52
    52

    View Slide

  53. Test layers
    53
    ▰ Unit tests
    ▰ Integration tests (mocha, jest, etc)
    ▰ REST API tests (abao, dredd)
    ▰ e2e functional tests (QA specific)
    ▰ performance tests (QA specific)

    View Slide

  54. const app = require('./../src/app');
    const request = require('supertest');
    describe('GET /ping', function (){
    it('should return status 200', function (done) {
    request(app)
    .get('/ping')
    .expect('Content-Type', /json/)
    .expect(200, {response: 'success'}, done);
    });
    });

    View Slide

  55. describe('CLI', function () {
    let messageCallback, errorCallback, closeCallback;
    function startCLI(commandOptions) {
    let childProcess = require('child_process')
    .spawn('path/to/cli', commandOptions);
    childProcess.stdout.on('data', messageCallback);
    childProcess.stderr.on('data', errorCallback);
    childProcess.on('close', closeCallback);
    return childProcess;
    }
    it('should stopping gracefully', function(done) {
    closeCallback = (code) => {
    expect(code).to.equal(0);
    done();
    }
    errorCallback = done;
    let cli = startCLI([`-f=example`]);
    setTimeout(() => cli.kill('SIGTERM'), 1000);
    })
    })

    View Slide

  56. 56
    What is quality
    assurance?

    View Slide

  57. Expectations Reality
    VS

    View Slide

  58. 58
    THANKS!
    HAPPY CODING
    WITH NODE.JS
    You can find me on Twitter as @galk_in
    Slides are available at speakerdeck.com/galkin
    or at my site galk.in

    View Slide