How to make high quality Node.js application at JSDevTalk

Transcript

1. How to Make a High
   Quality Node.js
   Application
   by Nikita Galkin
   Oct 28, 2018
   

   View Slide

2. 2
   “I picked Java because I felt the
   most people would be able to
   understand the code examples if
   they were written in Java. That
   was the case in 1997, but how
   about in 2017?
   …
   But I went for the alternative:
   JavaScript.”
   27 March 2018
   

   View Slide

3. 3
   Take notes and todos
   Ask questions
   Share ideas
   Keep calm and validate
   

   View Slide

4. Nikita
   Galkin
   Love and Know:
   ▰ How to make developers and business happy
   ▰ Technical and process debt elimination
   Believe that:
   ▰ Any problem must be solved at the right level
   ▰ Software is easy. People are hard
   ▰ A problem should be highlighted, an idea should
   be "sold", a solution should be demonstrated
   Links:
   Site GitHub Twitter Facebook
   4
   

   View Slide

5. 5
   What is Quality?
   

   View Slide

6. What is Quality?
   6
   Attributes:
   ▰ Portability
   ▰ Reliability
   ▰ Efficiency
   ▰ Usability
   ▰ Testability
   ▰ Understandability
   ▰ Modifiability
   

   View Slide

7. What is Quality?
   7
   ▰ Portability – DevOps Engineer
   ▰ Reliability – Product Owner
   ▰ Efficiency – System Architect
   ▰ Usability – UX/UI Designer
   ▰ Testability – QA Engineer
   ▰ Understandability – Developer
   ▰ Modifiability – Developer
   

   View Slide

8. 8
   Understandability
   

   View Slide

9. “Understandability is about
   a software product that is
   easy for a maintainer to
   comprehend.
   9
   9
   

   View Slide

10. Code style
    10
    ▰ Eslint (tslint) is your Wunderwaffe
    ▻ tslint -p tsconfig.json --fix
    ▻ eslint . --fix
    ▻ WebStorm can set style from eslint file
    ▰ If you don’t lint your code, then start with eslint helper
    ▰ More at Code Style and Violence by Anton Nemtsev
    

    View Slide

11. Node.js Async Programming Paradigms
    11
    ▰ callbacks
    ▰ usage async package
    ▰ promises
    ▰ generators
    ▰ async/await
    Please don’t use several of them in one project
    

    View Slide

12. makeRequest('http://server.com/path/request',
    function step1(req, resp) {
    processData(resp, function step2(err, processedData) {
    saveData(processedData, function step3(err, model) {
    loadRelatedData(model, function step4(err, relData) {
    model.rel = relData;
    sendResponse(model);
    });
    });
    });
    });
    

    View Slide

13. makeRequest('http://server.com/path/request')
    .then(processData)
    .then(saveData)
    .then(loadRelatedData)
    .then((relData) => {
    model.rel = relData;
    sendResponse(model);
    });
    

    View Slide

14. async function example() {
    const url = 'http://server.com/path/request';
    const resp = await makeRequest(url);
    const processedData = await processData(resp);
    const model = await saveData(processedData);
    model.rel = await loadRelatedData(model);
    await sendResponse(model);
    }
    

    View Slide

15. Specify the types
    15
    Choose typescript or JSDoc. Benefits:
    ▰ autocomplete for engineers
    ▰ always actual code documentation
    ▰ puppeteer source-code is awesome JSDoc
    example with autotype-checking
    

    View Slide

16. /**
    * @typedef {Object} CoverageEntry
    * @property {string} url
    * @property {string} text
    * @property {!Array} ranges
    */
    

    View Slide

17. 17
    Modifiability
    

    View Slide

18. “Modifiability is about a
    software product that is
    easy for a maintainer to
    change.
    18
    18
    

    View Slide

19. How Can You Increase Modifiability?
    19
    ▰ Be consistent!
    ▰ Keep your readme.md up-to-date
    ▰ Use generators
    ▻ plop
    ▻ Yeoman.io
    ▰ Remove unused code and dependencies
    ▻ depcheck
    

    View Slide

20. Readme structure
    20
    ▰ Overview
    ▰ Development
    ▻ Tooling
    ▻ Project structure
    ▰ Configuration
    ▰ Run and Deploy
    ▰ Testing approach
    

    View Slide

21. Generators
    21
    

    View Slide

22. 22
    Portability
    

    View Slide

23. “Portability is about creating a
    software product that is
    easily moved to another
    platform.
    23
    23
    

    View Slide

24. What is Quality?
    24
    Do you know:
    ▰ How do you release?
    ▰ How do you configure?
    ▰ How do you version?
    ▰ How do you scale?
    Your DevOps knows.
    

    View Slide

25. 12factor.net
    

    View Slide

26. 26
    Build
    Release
    Run
    

    View Slide

27. How to be Repeatable?
    27
    ▰ Explicitly declare and isolate dependencies
    ▰ Lock your dependencies package-lock.json
    or yarn.lock
    ▰ This lock file should be committed
    ▰ Use same tool versions on all environment
    ▰ Use Docker
    

    View Slide

28. FROM node:10.12-alpine
    ENV APP_WORKDIR=/usr/src/app
    RUN apk update && apk upgrade && \
    apk add --virtual build-deps git openssh-client py-pip make
    COPY package.json yarn.lock $APP_WORKDIR
    WORKDIR $APP_WORKDIR
    RUN yarn install
    COPY . $APP_WORKDIR
    RUN yarn build
    RUN rm -rf tsconfig.json src
    RUN apk del build-deps
    RUN yarn install --production
    RUN yarn cache clean
    EXPOSE 3000
    CMD ["node", "dist/index.js"]
    

    View Slide

29. How to Configure Your Application?
    29
    ▰ Use .env.example file for description
    ▰ Use .env file for local development
    ▰ Use environment variables for any other
    environment
    ▰ Don’t use default config values in your code
    ▰ Break your start process if any value is missed
    ▰ Use dotenv-safe
    

    View Slide

30. # .env.example
    # Default log level
    LOG_LEVEL=info
    # Graceful shutdown timeout in milliseconds
    SHUTDOWN_TIMEOUT=1000
    # HTTP Boundary port
    HTTP_PORT=8000
    # Redis connection string
    REDIS_URI=redis://127.0.0.1:6379
    

    View Slide

31. require('dotenv-safe').load({
    allowEmptyValues: true,
    path: join(__dirname, '..', '.env'),
    sample: join(__dirname, '..', '.env.example')
    });
    const logger = require('./logger');
    logger.levels('stdout', process.env.LOG_LEVEL);
    const settings = {
    shutdownTimeout: process.env.SHUTDOWN_TIMEOUT,
    http: {
    port: process.env.HTTP_PORT,
    },
    redis: {
    uri: process.env.REDIS_URI,
    },
    };
    module.exports = settings;
    

    View Slide

32. 32
    Reliability
    

    View Slide

33. “Reliability is about a
    software product that does
    what it's supposed to do,
    dependably.
    33
    33
    

    View Slide

34. What Should Your Application Do?
    34
    It depends, but your Node.js app should:
    ▰ start
    ▰ stop
    ▰ write logs, good packages for that: pino or bunyan
    ▰ handle errors, for example degrade functionality on
    problem with 3rd parties
    ▰ restart on issues. pm2 or Docker can be used for this.
    

    View Slide

35. How to Stop Your Node.js Application?
    35
    1. Add hooks on signals
    2. Log start of graceful shutdown
    3. Close incoming business logic flow
    4. Set a forced timeout
    5. Notify consumers about shutdown
    6. Correctly disconnect from all connections or
    mark them unref
    7. Clean all timeout and intervals
    8. Node.js will finish work
    

    View Slide

36. const server = app.listen(settings.port, () => {
    logger.info(`Server started on port ${settings.port}`);
    });
    function stopHandler(){
    logger.info(`Stopping server on port ${settings.port}`);
    const timeout = setTimeout(() => {
    logger.info(`Server on port ${settings.port} stop forcefully`);
    process.exit(1);
    }, settings.stopTimeout);
    server.close(() => {
    logger.info(`Server on port ${settings.port} stopped`);
    clearTimeout(timeout);
    });
    };
    process.on('exit', (code) => logger.info(`Exit with code: ${code}`));
    ['SIGTERM','SIGINT','SIGHUP']
    .forEach(signal => process.on(signal, stopHandler);
    

    View Slide

37. Error handling
    37
    ▰ Operational errors should be handled
    ▰ Programmer errors cannot be handled,
    application should be stopped
    ▰ Always use instance of Error Object at throw,
    error event, Promise reject or callback
    ▰ More
    

    View Slide

38. import './config';
    import logger from './logger';
    process.on('unhandledRejection', (reason) => {
    logger.fatal({ error: reason }, 'Unhandled Rejection');
    process.exit(1);
    });
    process.on('uncaughtException', (error) => {
    logger.fatal(error, 'Unhandled Exception');
    process.exit(1);
    });
    process.on('warning', (warning) => {
    logger.warn(warning, 'Warning detected');
    });
    

    View Slide

39. 39
    Efficiency
    

    View Slide

40. “Efficiency is about a software
    product that economizes on
    both running time and space
    consumption.
    40
    40
    

    View Slide

41. Node.js efficiency factors
    41
    ▰ Memory usage
    ▰ CPU usage
    ▰ Connections count
    ▰ Request per second
    ▰ Response time
    ▰ etc
    

    View Slide

42. Node.js efficiency advices
    42
    ▰ Reuse connections
    ▰ Use Promise.all() in async functions
    ▰ Make calculation in DB layer
    ▰ Use precalculated data
    ▰ Use right protocol
    ▻ REST
    ▻ Server-Side-Events
    ▻ WS
    ▻ TCP
    

    View Slide

43. const balancesP = getBalances(userAddr, tokenAddresses);
    const metricsP = getTokenMetricsObject();
    const pricesP = coinmarketcap.get();
    const [balances, metrics, prices] = await
    Promise.all([balancesP, metricsP, pricesP]);
    // 1-3 seconds
    const balances = await getBalances(userAddr,
    tokenAddresses);
    const metrics = await getTokenMetricsObject();
    const prices = await coinmarketcap.get();
    // 3-9 seconds
    

    View Slide

44. router.get('/someFile',
    function(req, res, next) {
    fs.createReadStream("./toSomeFile")
    .pipe(res);
    });
    router.get('/users,
    function(req, res, next) {
    users.find({})
    .pipe(res);
    });
    

    View Slide

45. 45
    Usability
    

    View Slide

46. “Usability is about a software
    product that is easy and
    comfortable to use.
    46
    46
    

    View Slide

47. Node.js Application Usability
    47
    Node.js Usability factors:
    ▰ response time
    ▰ response time
    ▰ and again response time
    What affects the response time:
    ▰ network latency
    ▰ Node.js event loop delay
    ▰ application load (so be ready to
    scale)
    ▰ processing type (use streams
    when they are required)
    ▰ 3rd parties like DB/cache/etc
    

    View Slide

48. View Slide

49. ▰ Use sync operations instead of async
    ▰ Make too many CPU usage in one process
    ▰ Use something like:
    while (Date.now() < end)
    ▰ Use long running event handlers without
    process.nextTick()
    How to Block Event Loop? DON’T DO IT
    49
    

    View Slide

50. let last = process.hrtime();
    const CHECK_DELAY_EVERY_SECOND = 1;
    setInterval(() => {
    const time = process.hrtime(last);
    let delay = (time[0] * 1e3 + time[1] / 1e6);
    delay = delay / CHECK_DELAY_EVERY_SECOND - 1000;
    console.log(delay.toFixed(2));
    last = process.hrtime();
    }, CHECK_DELAY_EVERY_SECOND * 1000);
    

    View Slide

51. 51
    Testability
    

    View Slide

52. “Testability is about a
    software product that is
    easy to test.
    52
    52
    

    View Slide

53. Test layers
    53
    ▰ Unit tests
    ▰ Integration tests (mocha, jest, etc)
    ▰ REST API tests (abao, dredd)
    ▰ e2e functional tests (QA specific)
    ▰ performance tests (QA specific)
    

    View Slide

54. const app = require('./../src/app');
    const request = require('supertest');
    describe('GET /ping', function (){
    it('should return status 200', function (done) {
    request(app)
    .get('/ping')
    .expect('Content-Type', /json/)
    .expect(200, {response: 'success'}, done);
    });
    });
    

    View Slide

55. describe('CLI', function () {
    let messageCallback, errorCallback, closeCallback;
    function startCLI(commandOptions) {
    let childProcess = require('child_process')
    .spawn('path/to/cli', commandOptions);
    childProcess.stdout.on('data', messageCallback);
    childProcess.stderr.on('data', errorCallback);
    childProcess.on('close', closeCallback);
    return childProcess;
    }
    it('should stopping gracefully', function(done) {
    closeCallback = (code) => {
    expect(code).to.equal(0);
    done();
    }
    errorCallback = done;
    let cli = startCLI([`-f=example`]);
    setTimeout(() => cli.kill('SIGTERM'), 1000);
    })
    })
    

    View Slide

56. 56
    What is quality
    assurance?
    

    View Slide

57. Expectations Reality
    VS
    

    View Slide

58. 58
    THANKS!
    HAPPY CODING
    WITH NODE.JS
    You can find me on Twitter as @galk_in
    Slides are available at speakerdeck.com/galkin
    or at my site galk.in
    

    View Slide