Painless data operation in Node.js

Painless data operation in Node.js

A2443b68836f2d166eae52b940e99bf6?s=128

Nikita Galkin

June 09, 2018
Tweet

Transcript

  1. Painless data operation in Node.js at Jun 9, 2018

  2. Nikita Galkin Love and Know: ▰ How to make developers

    and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 2
  3. None
  4. None
  5. None
  6. null undefined

  7. router.get('/file', function(req, res, next) { models.users.findById(req.params.id) .then(function(usr) { if (!usr)

    { return res.status(404).json({ msg: 'User not found' }); } return res.json(usr); }) .catch(next); });
  8. Big Response time

  9. 9 BE usability factors: ▰ response time ▰ response time

    ▰ and again response time
  10. 10 What affects the response time: ▰ network latency ▰

    Node.js event loop delay ▰ Application load (so be ready to scale) ▰ Bad data flow
  11. 11 Block Event loop

  12. 12 How block Event Loop (don’t do it): ▰ Use

    sync operations instead of async ▰ Make too many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end)
  13. None
  14. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() =>

    { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
  15. 15 How improve data flow: ▰ Use streams ▰ Use

    Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data
  16. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const

    pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
  17. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile").pipe(res); }); router.get('/users, function(req, res,

    next) { users.find({}).pipe(res); });
  18. CREATE MATERIALIZED VIEW "token_metrics" AS SELECT ... CREATE OR REPLACE

    FUNCTION refresh_metrics () RETURNS TRIGGER LANGUAGE plpgsql AS $$ BEGIN REFRESH MATERIALIZED VIEW token_metrics; RETURN NULL; END $$; CREATE TRIGGER trades_updated AFTER INSERT OR UPDATE OR DELETE ON trades EXECUTE PROCEDURE refresh_metrics();
  19. Event Loop is Okey

  20. DataBase is overloaded

  21. 21 How to fix DataBase performance: ▰ Use explain and

    logs ▰ Add indexes ▰ Add materialized views ▰ Add extra data storage as cache ▰ Scale!
  22. Entity vs Object

  23. 23

  24. 24 Plain Old Java Object

  25. class User { constructor(name, surname) { this.name = name; this.surname

    = surname; } } const hero = new User('Bart', 'Simpson'); hero instanceof User; // true JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}
  26. // class (constructor) objects new User('Bart', 'Simpson');

  27. class User { constructor(name, surname) { this.name = name; this.surname

    = surname; } } const hero = { name: 'Gomer', surname: 'Simpson' }; hero instanceof User; // false
  28. // plain (literal) object const hero = { name: 'Gomer',

    surname: 'Simpson' };
  29. 29 Don't use classes instances and plain Objects together

  30. 30 1. Use factories 2. Use class-transformer 3. Carefully use

    ORM, for example: sequelizejs typeorm
  31. Active Record VS Data Mapper

  32. const User = sequelize.define('user', { username: Sequelize.STRING, surname: Sequelize.STRING });

    // Active record User.findAll({where: {age: {Sequelize.Op.gte: 21 } }); // Repository UserRepository.getAllAdult();
  33. 33

  34. REST and WS

  35. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } toJSON() { return { name: this.name, surname: this.surname }; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}
  36. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } toWS() { return { name: this.name }; } } const hero = new User('Bart', 'Simpson'); JSON.stringify(hero.toWS()); // {"name":"Bart"}
  37. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero, ['name', 'surname']); // can be function
  38. GraphQL

  39. import { Field, ObjectType } from 'type-graphql'; import { Column,

    Entity, ObjectIdColumn } from 'typeorm'; @Entity({ name: 'verbs' }) @ObjectType({ description: 'Object representing Verb' }) export class Verb { @ObjectIdColumn() @Field() id: string; @Column() @Field(type => String) name: string; }
  40. Test Driven Development 40 FAIL PASS TDD REFACTORING “Repetition of

    a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only.” Kent Beck
  41. Types or Documentation Driven Development 41 tsc Add JSDoc TDD

    ▰ choose typescript or JSDoc ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example REFACTORING
  42. /** * @typedef {Object} CoverageEntry * @property {string} url *

    @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
  43. Theft Driven Development 43 STEAL ADOPT TDD SHARE

  44. I hope you stole some useful ideas from my talk

  45. 45 Thank you for attention! Be consistent in your node.js

    data!!! You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or on my site galk.in