Painless data operation in Node.js

Transcript

1. Painless data operation in Node.js at Jun 9, 2018

2. Nikita Galkin Love and Know: ▰ How to make developers

   and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 2
3. None
4. None
5. None

6. null undefined

7. router.get('/file', function(req, res, next) { models.users.findById(req.params.id) .then(function(usr) { if (!usr)

   { return res.status(404).json({ msg: 'User not found' }); } return res.json(usr); }) .catch(next); });

8. Big Response time

9. 9 BE usability factors: ▰ response time ▰ response time

   ▰ and again response time

10. 10 What affects the response time: ▰ network latency ▰

    Node.js event loop delay ▰ Application load (so be ready to scale) ▰ Bad data flow

11. 11 Block Event loop

12. 12 How block Event Loop (don’t do it): ▰ Use

    sync operations instead of async ▰ Make too many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end)
13. None

14. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() =>

    { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);

15. 15 How improve data flow: ▰ Use streams ▰ Use

    Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data

16. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const

    pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds

17. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile").pipe(res); }); router.get('/users, function(req, res,

    next) { users.find({}).pipe(res); });

18. CREATE MATERIALIZED VIEW "token_metrics" AS SELECT ... CREATE OR REPLACE

    FUNCTION refresh_metrics () RETURNS TRIGGER LANGUAGE plpgsql AS $$ BEGIN REFRESH MATERIALIZED VIEW token_metrics; RETURN NULL; END $$; CREATE TRIGGER trades_updated AFTER INSERT OR UPDATE OR DELETE ON trades EXECUTE PROCEDURE refresh_metrics();

19. Event Loop is Okey

20. DataBase is overloaded

21. 21 How to fix DataBase performance: ▰ Use explain and

    logs ▰ Add indexes ▰ Add materialized views ▰ Add extra data storage as cache ▰ Scale!

22. Entity vs Object

23. 23

24. 24 Plain Old Java Object

25. class User { constructor(name, surname) { this.name = name; this.surname

    = surname; } } const hero = new User('Bart', 'Simpson'); hero instanceof User; // true JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}

26. // class (constructor) objects new User('Bart', 'Simpson');

27. class User { constructor(name, surname) { this.name = name; this.surname

    = surname; } } const hero = { name: 'Gomer', surname: 'Simpson' }; hero instanceof User; // false

28. // plain (literal) object const hero = { name: 'Gomer',

    surname: 'Simpson' };

29. 29 Don't use classes instances and plain Objects together

30. 30 1. Use factories 2. Use class-transformer 3. Carefully use

    ORM, for example: sequelizejs typeorm

31. Active Record VS Data Mapper

32. const User = sequelize.define('user', { username: Sequelize.STRING, surname: Sequelize.STRING });

    // Active record User.findAll({where: {age: {Sequelize.Op.gte: 21 } }); // Repository UserRepository.getAllAdult();

33. 33

34. REST and WS

35. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } toJSON() { return { name: this.name, surname: this.surname }; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}

36. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } toWS() { return { name: this.name }; } } const hero = new User('Bart', 'Simpson'); JSON.stringify(hero.toWS()); // {"name":"Bart"}

37. class User { constructor(name, surname, hidden) { this.name = name;

    this.surname = surname; this.hidden = hidden; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero, ['name', 'surname']); // can be function

38. GraphQL

39. import { Field, ObjectType } from 'type-graphql'; import { Column,

    Entity, ObjectIdColumn } from 'typeorm'; @Entity({ name: 'verbs' }) @ObjectType({ description: 'Object representing Verb' }) export class Verb { @ObjectIdColumn() @Field() id: string; @Column() @Field(type => String) name: string; }

40. Test Driven Development 40 FAIL PASS TDD REFACTORING “Repetition of

    a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only.” Kent Beck

41. Types or Documentation Driven Development 41 tsc Add JSDoc TDD

    ▰ choose typescript or JSDoc ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example REFACTORING

42. /** * @typedef {Object} CoverageEntry * @property {string} url *

    @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */

43. Theft Driven Development 43 STEAL ADOPT TDD SHARE

44. I hope you stole some useful ideas from my talk

45. 45 Thank you for attention! Be consistent in your node.js

    data!!! You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or on my site galk.in