Painless data operation in Node.js

Transcript

1. Painless data operation in Node.js at Jun 9, 2018
2. Nikita Galkin Love and Know: ▰ How to make developers and business happy ▰ Technical and process debt elimination Believe that: ▰ Any problem must be solved at the right level ▰ Software is easy. People are hard ▰ A problem should be highlighted, an idea should be "sold", a solution should be demonstrated Links: Site GitHub Twitter Facebook 2
6. null undefined
7. router.get('/file', function(req, res, next) { models.users.findById(req.params.id) .then(function(usr) { if (!usr) { return res.status(404).json({ msg: 'User not found' }); } return res.json(usr); }) .catch(next); });
8. Big Response time
9. 9 BE usability factors: ▰ response time ▰ response time ▰ and again response time
10. 10 What affects the response time: ▰ network latency ▰ Node.js event loop delay ▰ Application load (so be ready to scale) ▰ Bad data flow
11. 11 Block Event loop
12. 12 How block Event Loop (don’t do it): ▰ Use sync operations instead of async ▰ Make too many CPU usage manipulation in same process ▰ Use something like: while (Date.now() < end)
14. let last = process.hrtime(); const CHECK_DELAY_EVERY_SECOND = 1; setInterval(() => { const time = process.hrtime(last); let delay = (time[0] * 1e3 + time[1] / 1e6); delay = delay / CHECK_DELAY_EVERY_SECOND - 1000; console.log(delay.toFixed(2)); last = process.hrtime(); }, CHECK_DELAY_EVERY_SECOND * 1000);
15. 15 How improve data flow: ▰ Use streams ▰ Use Promise.all() in async functions ▰ Make calculation in DB layer ▰ Use precalculated data
16. const balancesP = getBalances(userAddr, tokenAddresses); const metricsP = getTokenMetricsObject(); const pricesP = coinmarketcap.get(); const [balances, metrics, prices] = await Promise.all([balancesP, metricsP, pricesP]); // 1-3 seconds const balances = await getBalances(userAddr, tokenAddresses); const metrics = await getTokenMetricsObject(); const prices = await coinmarketcap.get(); // 3-9 seconds
17. router.get('/someFile', function(req, res, next) { fs.createReadStream("./toSomeFile").pipe(res); }); router.get('/users, function(req, res, next) { users.find({}).pipe(res); });
18. CREATE MATERIALIZED VIEW "token_metrics" AS SELECT ... CREATE OR REPLACE FUNCTION refresh_metrics () RETURNS TRIGGER LANGUAGE plpgsql AS $$ BEGIN REFRESH MATERIALIZED VIEW token_metrics; RETURN NULL; END $$; CREATE TRIGGER trades_updated AFTER INSERT OR UPDATE OR DELETE ON trades EXECUTE PROCEDURE refresh_metrics();
19. Event Loop is Okey
20. DataBase is overloaded
21. 21 How to fix DataBase performance: ▰ Use explain and logs ▰ Add indexes ▰ Add materialized views ▰ Add extra data storage as cache ▰ Scale!
22. Entity vs Object
24. 24 Plain Old Java Object
25. class User { constructor(name, surname) { this.name = name; this.surname = surname; } } const hero = new User('Bart', 'Simpson'); hero instanceof User; // true JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}
26. // class (constructor) objects new User('Bart', 'Simpson');
27. class User { constructor(name, surname) { this.name = name; this.surname = surname; } } const hero = { name: 'Gomer', surname: 'Simpson' }; hero instanceof User; // false
28. // plain (literal) object const hero = { name: 'Gomer', surname: 'Simpson' };
29. 29 Don't use classes instances and plain Objects together
30. 30 1. Use factories 2. Use class-transformer 3. Carefully use ORM, for example: sequelizejs typeorm
31. Active Record VS Data Mapper
32. const User = sequelize.define('user', { username: Sequelize.STRING, surname: Sequelize.STRING }); // Active record User.findAll({where: {age: {Sequelize.Op.gte: 21 } }); // Repository UserRepository.getAllAdult();
34. REST and WS
35. class User { constructor(name, surname, hidden) { this.name = name; this.surname = surname; this.hidden = hidden; } toJSON() { return { name: this.name, surname: this.surname }; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero); // {"name":"Bart","surname":"Simpson"}
36. class User { constructor(name, surname, hidden) { this.name = name; this.surname = surname; this.hidden = hidden; } toWS() { return { name: this.name }; } } const hero = new User('Bart', 'Simpson'); JSON.stringify(hero.toWS()); // {"name":"Bart"}
37. class User { constructor(name, surname, hidden) { this.name = name; this.surname = surname; this.hidden = hidden; } } const hero = new User('Bart', 'Simpson', 'iLoveDaddy'); JSON.stringify(hero, ['name', 'surname']); // can be function
38. GraphQL
39. import { Field, ObjectType } from 'type-graphql'; import { Column, Entity, ObjectIdColumn } from 'typeorm'; @Entity({ name: 'verbs' }) @ObjectType({ description: 'Object representing Verb' }) export class Verb { @ObjectIdColumn() @Field() id: string; @Column() @Field(type => String) name: string; }
40. Test Driven Development 40 FAIL PASS TDD REFACTORING “Repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only.” Kent Beck
41. Types or Documentation Driven Development 41 tsc Add JSDoc TDD ▰ choose typescript or JSDoc ▰ autocomplete for engineers ▰ always actual code documentation ▰ puppeteer source-code is awesome JSDoc example REFACTORING
42. /** * @typedef {Object} CoverageEntry * @property {string} url * @property {string} text * @property {!Array<!{start: number, end: number}>} ranges */
43. Theft Driven Development 43 STEAL ADOPT TDD SHARE
44. I hope you stole some useful ideas from my talk
45. 45 Thank you for attention! Be consistent in your node.js data!!! You can find me on Twitter as @galk_in Slides are available at speakerdeck.com/galkin or on my site galk.in