Milliseconds to Millions - Benchmarking & Optimization for Fun and Profit!! @ FullStack 2017

The past 10+ years of affordable and easy to scale cloud services & platforms has dulled our collective edge when it comes to matters of performance optimization and benchmarking our software creations.

While computing power goes higher costs go lower, and along with that, power and resource hungry software platforms, libraries and practices are becoming more common!

When was the last time you benchmarked the library you are using? how about all it's dependencies? come to think of it, when was the last time you inspected your dependency tree for inefficiencies? How deep does that rabbit hole go?

In this talk, I will showcase:

- How to accurately benchmark & measure your code
- Where to start in identifying bottlenecks inside and outside your project
- How to contribute your findings back to the community to improve our libraries ecosystem

Transcript

1. MILLISECONDS TO MILLIONS! Benchmarking & Optimization for fun and profit!!!
2. HELLO WORLD! SYRIAN-CANADIAN, ENTREPRENEUR, HACKER, DEVELOPER, OPEN SOURCE ADVOCATE & DOG LOVER @AhmadNassri http://ahmad.codes Principal Architect at TELUS Digital, Founder at Bench CI, Founder at Tech Masters, Mentor at Node School Toronto, Board Member at Full Stack Toronto, Organizer at Functions Conf, Editor at The RESTful Web. Tech Outlaw, Wanted by a third-world dictator (true story)
3. AGENDA What is benchmarking? Why should you care? When & Where to start How!? Q&A OUTCOME Get excited about performance & optimization Learn about the tools, tricks and Gotchas! Benchmark all the things!!! Save your company $$$
4. THE WHAT
5. PERFORMANCE! ... MAYBE?
6. PERFORMANCE ON THE FRONT END RAIL: A User-Centric Model For Performance
7. PERFORMANCE ENGINEERING Response time The amount of time that it takes for a server to respond to a request Throughput The number of requests that can be served by your application per unit time Resource utilization CPU, memory, disk I/O, and network I/O Workload The total number of users and concurrent active users, data volumes, and transaction volumes
8. WHY SHOULD YOU CARE?
9. BANK SCENARIO Consider a system that processes CAD GBP transactions Throughput of 1 Billion CAD GBP worth of transactions per minute You are the best coder in the world, but you're not perfect Your error rate is 0.1% A single bug could cost the company millions, but it's detectable Performance degradation could cost billions! Not easily detectable
10. WHY SHOULD YOU CARE?
11. WHY SHOULD YOU CARE?
12. WHY SHOULD YOU CARE? MEASURE PERFORMANCE Code quality is not enough, keep track of performance and efficient resource usage. REDUCE RESOURCE CONSUMPTION Memory capacity & CPU cycles are precious in resource constrained environments SHIP WITH CONFIDENCE Ensure your software operates on and within any hardware & resource constraints.
13. WHEN & WHERE
14. WHEN & WHERE TO MEASURE PERFORMANCE?
15. MOST COMMON...
17. BENCHMARKING & PROFILING DO IT AS EARLY AS POSSIBLE
18. OK, HOW? FINALLY!
19. BENCHMARKING IT'S EASY! Discover optimizations that could impact the largest number of users
20. $ time for a in {1..10}; do node index.js; done real 0m13.042s user 0m0.021s sys 0m0.044s IT'S SUPER EASY! real: time from start to finish. (all elapsed time including time slices used by other processes + process block) user: actual CPU time used in executing the process sys: CPU time spent in system calls (within the kernel)
21. FUN IN THE BROWSER
22. LET'S GO DEEPER! BENCHMARK.JS https://benchmarkjs.com
23. new Benchmark.Suite() .add({ name: 'JSON.parse', fn: () => JSON.parse(json) }) .add({ name: 'jju', fn: () => jju.parse(json) }) ... .add({ name: 'vuvuzela', fn: () => vuvuzela.parse(json) }) .run() BENCHMARK.JS clarinet x 2,636 ops/sec ±2.13% (78 runs sampled) JSON.parse x 80,639 ops/sec ±2.14% (88 runs sampled) jju x 7,343 ops/sec ±3.12% (84 runs sampled) jsonparse x 10,719 ops/sec ±4.54% (45 runs sampled) json-parse-stream x 2,039 ops/sec ±6.39% (69 runs sampled) stream-json x 1,775 ops/sec ±1.14% (82 runs sampled) vuvuzela x 15,644 ops/sec ±4.07% (87 runs sampled)
24. PROFILING! OKAY, THIS ONE IS NOT AS EASY
25. BROWSER: BUILT IN PROFILER!
26. app.get('/auth', function (req, res) { const username = req.query.username || '' const password = req.query.password || '' username = username.replace(/[!@#$%^&*]/g, '') if (!username || !password || !users[username]) { return res.sendStatus(400) } const hash = crypto.pbkdf2Sync(password, users[username].salt, 10000, 512) if (users[username].hash.toString() === hash.toString()) { res.sendStatus(200) } else { res.sendStatus(401) } }) NODE.JS PROFILER
27. [Summary]: ticks total nonlib name 79 0.2% 0.2% JavaScript 36703 97.2% 99.2% C++ 7 0.0% 0.0% GC 767 2.0% Shared libraries 215 0.6% Unaccounted NODE.JS PROFILER $ node --prof ./app.js $ node --prof-process ./log-file [C++]: ticks total nonlib name 19557 51.8% 52.9% node::crypto::PBKDF2(v8::FunctionCallbackInfo<v8::Value> const&) 4510 11.9% 12.2% _sha1_block_data_order 3165 8.4% 8.6% _malloc_zone_malloc
28. [Bottom up (heavy) profile] ticks parent name 19557 51.8% node::crypto::PBKDF2(v8::FunctionCallbackInfo<v8::Value> const&) 19557 100.0% v8::internal::Builtins::~Builtins() 19557 100.0% LazyCompile: ~pbkdf2 crypto.js:557:16 4510 11.9% _sha1_block_data_order 4510 100.0% LazyCompile: *pbkdf2 crypto.js:557:16 4510 100.0% LazyCompile: *exports.pbkdf2Sync crypto.js:552:30 3165 8.4% _malloc_zone_malloc 3161 99.9% LazyCompile: *pbkdf2 crypto.js:557:16 3161 100.0% LazyCompile: *exports.pbkdf2Sync crypto.js:552:30 NODE.JS PROFILER
29. NODE.JS PROFILER app.get('/auth', function (req, res) { const username = req.query.username || '' const password = req.query.password || '' username = username.replace(/[!@#$%^&*]/g, '') if (!username || !password || !users[username]) { return res.sendStatus(400); } crypto.pbkdf2(password, users[username].salt, 10000, 512, function (err, hash) { if (users[username].hash.toString() === hash.toString()) { res.sendStatus(200) } else { res.sendStatus(401) } }) })
30. PRO MODE++
31. GOTCHAS
32. TRANSPILERS!
33. BENCHMARKING != MONITORING
34. SYSTEM CALLS!
35. DEPENDENCIES
36. END-2-END
37. THANK YOU! @AhmadNassri http://ahmad.codes Slides & Links: ahmad.codes/talks TechMasters.chat Functions.world FSTO.co