Make it Fast: The Power of Benchmarking

A talk I presented at FunctionalConf 2017 in Bangalore. How, when, and why to use benchmarking in Elixir

Transcript

1. Making it Fast The power of benchmarking
2. Hi! I’m Cameron (@cameronp)
5. I write software…
6. sometimes.
9. Making it fast…
10. GC optimization, coding tricks, compiler hacks, VM flags, etc
11. GC optimization, coding tricks, compiler hacks, VM flags, etc
12. … fast is a process
13. Benchmarking
14. Benchmarking != Profiling
15. Benchmarking ~= Unit Testing
16. Benchmarking • Granular • Repeatable • Relative
17. A short and only slightly contrived story about performance…
18. When learning a new language / stack…
19. Small problems > Big projects
20. Small_problem |> Refactor
 |> Optimize
 |> Document |> …
21. Where to get small problems?
22. My favorite: 
 http://adventofcode.com
25. December 6, 2015
26. Xmas lights • 1000x1000 grid of lights • Addressable 0,0 to 999,999 • Controllable using special commands • Process the commands and count how many lights are lit at the end.
27. turn off 660,55 through 986,197 turn off 341,304 through 638,850 turn off 199,133 through 461,193 toggle 322,558 through 977,958 toggle 537,781 through 687,941 turn on 226,196 through 599,390 …
28. Not ideal for Elixir • 2-d Array. We ain’t got ‘em. • Mutating a giant data structure • Difficult, maybe impossible to parallelize
29. Too bad. We’re doing it anyway.
30. Make it Work
 Make it Right
 Make it Fast
31. Make it Work
 Make it Right
 Make it Fast - Kent Beck
32. Make it Work
 Make it Right
 Make it Fast - Kent Beck?
33. Make it Work
34. What does that mean?
35. Why?
36. Understanding
37. To the code…
38. Make it Right
39. What’s that mean?
40. Make it Right
41. Make it Right Refactored
42. Ok, let’s look at the refactored code…
43. Make it Fast
44. Take out the slow parts
45. Benchmarking ~= Unit Testing
46. Benchfella https://github.com/alco/benchfella
47. defp deps do [ {:benchfella, "~> 0.3.0"} ] end
49. turn off 66,55 through 98,197 turn off 34,30 through 63,85 turn off 19,13 through 46,19 toggle 32,55 through 97,95
50. Let’s benchmark!
51. Ok, we know that process() is slow. Now what?
52. Erlang/OTP to the rescue!
54. Let’s see how that looks…
55. WAT?
56. cameronp@gabriel ~/p/s/b/fc2017> time mix run -e 'Lights.run' 400410 35.16 real 30.07 user 5.41 sys cameronp@gabriel ~/p/s/b/fc2017> time mix run -e 'Lights.run' 400410 18.48 real 17.65 user 1.38 sys cameronp@gabriel ~/p/s/b/fc2017>
57. WAT!!??
58. turn off 660,55 through 986,197 turn off 341,304 through 638,850 turn off 199,133 through 461,193 toggle 322,558 through 977,958
59. Any difference?
60. The size of your data matters, be careful when simplifying!
61. Benchmarking Tips • Write benchmarks early, but not first • Simplify the inputs • But NOT TOO MUCH, consider size • Keep running them
62. Thanks! I am “cameronp” on: • twitter • github • Elixir slack • medium • gmail
63. Questions?
64. NYC Traffic: joiseyshowaa. on Flickr Brooklyn Bridge: rodrigoparedes, on Flickr Photo Credits: © 2017, Cameron Price