Performance and Lies, Euroclojure 2015

A deep understanding of Clojure, how it executes, and the runtime it sits on is a whole lot of work to acquire. But there's a shortcut. Doing performance work forces you to peel apart the layers of abstraction your software sits on, and understand what really goes on. This is an exploration of war stories from performance tuning a production Clojure application, with an eye towards surprising lessons about Clojure and the runtime it executes on: the JVM.
About the speaker: Tom Crayford has been using Clojure since before Leiningen was released. He runs Yeller, which tracks the exceptions your Clojure apps hit in production and helps you fix them.

Transcript

1. Hi (some code)
2. Fun
3. Learn
4. Software
5. Works
6. Performance and Lies
7. Tom Crayford @t_crayford
8. Yeller
9. Input
10. Input DB
11. 1 million events/s/ node
12. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES WIN WIN
13. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
14. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
15. DISCLAIMER
17. Always Be Measuring A B M
19. Continous Automated Performance Testing
21. Experiment with new libs & techniques
22. benchmarking…
24. 3 ms 5s
25. Couldn’t Reproduce
26. Time
27. Time Boot
28. Time Boot Bench 1
29. Time Boot Bench 1 Bench 2
30. Time
31. Bench 1
32. Bench 1
33. Bench 1
34. Bench 1 Bench 2
35. Bench 1 Bench 2
36. PATH DEPENDENCY
37. Time
38. Time
39. Time
40. Optimization Flags
42. FLAG WIN
43. FLAG WIN -Xmn -Xmx etc
44. FLAG WIN -Xmn -Xmx etc 15%
45. FLAG WIN AggressiveOpts -Xmn -Xmx etc 15%
46. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15%
47. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15% UseCompressedOops
48. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15% UseCompressedOops 8.1%
49. 32%
50. -XX:TieredStopAtLevel=1
51. Lein
52. Startup Time
53. -XX:TieredStopAtLevel=1
54. 1 million 380k
55. :jvm-opts ^:replace [“-server”]
56. Inlining
57. (defn b [foo] (inc foo))
58. (defn a [foo] (b foo)) (defn b [foo] (inc foo))
59. (defn a [foo] (b foo)) (defn b [foo] (inc foo))
60. (defn a [foo] (inc foo))
61. enables optimizations
62. (defn things [foo n]
63. (defn things [foo n] (dotimes [_ n]
64. (defn things [foo n] (dotimes [_ n] (b foo)))
65. (defn things [foo n] (dotimes [_ n] (b foo))) (defn something [foo]
66. (defn things [foo n] (dotimes [_ n] (b foo))) (defn something [foo] (things foo 0))
67. (defn things [foo n] (dotimes [_ n] (b foo))) (defn something [foo] (things foo 0))
68. (defn something [foo] (dotimes [_ 0] (b foo))
69. (defn something [foo] nil)
70. August
71. “this error always happens on Chrome 36.0”
72. 250k 25k
74. Hammock
75. 20 minutes of function extracting later
76. 248k 25k
77. (assoc thing :key1 (f-a a-thing) :key-2 (f-b a-thing) :key3 (f-c a-thing) :key4 (f-d a-thing))
78. FreqInlineSize=325
79. MaxInlineLevel=9
80. no.dissassemble
81. Profilers
82. Safe Points
83. Known Memory
84. Time
85. Time
86. Time
87. Time
88. Time
89. Time
90. Time
91. Time
92. Time
93. Time
94. Time
95. Time
96. Back edge of loops Method Return
97. Sampling Profilers
98. Time
99. Time
100. Flight Recorder
101. -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints
102. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES WIN WIN
103. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
104. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
105. Clojure
106. maps are cheap?
107. defrecord
108. [1 2]
109. (deftype Tuple1 [e1]) (deftype Tuple2 [e1 e2])
110. test.check 23% [h c]
111. onyx 6.4% (:foo blah)
112. cambrian collections CLJ-1517 & CLJ-1610
113. (.__extmap record)
115. Profiler caught bug
116. Fun
117. Learn
118. Software
119. Works
120. (:or (:fingerprint error) (compute-fingerprint error))
121. :or
122. :or
123. :or
124. :or :or :or :or :or :o or
125. (:or "2e4312c83e985..." (compute-fingerprint error))
126. 1.4ms 0.34ms
127. (:or (:fingerprint error) (compute-fingerprint error))
128. (or (:fingerprint error) (compute-fingerprint error))
129. commit 0c9d7392b7b2ea51383cc46394b2b0 Author: Tom Crayford <tcrayford@yellerapp.com> Date: Mon Jul 1 10:52:04 2013 +0100
130. References
133. Conclusion
134. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES WIN
135. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
136. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND TOOLS
137. Fun
138. Learn
139. Software
140. Works
141. @t_crayford
142. Performance and Lies
143. Thanks @t_crayford