Performance and Lies, Euroclojure 2015

Performance and Lies, Euroclojure 2015

A deep understanding of Clojure, how it executes, and the runtime it sits on is a whole lot of work to acquire. But there's a shortcut. Doing performance work forces you to peel apart the layers of abstraction your software sits on, and understand what really goes on. This is an exploration of war stories from performance tuning a production Clojure application, with an eye towards surprising lessons about Clojure and the runtime it executes on: the JVM.
About the speaker: Tom Crayford has been using Clojure since before Leiningen was released. He runs Yeller, which tracks the exceptions your Clojure apps hit in production and helps you fix them.

E5edf448cb9e39e33ea6279ac4403ff8?s=128

tcrayford

June 26, 2015
Tweet

Transcript

  1. Hi (some code)

  2. Fun

  3. Learn

  4. Software

  5. Works

  6. Performance and Lies

  7. Tom Crayford @t_crayford

  8. Yeller

  9. Input

  10. Input DB

  11. 1 million events/s/ node

  12. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES

    WIN WIN
  13. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  14. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  15. DISCLAIMER

  16. None
  17. Always Be Measuring A B M

  18. None
  19. Continous Automated Performance Testing

  20. None
  21. Experiment with new libs & techniques

  22. benchmarking…

  23. None
  24. 3 ms 5s

  25. Couldn’t Reproduce

  26. Time

  27. Time Boot

  28. Time Boot Bench 1

  29. Time Boot Bench 1 Bench 2

  30. Time

  31. Bench 1

  32. Bench 1

  33. Bench 1

  34. Bench 1 Bench 2

  35. Bench 1 Bench 2

  36. PATH DEPENDENCY

  37. Time

  38. Time

  39. Time

  40. Optimization Flags

  41. None
  42. FLAG WIN

  43. FLAG WIN -Xmn -Xmx etc

  44. FLAG WIN -Xmn -Xmx etc 15%

  45. FLAG WIN AggressiveOpts -Xmn -Xmx etc 15%

  46. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15%

  47. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15% UseCompressedOops

  48. FLAG WIN AggressiveOpts 6.4% -Xmn -Xmx etc 15% UseCompressedOops 8.1%

  49. 32%

  50. -XX:TieredStopAtLevel=1

  51. Lein

  52. Startup Time

  53. -XX:TieredStopAtLevel=1

  54. 1 million 380k

  55. :jvm-opts ^:replace [“-server”]

  56. Inlining

  57. (defn b [foo] (inc foo))

  58. (defn a [foo] (b foo)) (defn b [foo] (inc foo))

  59. (defn a [foo] (b foo)) (defn b [foo] (inc foo))

  60. (defn a [foo] (inc foo))

  61. enables optimizations

  62. (defn things [foo n]

  63. (defn things [foo n] (dotimes [_ n]

  64. (defn things [foo n] (dotimes [_ n] (b foo)))

  65. (defn things [foo n] (dotimes [_ n] (b foo))) (defn

    something [foo]
  66. (defn things [foo n] (dotimes [_ n] (b foo))) (defn

    something [foo] (things foo 0))
  67. (defn things [foo n] (dotimes [_ n] (b foo))) (defn

    something [foo] (things foo 0))
  68. (defn something [foo] (dotimes [_ 0] (b foo))

  69. (defn something [foo] nil)

  70. August

  71. “this error always happens on Chrome 36.0”

  72. 250k 25k

  73. None
  74. Hammock

  75. 20 minutes of function extracting later

  76. 248k 25k

  77. (assoc thing :key1 (f-a a-thing) :key-2 (f-b a-thing) :key3 (f-c

    a-thing) :key4 (f-d a-thing))
  78. FreqInlineSize=325

  79. MaxInlineLevel=9

  80. no.dissassemble

  81. Profilers

  82. Safe Points

  83. Known Memory

  84. Time

  85. Time

  86. Time

  87. Time

  88. Time

  89. Time

  90. Time

  91. Time

  92. Time

  93. Time

  94. Time

  95. Time

  96. Back edge of loops Method Return

  97. Sampling Profilers

  98. Time

  99. Time

  100. Flight Recorder

  101. -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints

  102. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES

    WIN WIN
  103. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  104. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  105. Clojure

  106. maps are cheap?

  107. defrecord

  108. [1 2]

  109. (deftype Tuple1 [e1]) (deftype Tuple2 [e1 e2])

  110. test.check 23% [h c]

  111. onyx 6.4% (:foo blah)

  112. cambrian collections CLJ-1517 & CLJ-1610

  113. (.__extmap record)

  114. None
  115. Profiler caught bug

  116. Fun

  117. Learn

  118. Software

  119. Works

  120. (:or (:fingerprint error) (compute-fingerprint error))

  121. :or

  122. :or

  123. :or

  124. :or :or :or :or :or :o or

  125. (:or "2e4312c83e985..." (compute-fingerprint error))

  126. 1.4ms 0.34ms

  127. (:or (:fingerprint error) (compute-fingerprint error))

  128. (or (:fingerprint error) (compute-fingerprint error))

  129. commit 0c9d7392b7b2ea51383cc46394b2b0 Author: Tom Crayford <tcrayford@yellerapp.com> Date: Mon Jul 1

    10:52:04 2013 +0100
  130. References

  131. None
  132. None
  133. Conclusion

  134. WIN LIES WIN LIES WIN LIES LIES WIN WIN LIES

    WIN
  135. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  136. WIN LIES WIN LIES WIN WIN LIES WIN JVM HAND

    TOOLS
  137. Fun

  138. Learn

  139. Software

  140. Works

  141. @t_crayford

  142. Performance and Lies

  143. Thanks @t_crayford