Profiling Go Application

6247c099ad62bf727a4f9df10b6c6f23?s=47 orisano
November 25, 2018

Profiling Go Application

6247c099ad62bf727a4f9df10b6c6f23?s=128

orisano

November 25, 2018
Tweet

Transcript

  1. Profiling Go Application Go Conference 2018 Autumn #gocon #gocon_a @orisano

  2. ࣗݾ঺հ • @orisano • ۦ͚ग़͠ΞϓϦέʔγϣϯΤϯδχΞ • ޮ཰తͳdocker buildΛ໨ࢦ͍ͯ͠Δ • ύϑΥʔϚϯεʹڵຯ͕͋Δ

  3. ࠓ೔࿩͢͜ͱ • ߴ଎Խ΍, ϓϩϑΝΠϦϯάΛߦ͖࣮ͬͯͨྫ • Ͳ͏͍͏վળΛ͢Δͷ͔ • Ͳ͏͍͏ϓϩηεͳͷ͔

  4. ը૾ม׵αʔόͷ࿩ kubeletͷ࿩ go-swaggerͷ࿩

  5. ը૾ม׵αʔόͷ࿩ kubeletͷ࿩ go-swaggerͷ࿩

  6. ৄղ γεςϜɾύϑΥʔϚϯε ྠಡձ

  7. ୊ࡐͱͯ͠؆୯ͳ ը૾ม׵αʔόΛѻͬͨ

  8. ෛՙΛ͔͚Δͱ ΞϓϦέʔγϣϯ͕མͪͨ

  9. topΛΈͨ

  10. མͪͨݪҼ͸ ϝϞϦͷ࢖༻ྔ

  11. ϝϞϦϦʔΫʁ

  12. pprofΛ࢖͏ GODEBUG=allocfreetrace=1 ͢Δ

  13. runtimeʹखΛೖΕΔ

  14. GC͍ͯ͠Δͱ͖ʹ ࢀরgraphΛdump͢ΔΑ͏ ʹͨ͠

  15. None
  16. ͪΌΜͱpprofͷmemory goroutineͷ਺ΛݟΔ

  17. ಉ࣌ม׵਺ʹ੍ݶΛ͚ͭΔ

  18. ແࣄ໰୊͸ղܾͨ͠

  19. ͋ͱ͸ߴ଎Խ

  20. disintegration/imaging

  21. Boundary Check Elimination

  22. Go͸جຊఴࣈΞΫηε͢Δͱ ൣғνΣοΫ͕ೖΔ

  23. ίϯύΠϥ͕҆શͩͱ ൑அ͢Δͱফ͑Δ

  24. ͜ΕΛ Boundary Check Elimination (BCE)ͱ͍͏

  25. ͜ΕΛ Boundary Check Elimination (BCE)ͱ͍͏ ref: https://go101.org/article/bounds-check-elimination.html

  26. goͷඪ४ϥΠϒϥϦͰ ͦͦ͜͜ݟΔ

  27. ը૾ͷಡΈࠐΈ, ม׵ͳͲ heavyʹఴࣈΞΫηε͢Δ ৔߹ͩͱޮՌ͕େ͖͍

  28. https://github.com/ disintegration/imaging/ pull/81

  29. https://github.com/ disintegration/imaging/ pull/81

  30. None
  31. pprofͳͲͰ ݪҼΛಛఆͨ͠Β

  32. ·ͣ BenchmarkΛॻ͜͏ *testing.B

  33. Benchmarkͷ݁ՌΛ ͪΌΜͱݟΔ

  34. golang.org/x/tools/cmd/ benchcmp golang.org/x/perf/cmd/ benchstat

  35. https://github.com/ disintegration/imaging/ pull/81

  36. ը૾ม׵αʔόͷ࿩ kubeletͷ࿩ go-swaggerͷ࿩

  37. kubeletͷCPU͕ෛՙ͕ߴ͍ ͱ૬ஊΛड͚ͨ

  38. Datadog͕ୟ͍ͯΔ Metrics༻ͷEndpoint͕ ݪҼͳͷͰ͸ͱ͍͏࿩ͩͬͨ

  39. kubelet͸net/http/pprofΛ import͍ͯ͠Δ

  40. ໰୊͕ى͖͍ͯΔ؀ڥͷ ৘ใ͕औͬͯ͜ΕΔ

  41. None
  42. prometheus/common google/cadvisor ͷҰ෦͕ॏ͍

  43. None
  44. ౰֘Օॴ https://github.com/prometheus/ common/pull/148 ൃ֮ͷ1िؒલ͘Β͍ʹ allocation freeʹͳͬͯ 2ഒ͘Β͍ૣ͘ͳ͍ͬͯͨ

  45. None
  46. ͳΜ͔չ͔ͬͨ͠ͷͰ https://github.com/prometheus/ common/pull/150 ߋʹͦΕΛ2ഒ͘Β͍ૣ͘͠·ͨ͠

  47. None
  48. ७ਮʹWrite͕ଟ͔ͬͨ

  49. strings.Replacerʹ͸ WriteString͕͋Γ·͢

  50. ஔ׵ର৅ՕॴҎ֎͸ ·ͱΊͯWriteͯ͘͠ΕΔ

  51. allocation΋ى͜Βͳ͍ͷͰ࠷ߴ

  52. ࣍ʹ

  53. google/cadvisorͷ ListContainers͕ॏ͍

  54. None
  55. google/cadvisorͷ ListContainers͕ॏ͍ https://github.com/google/cadvisor/blob/8faf19092784b75fce10ce190d37e66f89de3612/container/common/helpers.go

  56. ioutil.ReadDirͱsyscall ͕ॏ͍ʁ

  57. ioutil.ReadDirͷத਎ΛݟͯΈΔ

  58. ref: https://golang.org/src/io/ioutil/ioutil.go

  59. ref: https://golang.org/src/os/dir_unix.go

  60. ͳΜ͔ҎԼʹ͋ΔϑΝΠϧશ෦ʹ lstatൃߦ͍ͯ͠Δ

  61. ref: https://golang.org/src/os/dir_unix.go

  62. readdirentͬͯ΍͕ͭ͋Δ

  63. Ͳ͏΍Β͜Ε͸ entryͷtypeΛฦͯ͘͠ΕΔΒ͍͠

  64. google/cadvisorͷ ListContainers͕ॏ͍ https://github.com/google/cadvisor/blob/8faf19092784b75fce10ce190d37e66f89de3612/container/common/helpers.go

  65. readdirentΛ͏·͘࢖͑Ε͹ syscallͷճ਺ΛݮΒͤΔ

  66. ͔͠͠ golangඪ४ͷreaddirent͸ typeΛऔಘͰ͖ͳ͍

  67. ͔͠΋ ࣮૷ʹඞཁͳॾʑ͕ privateʹͳ͍ͬͯΔ

  68. ඪ४͔Βͦ͜Λίϐϖ͢Δͱ cadvisorͷ LICENSE Checkʹ௨Βͳ͍

  69. ࣮͸͜ͷํ๏ ͍ΖΜͳϥΠϒϥϦͰ࢖ΘΕ͍ͯΔ goimports/internal/fastwalk ֎෦͔Βར༻Ͱ͖ͳ͍

  70. ѹ౗తʹ֎෦͔Βར༻Ͱ͖ͳ͍

  71. github.com/karrick/godirwalk Α͏΍͘ݟ͚ͭͨ

  72. https://github.com/google/ cadvisor/pull/2085

  73. None
  74. Memoryܥͷऔಘ͚ͩ΍ͨΒ஗͍

  75. https://lkml.org/lkml/ 2018/7/3/101 cgroups v1ܥͷ໰୊Β͍͠ kernelͷversionΛ͋͛Α͏

  76. ը૾ม׵αʔόͷ࿩ kubeletͷ࿩ go-swaggerͷ࿩

  77. ࣗಈੜ੒Λ݁ߏ࢖͍ͬͯΔͷͰ͕͢

  78. go-swagger͕஗͍

  79. ૣ͘͠Α͏

  80. cliܥͷϓϩϑΝΠϦϯάʹ͸ pkg/profile͕ศར

  81. None
  82. https://github.com/go-openapi/ swag/pull/25

  83. None
  84. ಺෦Ͱຖճ MustCompileͯͨ͠

  85. None
  86. https://github.com/go-openapi/ loads/pull/19

  87. None
  88. ஗͍json.UnmarshalΛ deepcopyͷͨΊʹ2ճͯͨ͠

  89. json.Unmarshal 1ճ {de,en}coding/gob Ͱ͓஡Λ୙ͨ͠

  90. None
  91. ·ͱΊ

  92. ීஈ࢖͍ͬͯΔ΋ͷͰ΋ ΧϦΧϦʹνϡʔχϯά ͞Ε͍ͯΔΘ͚Ͱ͸ͳ͍

  93. ࣗ෼͕஗͍ͱײͨ͡Β ܭଌͯ͠ΈΔ վળͯ͠ΈΔ

  94. cliͳΒpkg/profileΛ࢓ࠐΜͰݟΔ

  95. serverͳΒnet/http/pprofΛ࢖͏

  96. ݪҼͷಛఆͷखॿ͚ͷͨΊʹ svgʹͨ͠Γ uber/go-torchΛ࢖ͬͯ flamegraphʹͨ͠Γ flamescopeΛ࢖ͬͨΓ

  97. ݪҼΛಛఆͨ͠Β BenchmarkΛॻ͘

  98. -count=10ͳͲΛ࢖ͬͨ͋ͱʹ benchcmp or benchstat ͯ͠ޮՌ͕ग़͍ͯΔ͜ͱΛ֬ೝ

  99. ͋ͱ͸PRΛ౤͛Δ

  100. ͜Μͳʹ ύϑΥʔϚϯενϡʔχϯά ͠΍͍͢ݴޠଞʹͳ͍

  101. ΍͍ͬͯ͜͏

  102. ඞཁΛײ͡Δ·Ͱ͸ νϡʔχϯά͠ͳ͍

  103. Ͱ΋ ܭଌ͕Ͱ͖Δ؀ڥ͸༻ҙ͢΂͖