Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep dive into sync.Pool

71c783224e1fccdb1d02ed37d494247f?s=47 moricho
February 07, 2020

Deep dive into sync.Pool

golang.tokyo#29 20min

71c783224e1fccdb1d02ed37d494247f?s=128

moricho

February 07, 2020
Tweet

More Decks by moricho

Other Decks in Programming

Transcript

  1. golang.tokyo#29 Deep dive into sync.Pool Morito Ikeda | January 2020

  2. ABOUT ME ஑ా ৿ਓ(@_moricho_) - ܚጯٛक़େֶ ཧ޻ֶ෦ - Go, Kubernetes,

    ... - AbemaTV, ϦΫϧʔτ, BASE, … - ࠷ۙͷझຯɿϑΥϯτΛோΊΔ
  3. Έͳ͞Μɺsync.Pool ࢖ͬͯ·͔͢ʁ

  4. TABLE OF CONTENTS Deep dive into sync.Pool ɾsync.Poolͷ֓ཁ/࢖͍ํ ɾOSSͰͷsync.Poolར༻ྫ ɾsync.PoolͷΞϧΰϦζϜ

    ɾGo1.13ͰͷύϑΥʔϚϯεվળ
  5. Overview of sync.Pool Introduction

  6. ɾGo1.3͔Βඪ४ύοέʔδʹ sync.Poolͷ֓ཁ: ɾgoroutineηʔϑͳϝϞϦϓʔϧ ɾແବͳϝϞϦׂ౰ͯ/GCίετͷ࡟ݮ

  7. sync.Poolͷ࢖͍ํ: ɾάϩʔόϧͳPoolͷ࡞੒ େ·͔ͳྲྀΕ ɾPool͔ΒΦϒδΣΫτΛGet (ͳ͔ͬͨΒ৽ن࡞੒) ɾ࢖͍ऴΘͬͨΒۭʹͯ͠PoolʹPut

  8. sync.Poolͷ࢖͍ํ: (Poolͳ͠) ɾbytes.Bufferʹจࣈྻ٧ΊΔ͚ͩ ɾ౎౓bytes.BufferΛ৽ن࡞੒

  9. sync.Poolͷ࢖͍ํ: (Pool͋Γ) ɾbytes.BufferΛPool͔Βऔಘ ɾdeferͰbufferͷϦηοτ ͱPoolʹ໭͢ॲཧ

  10. sync.Poolͷ࢖͍ํ: (ϕϯνϚʔΫ) Poolͳ͠ɿ ɾ1࣮ߦ͋ͨΓ1781ms, 1072Bͷallocate, 4ճͷϝϞϦallocate Pool͋Γɿ ɾ1࣮ߦ͋ͨΓ1192ms, 0Bͷallocate, 0ճͷϝϞϦallocate

  11. sync.Poolͷ࢖͍ํ: (Tips) ΦϒδΣΫτΛϦηοτ͢Δͱ͖ ɾbytes.BufferͳΒReset()ؔ਺ ɾεϥΠεΛ௚઀౉͢৔߹͸ʁ

  12. sync.Poolͷ࢖͍ํ: (Tips) ΦϒδΣΫτΛϦηοτ͢Δͱ͖ ɾbytes.BufferͳΒReset()ؔ਺ ɾεϥΠεΛ௚઀౉͢৔߹͸ʁ =>

  13. sync.Pool usecase in OSS usecase

  14. sync.Poolͷར༻ࣄྫ: ɾfmt ɾsirupsen/logrus ɾuber-go/zap

  15. sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ Λᨳ͍ͬͯΔ

  16. sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ => ౉͢৘ใΛzap.String΍zap.IntͰϥοϓ interface{}Ͱड͚Δ͜ͱΛճආ

  17. sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ => jsonͷEncoderͱBufferΛPoolͰ؅ཧ

  18. Algorithm of sync.Pool Internal sync.Pool

  19. sync.PoolͷΞϧΰϦζϜ: FreeList ɾϝϞϦͷະׂΓ౰ͯྖҬΛ࿈݁ϦετͰ઀ଓ ฦ٫͞Εͨ΋ͷ(࠶ར༻͢ΔϝϞϦྖҬ)Λ Ϧετʹͭͳ͍Ͱ͍͘

  20. sync.PoolͷΞϧΰϦζϜ: FreeList ɾϝϞϦͷະׂΓ౰ͯྖҬΛ࿈݁ϦετͰ઀ଓ ฦ٫͞Εͨ΋ͷ(࠶ར༻͢ΔϝϞϦྖҬ)Λ Ϧετʹͭͳ͍Ͱ͍͘ => ྖҬΛ༗ޮʹ࢖͑ΔɻϝϞϦޮ཰up!

  21. sync.PoolͷΞϧΰϦζϜ: FreeList ɾͱ͘ʹɺ͢΂ͯͷΦϒδΣΫταΠζ͕ಉ͡Ͱ͋Δ ɹϝϞϦϓʔϧ͔ΒͷׂΓ౰ͯʹ࠷ద

  22. sync.PoolͷΞϧΰϦζϜ: FreeList ɾͱ͘ʹɺ͢΂ͯͷΦϒδΣΫταΠζ͕ಉ͡Ͱ͋Δ ɹϝϞϦϓʔϧ͔ΒͷׂΓ౰ͯʹ࠷ద => ϝϞϦͷஅยԽ͕ى͜Γʹ͍͘

  23. sync.PoolͷΞϧΰϦζϜ: ͦΜͳsync.Poolʹ΋໰୊͕ɺɺɺ

  24. sync.PoolͷΞϧΰϦζϜ: Pool(FreeList)͸ConcurrentηʔϑʹͳΔΑ͏ʹ MutexͰLock͕औΒΕ͍ͯΔ => ෳ਺ΞΫηε͕͋Δࡍʹ ɹ ౰વΦʔόʔϔου͕͋Δ ࠔͬͨͱ͜Ζ̍

  25. sync.PoolͷΞϧΰϦζϜ: GC͕࢝·Δલʹ͸Pool͕ۭʹͳΔ࢓༷ => GCޙ͸ίʔϧυελʔτͷঢ়ଶʹͳͬͯ ɹ ύϑΥʔϚϯεͷεύΠΫ͕͘Δ ࠔͬͨͱ͜Ζ̎ sync/pool.go: 224-225

  26. Performance improvement on Go1.13 Internal sync.Pool

  27. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾMutexͷLock͔͔Δ໰୊

  28. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾMutexͷLock͔͔Δ໰୊ => ૒ํ޲࿈݁ͳFreeList΁

  29. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾݺͼग़͠ݩ͸headPop()ͱheadPush ɾଞͷgoroutine͸tailPop()

  30. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾGCޙͷίʔϧυελʔτ໰୊

  31. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾGCޙͷίʔϧυελʔτ໰୊ => Victim Cacheͷ࠾༻ ɾطଘͷΩϟογϡΛҰஈޙΖͷΩϟογϡʹୀආ ɾগͳ͘ͱ΋GC2αΠΫϧ෼ੜ͖࢒Δʂ

  32. Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ:

  33. ੋඇsync.PoolΛ࢖ͬͯ ύϑΥʔϚϯεվળͯ͠Έ͍ͯͩ͘͞

  34. ࠷ޙʹએ఻

  35. ٕज़ॻయ8 (02/29ͷํ) ɹͰgolang.tokyo͞Μʹ ɹدߘ͍͖ͤͯͨͩ͞·͢

  36. ػձΛ༩͑ͯͩͬͨ͘͞ hgshgt͞Μɺbudougumi͞Μɺtenntenn͞Μɺ Ͳ͏΋͋Γ͕ͱ͏͍͟͝·͢ɻ

  37. ࢀߟ ɾCA.go #1 Ͱ sync.Pool ʹ͍ͭͯ஻Γ·ͨ͠ #ca_go
 ɾgolangͰsync.PoolΛ࢖ͬͯলίετˍߴ଎Խ - Carpe

    Diem
 ɾGoݴޠͷLoggerʮzapʯ͸Կނߴ଎ʹߏ଄Խ͞ΕͨϩάΛग़ྗ͢Δࣄ͕ग़དྷΔͷ͔ʛגࣜձࣾCA Wise ɾGo: Understand the Design of Sync.Pool - A Journey With Go - Medium
 ɾsrc/sync/pool.go - The Go Programming Language
 ɾsync: use lock-free structure for Pool stealing · golang/go@d5fd2dd · GitHub
 ɾsync: smooth out Pool behavior over GC with a victim cache · golang/go@2dcbf8b · GitHub ɾGo 1.13த sync.Pool ੋ೗ԿႪԽత? | ୔໠
 ɾGO: sync.Pool తൌགྷ༩ԋࣉ - Goე࿽ᇏ໓ຩ - Golangதจࣾ۠