Tweet
Tweet

Slide 1

Slide 1 text

golang.tokyo#29 Deep dive into sync.Pool Morito Ikeda | January 2020

Slide 2

Slide 2 text

ABOUT ME ஑ా ৿ਓ(@_moricho_) - ܚጯٛक़େֶ ཧ޻ֶ෦ - Go, Kubernetes, ... - AbemaTV, ϦΫϧʔτ, BASE, … - ࠷ۙͷझຯɿϑΥϯτΛோΊΔ

Slide 3

Slide 3 text

Έͳ͞Μɺsync.Pool ࢖ͬͯ·͔͢ʁ

Slide 4

Slide 4 text

TABLE OF CONTENTS Deep dive into sync.Pool ɾsync.Poolͷ֓ཁ/࢖͍ํ ɾOSSͰͷsync.Poolར༻ྫ ɾsync.PoolͷΞϧΰϦζϜ ɾGo1.13ͰͷύϑΥʔϚϯεվળ

Slide 5

Slide 5 text

Overview of sync.Pool Introduction

Slide 6

Slide 6 text

ɾGo1.3͔Βඪ४ύοέʔδʹ sync.Poolͷ֓ཁ: ɾgoroutineηʔϑͳϝϞϦϓʔϧ ɾແବͳϝϞϦׂ౰ͯ/GCίετͷ࡟ݮ

Slide 7

Slide 7 text

sync.Poolͷ࢖͍ํ: ɾάϩʔόϧͳPoolͷ࡞੒ େ·͔ͳྲྀΕ ɾPool͔ΒΦϒδΣΫτΛGet (ͳ͔ͬͨΒ৽ن࡞੒) ɾ࢖͍ऴΘͬͨΒۭʹͯ͠PoolʹPut

Slide 8

Slide 8 text

sync.Poolͷ࢖͍ํ: (Poolͳ͠) ɾbytes.Bufferʹจࣈྻ٧ΊΔ͚ͩ ɾ౎౓bytes.BufferΛ৽ن࡞੒

Slide 9

Slide 9 text

sync.Poolͷ࢖͍ํ: (Pool͋Γ) ɾbytes.BufferΛPool͔Βऔಘ ɾdeferͰbufferͷϦηοτ ͱPoolʹ໭͢ॲཧ

Slide 10

Slide 10 text

sync.Poolͷ࢖͍ํ: (ϕϯνϚʔΫ) Poolͳ͠ɿ ɾ1࣮ߦ͋ͨΓ1781ms, 1072Bͷallocate, 4ճͷϝϞϦallocate Pool͋Γɿ ɾ1࣮ߦ͋ͨΓ1192ms, 0Bͷallocate, 0ճͷϝϞϦallocate

Slide 11

Slide 11 text

sync.Poolͷ࢖͍ํ: (Tips) ΦϒδΣΫτΛϦηοτ͢Δͱ͖ ɾbytes.BufferͳΒReset()ؔ਺ ɾεϥΠεΛ௚઀౉͢৔߹͸ʁ

Slide 12

Slide 12 text

sync.Poolͷ࢖͍ํ: (Tips) ΦϒδΣΫτΛϦηοτ͢Δͱ͖ ɾbytes.BufferͳΒReset()ؔ਺ ɾεϥΠεΛ௚઀౉͢৔߹͸ʁ =>

Slide 13

Slide 13 text

sync.Pool usecase in OSS usecase

Slide 14

Slide 14 text

sync.Poolͷར༻ࣄྫ: ɾfmt ɾsirupsen/logrus ɾuber-go/zap

Slide 15

Slide 15 text

sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ Λᨳ͍ͬͯΔ

Slide 16

Slide 16 text

sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ => ౉͢৘ใΛzap.String΍zap.IntͰϥοϓ interface{}Ͱड͚Δ͜ͱΛճආ

Slide 17

Slide 17 text

sync.Poolͷར༻ࣄྫ: uber-go/zap ɾʮreflection-free, zero-allocation JSON encoderʯ => jsonͷEncoderͱBufferΛPoolͰ؅ཧ

Slide 18

Slide 18 text

Algorithm of sync.Pool Internal sync.Pool

Slide 19

Slide 19 text

sync.PoolͷΞϧΰϦζϜ: FreeList ɾϝϞϦͷະׂΓ౰ͯྖҬΛ࿈݁ϦετͰ઀ଓ ฦ٫͞Εͨ΋ͷ(࠶ར༻͢ΔϝϞϦྖҬ)Λ Ϧετʹͭͳ͍Ͱ͍͘

Slide 20

Slide 20 text

sync.PoolͷΞϧΰϦζϜ: FreeList ɾϝϞϦͷະׂΓ౰ͯྖҬΛ࿈݁ϦετͰ઀ଓ ฦ٫͞Εͨ΋ͷ(࠶ར༻͢ΔϝϞϦྖҬ)Λ Ϧετʹͭͳ͍Ͱ͍͘ => ྖҬΛ༗ޮʹ࢖͑ΔɻϝϞϦޮ཰up!

Slide 21

Slide 21 text

sync.PoolͷΞϧΰϦζϜ: FreeList ɾͱ͘ʹɺ͢΂ͯͷΦϒδΣΫταΠζ͕ಉ͡Ͱ͋Δ ɹϝϞϦϓʔϧ͔ΒͷׂΓ౰ͯʹ࠷ద

Slide 22

Slide 22 text

sync.PoolͷΞϧΰϦζϜ: FreeList ɾͱ͘ʹɺ͢΂ͯͷΦϒδΣΫταΠζ͕ಉ͡Ͱ͋Δ ɹϝϞϦϓʔϧ͔ΒͷׂΓ౰ͯʹ࠷ద => ϝϞϦͷஅยԽ͕ى͜Γʹ͍͘

Slide 23

Slide 23 text

sync.PoolͷΞϧΰϦζϜ: ͦΜͳsync.Poolʹ΋໰୊͕ɺɺɺ

Slide 24

Slide 24 text

sync.PoolͷΞϧΰϦζϜ: Pool(FreeList)͸ConcurrentηʔϑʹͳΔΑ͏ʹ MutexͰLock͕औΒΕ͍ͯΔ => ෳ਺ΞΫηε͕͋Δࡍʹ ɹ ౰વΦʔόʔϔου͕͋Δ ࠔͬͨͱ͜Ζ̍

Slide 25

Slide 25 text

sync.PoolͷΞϧΰϦζϜ: GC͕࢝·Δલʹ͸Pool͕ۭʹͳΔ࢓༷ => GCޙ͸ίʔϧυελʔτͷঢ়ଶʹͳͬͯ ɹ ύϑΥʔϚϯεͷεύΠΫ͕͘Δ ࠔͬͨͱ͜Ζ̎ sync/pool.go: 224-225

Slide 26

Slide 26 text

Performance improvement on Go1.13 Internal sync.Pool

Slide 27

Slide 27 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾMutexͷLock͔͔Δ໰୊

Slide 28

Slide 28 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾMutexͷLock͔͔Δ໰୊ => ૒ํ޲࿈݁ͳFreeList΁

Slide 29

Slide 29 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾݺͼग़͠ݩ͸headPop()ͱheadPush ɾଞͷgoroutine͸tailPop()

Slide 30

Slide 30 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾGCޙͷίʔϧυελʔτ໰୊

Slide 31

Slide 31 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ: ɾGCޙͷίʔϧυελʔτ໰୊ => Victim Cacheͷ࠾༻ ɾطଘͷΩϟογϡΛҰஈޙΖͷΩϟογϡʹୀආ ɾগͳ͘ͱ΋GC2αΠΫϧ෼ੜ͖࢒Δʂ

Slide 32

Slide 32 text

Go1.13ʹ͓͚ΔύϑΥʔϚϯεվળ:

Slide 33

Slide 33 text

ੋඇsync.PoolΛ࢖ͬͯ ύϑΥʔϚϯεվળͯ͠Έ͍ͯͩ͘͞

Slide 34

Slide 34 text

࠷ޙʹએ఻

Slide 35

Slide 35 text

ٕज़ॻయ8 (02/29ͷํ) ɹͰgolang.tokyo͞Μʹ ɹدߘ͍͖ͤͯͨͩ͞·͢

Slide 36

Slide 36 text

ػձΛ༩͑ͯͩͬͨ͘͞ hgshgt͞Μɺbudougumi͞Μɺtenntenn͞Μɺ Ͳ͏΋͋Γ͕ͱ͏͍͟͝·͢ɻ

Slide 37

Slide 37 text

ࢀߟ ɾCA.go #1 Ͱ sync.Pool ʹ͍ͭͯ஻Γ·ͨ͠ #ca_go
 ɾgolangͰsync.PoolΛ࢖ͬͯলίετˍߴ଎Խ - Carpe Diem
 ɾGoݴޠͷLoggerʮzapʯ͸Կނߴ଎ʹߏ଄Խ͞ΕͨϩάΛग़ྗ͢Δࣄ͕ग़དྷΔͷ͔ʛגࣜձࣾCA Wise ɾGo: Understand the Design of Sync.Pool - A Journey With Go - Medium
 ɾsrc/sync/pool.go - The Go Programming Language
 ɾsync: use lock-free structure for Pool stealing · golang/go@d5fd2dd · GitHub
 ɾsync: smooth out Pool behavior over GC with a victim cache · golang/go@2dcbf8b · GitHub ɾGo 1.13த sync.Pool ੋ೗ԿႪԽత? | ୔໠
 ɾGO: sync.Pool తൌགྷ༩ԋࣉ - Goე࿽ᇏ໓ຩ - Golangதจࣾ۠