Cleanup handling in Go / Go Conference 2024

Transcript

1. Cleanup handling in Go খࢁ݈Ұ࿠ / GMO PEPABO inc. 2024.6.8 Go Conference 2024 1
2. ࣗݾ঺հ ٕज़෦ɹٕज़ج൫νʔϜ 2018೥ த్ೖࣾ খࢁ ݈Ұ࿠ Ken’ichiro Oyama গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ • GitHub : k1LoW • Twitter : @k1LoW 2
3. GMOϖύϘʹ͍ͭͯ 3 https://pepabo.com
4. ΞδΣϯμ • ຊηογϣϯͷϞνϕʔγϣϯ • ʮΫϦʔϯΞοϓʯͱ͸ • ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ • ෳࡶͳΞϓϦέʔγϣϯʹ͓͍ͯٻΊΒΕΔΫϦʔϯΞοϓ • ఏҊख๏ • ͓ΘΓʹ 4
5. 5 ຊηογϣϯͷϞνϕʔγϣϯ
6. • Go͸goroutineʹΑΔฒߦॲཧΛ༰қʹ࣮ݱͰ͖Δɻ͔͠͠ɺͦͷฒߦॲཧͷऴྃॲཧ͸։ൃऀ ʹҕͶΒΕ͍ͯΔɻ • ॲཧΛͨͩఀࢭ͢Δ͚ͩͰ͸ࡁ·ͳ͍৔߹΋ଟ͘ɺద੾ͳऴྃॲཧɺͭ·ΓʮΫϦʔϯΞοϓʯ ͕ඞཁɻ • ྫ͑͹golang.org/x/sync/errgroup͸ʮ։࢝ͨ͠ฒߦॲཧͷ׬ྃʢʹऴྃʣΛ଴ͭʯͱ͍͏ ҙຯͰฒߦॲཧͷऴྃॲཧͷͨΊͷpackageͱݴ͑Δ • ຊηογϣϯΛ௨ͯ͡ɺօ͞ΜʹΫϦʔϯΞοϓॲཧʹ͍ͭͯվΊͯߟ͑ͯ΋Β͍ɺ֤։ൃݱ৔ ʹ͋ΔͰ͋Ζ͏ΫϦʔϯΞοϓॲཧʹ͍ͭͯGoίϛϡχςΟͰձ࿩͕͞ΕΔ͜ͱΛظ଴͍ͯ͠ Δɻ 6 ຊηογϣϯͷϞνϕʔγϣϯ ద੾ͳऴྃॲཧ=ΫϦʔϯΞοϓ
7. 7 ʮΫϦʔϯΞοϓʯͱ͸
8. • ʮޙ࢝຤ॲཧʯ΍ʮద੾ͳऴྃॲཧʯ • τϥϯβΫγϣϯॲཧ: த్ఀࢭ͢Δͱσʔλ੔߹ੑ͕ࣦΘΕΔͨΊɺ࠷ޙ·Ͱ׬ྃͤ͞Δ ʢϩʔϧόοΫؚΉʣඞཁ͕͋Δɻ • ֎෦Ϧιʔεͱͷ઀ଓ: Ϧιʔεղ์΍઀ଓղআΛߦ͏ඞཁ͕͋Δɻ • Ұ࣌ϑΝΠϧ΍σʔλ: ऴྃޙʹෆཁʹͳΔͨΊɺ࡟আ͢Δඞཁ͕͋Δɻ 8 ʮΫϦʔϯΞοϓʯͱ͸ ຊηογϣϯʹ͓͚ΔʮΫϦʔϯΞοϓʯ
9. 9 ݴޠػೳ΍ඪ४ύοέʔδʹ͋Δ ΫϦʔϯΞοϓػߏ
10. defer 10
11. • Goͷݴޠػೳͱͯ͠ఏڙ͞Ε͍ͯΔεςʔτϝϯτ • ొ࿥ͨؔ͠਺Λݺͼग़͠ݩͷؔ਺ͷऴΘΓʢreturnʣ·Ͱ࣮ߦΛ஗Ԇͤ͞Δ͜ͱ͕Ͱ͖Δ • A defer statement pushes a function call onto a list. The list of saved calls is executed after the surrounding function returns. Defer is commonly used to simplify functions that perform various clean-up actions.
 Deferred function calls are executed in Last In First Out order after the surrounding function returns. • Ref: https://go.dev/blog/defer-panic-and-recover • 11 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ defer
12. • ొ࿥͢Δؔ਺ͷγάωνϟ • func() • ࣮ߦλΠϛϯά • ݺͼग़͠ݩͷؔ਺ͷऴΘΓ • ࣮ߦॱ൪ • LIFO͔ͭஞ࣮࣍ߦ • ొ࿥͞Εͨؔ਺ͷ࣮ߦ׬ྃʢpanicҎ֎ʣ • ଴ͭ 12 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ defer
13. net/http func (*Server) RegisterOnShutdown 13
14. • net/httpύοέʔδͷ*ServerΛϨγʔόͱͯ͠΋ͭϝιου • RegisterOnShutdownͰొ࿥ͨؔ͠਺͸Server.Shutdown಺Ͱ࣮ߦ͞ΕΔ • RegisterOnShutdown registers a function to call on [Server.Shutdown]. This can be used to gracefully shutdown connections that have undergone ALPN protocol upgrade or that have been hijacked. This function should start protocol-specific graceful shutdown, but should not wait for shutdown to complete. • Ref: https://github.com/golang/go/blob/adbfb672ba485630d75f8b5598228a63f4af08a4/src/ net/http/server.go#L2985-L2994 • 14 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func (*Server) RegisterOnShutdown
15. • ొ࿥͢Δؔ਺ͷγάωνϟ • func() • ࣮ߦλΠϛϯά • Server.Shutdown࣮ߦ࣌ • ࣮ߦॱ൪ • FIFO͔ͭฒߦ࣮ߦ • ొ࿥͞Εͨؔ਺ͷ࣮ߦ׬ྃʢpanicҎ֎ʣ • ଴ͨͳ͍ 15 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func (*Server) RegisterOnShutdown
16. testing func (*T) Cleanup 16
17. • testingύοέʔδͷ*TΛϨγʔόͱͯ͠΋ͭϝιου • CleanupͰొ࿥ͨؔ͠਺͸ͦͷςετͱαϒςετ͕શͯ׬ྃͨ͠ͱ͖ʹ࣮ߦ͞ΕΔ • t.Parallel()ʹΑͬͯςετ͕ฒߦ࣮ߦ͞Ε͍ͯͯ΋ͦͷ࣮ߦ৚݅͸มΘΒͳ͍ • Cleanup registers a function to be called when the test (or subtest) and all its subtests complete. Cleanup functions will be called in last added, first called order. • Ref: https://github.com/golang/go/blob/adbfb672ba485630d75f8b5598228a63f4af08a4/src/ testing/testing.go#L1151-L1153 17 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func (*T) Cleanup
18. • ొ࿥͢Δؔ਺ͷγάωνϟ • func() • ࣮ߦλΠϛϯά • ର৅ͷςετͱͦͷαϒςετͷऴྃ࣌ • ࣮ߦॱ൪ • LIFO͔ͭஞ࣮࣍ߦ • ొ࿥͞Εͨؔ਺ͷ࣮ߦ׬ྃʢpanicҎ֎ʣ • ଴ͭ 18 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func (*T) Cleanup
19. context func AfterFunc 19
20. • ctx͕ऴྃͨ͠ͱ͖ʹొ࿥ͨؔ͠਺fΛ࣮ߦ͢Δ • Multiple calls to AfterFunc on a context operate independently; one does not replace another.
 (ུ) The stop function does not wait for f to complete before returning. If the caller needs to know whether f is completed, it must coordinate with f explicitly. • Ref: https://github.com/golang/go/blob/adbfb672ba485630d75f8b5598228a63f4af08a4/src/ context/context.go#L297-L314 • 20 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func AfterFunc(ctx Context, f func()) (stop func() bool)
21. • ొ࿥͢Δؔ਺ͷγάωνϟ • func() • ࣮ߦλΠϛϯά • ctxͷΩϟϯηϧ࣌ • ࣮ߦॱ൪ • FIFO͔ͭฒߦ࣮ߦ • ొ࿥͞Εͨؔ਺ͷ࣮ߦ׬ྃʢpanicҎ֎ʣ • ଴ͨͳ͍ 21 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ func AfterFunc(ctx Context, f func()) (stop func() bool)
22. ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ 22 ൺֱ
23. 23 ݴޠػೳ΍ඪ४ύοέʔδʹ͋ΔΫϦʔϯΞοϓػߏ ֤ػೳͷൺֱ ൃදऀ͕૬ରతʹྑ͍ಛੑͱߟ͍͑ͯΔ ※ defer func (*Server) RegisterOnShutdown func (*T) Cleanup func AfterFunc (ctx Context, f func()) (func() bool) ొ࿥͢Δ ؔ਺γάωνϟ func() func() func() func() ొ࿥Մೳͳείʔϓ ର৅ͱ͢Δؔ਺಺ *Server͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ *T͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ ctx͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ ࣮ߦλΠϛϯά ݺͼग़͠ݩͷ ؔ਺ͷऴΘΓ Server.Shutdown࣮ߦ࣌ ݺͼग़͠ݩͷ ςετؔ਺ͷऴΘΓ ctxͷΩϟϯηϧ࣌ ࣮ߦॱ൪ LIFO FIFO LIFO FIFO ஞ࣍/ฒߦ ஞ࣍ ฒߦ ஞ࣍ ฒߦ ొ࿥͞Εͨؔ਺ͷ ࣮ߦ׬ྃ ʢpanicҎ֎ʣ ଴ͭ ଴ͨͳ͍ ଴ͭ ଴ͨͳ͍
24. 24 ෳࡶͳΞϓϦέʔγϣϯʹ͓͍ͯ ٻΊΒΕΔΫϦʔϯΞοϓ
25. ✓ ෳ਺ͷύοέʔδʢNot ֎෦ύοέʔδʣͰߏ੒͞Ε͍ͯͯύοέʔδؒͰґଘؔ܎͕͋Δ ✓ GoroutineʹΑΔฒߦॲཧ͕׆༻͞Ε͍ͯΔ ✓ ͦͯͦ͠ΕΒ͕ࠓޙ΋૿͍͑ͯ͘༨஍͕͋Δ • ؆қͳWebΞϓϦέʔγϣϯ͕গ͠Ͱ΋ෳࡶʹͳΔͱ౰ͯ͸·ΔՄೳੑ͕͋Δ 25 ෳࡶͳΞϓϦέʔγϣϯʹ͓͍ͯٻΊΒΕΔΫϦʔϯΞοϓ ຊηογϣϯʹ͓͚ΔʮෳࡶͳΞϓϦέʔγϣϯʯ
26. ✓ ෳ਺ͷύοέʔδ΍ؔ਺Λ·͍ͨͰΫϦʔϯΞοϓॲཧΛొ࿥͍ͨ͠ ✓ ฒߦॲཧͰ΋ؔ܎ͳ͘ΫϦʔϯΞοϓॲཧΛొ࿥͍ͨ͠ ✓ ʢͰ͖Ε͹ʣΞϓϦέʔγϣϯΞʔΩςΫνϟʹӨڹΛ༩͑ͳ͍ܗͰ࣮ݱ͍ͨ͠ ✓ ඞཁʹԠͯ͡ΫϦʔϯΞοϓॲཧͷ׬ྃΛ଴͍ͪͨ ✓ ΫϦʔϯΞοϓॲཧͷΤϥʔ΋ัଊ͍ͨ͠ • Ͳ͜Ͱ΋ΫϦʔϯΞοϓॲཧΛొ࿥Ͱ͖ͯɺඞཁʹԠͯͦ͡ΕΒͷॲཧͷ׬ྃΛ଴ͯͯɺΤϥʔ ͷัଊ΋Ͱ͖Δͱྑ͍ 26 ෳࡶͳΞϓϦέʔγϣϯʹ͓͍ͯٻΊΒΕΔΫϦʔϯΞοϓ ෳࡶͳΞϓϦέʔγϣϯʹ͓͍ͯٻΊΒΕΔΫϦʔϯΞοϓ
27. 27 ఏҊख๏
28. + ύοέʔδ΍ؔ਺Λԣஅͨ͠ΫϦʔϯΞοϓॲཧͷ࣮ݱํ๏ • ෳ਺ͷύοέʔδ΍ؔ਺Λ·͙ͨඞཁ͕͋Δɻ • Goͷίʔυʹ͓͍ͯෳ਺ͷύοέʔδ΍ؔ਺Λ·͍ͨͰ͍Δͷ͸ context.Context • context.ValueΛ࢖͏͜ͱͰʮෳ਺ͷύοέʔδ΍~ʯ΍ʮฒߦॲཧʯ΋ղܾͰ͖ͦ͏ • طଘͷ context.Context Λ࢖͏͜ͱͰΞʔΩςΫνϟ΁ͷӨڹ΋࠷খݶͰࡁΉ + ΫϦʔϯΞοϓॲཧͷ׬ྃ଴ͪͷ࣮ݱํ๏ • ٻΊΔڍಈͱͯ͠͸ sync.WaitGroup ΍ golang.org/x/sync/errgroup.Group ͕͍ۙ = context.WithValueܦ༝Ͱ context.Context ʹ *sync.WaitGroup / *errgroup.Group Λ࣋ͨͤͯ ֤ॴͰΫϦʔϯΞοϓॲཧΛొ࿥͢Δ 28 ఏҊख๏ ఏҊΞʔΩςΫνϟ
29. 29 donegroup
30. 30 جຊతͳ࢖͍ํ donegroup.Cleanup
31. 31 جຊతͳ࢖͍ํ 1. ·ͣɺcontext.WithCancelΛॻ͖׵͑Δ
32. • donegroup.ClanupͰΫϦʔϯΞοϓ༻ͷؔ਺Λcontext.Contextʹؔ࿈͚ͮͯ௥Ճ͢Δɻ • ύοέʔδ΍ؔ਺Λ·͍ͨͰ͍ͯ΋goroutine಺Ͱ΋OK 32 جຊతͳ࢖͍ํ 2. ೚ҙͷ৔ॴͰcontext.Contextʹؔ࿈͚ͮͨΫϦʔϯΞοϓॲཧΛ௥Ճ͢Δ
33. • context.Context͕Ωϟϯηϧ͞Εͨʢऴྃͨ͠ʣλΠϛϯάͰొ࿥͍ͯͨؔ͠਺ΛҰؾʹ࣮ߦ͢Δ ʢFIFO/ฒߦ࣮ߦʣ • donegroup.Wait ͰॲཧΛϒϩοΫ͠શͯͷΫϦʔϯΞοϓॲཧͷ׬ྃΛ଴ͭ • ฦΓ஋͔Β֤ΫϦʔϯΞοϓॲཧͷΤϥʔ΋·ͱΊͯิ଍Ͱ͖Δʢerrors.Joinʣ 33 جຊతͳ࢖͍ํ 3. ొ࿥ͨ͠ΫϦʔϯΞοϓॲཧͷ࣮ߦ׬ྃΛ଴ͭ
34. • donegroup is a package that provides a graceful cleanup transaction to context.Context when the context is canceled. • https://github.com/k1LoW/donegroup • طଘίʔυͰ఻ൖ͍ͤͯ͞Δcontext.ContextΛ࢖ͬͯύοέʔδ΍ؔ਺Λԣஅͨ͠τϥϯβΫ γϣϯʢಛʹॲཧͷ׬ྃͷ؅ཧʣΛఏڙ͢ΔύοέʔδͰ͢ɻ • ඪ४ύοέʔδͷΈͰ࣮૷͍ͯ͠ΔͷͰZero dependency 34 ఏҊख๏ donegroup
35. • ొ࿥͢Δؔ਺ͷγάωνϟ • func() error • ࣮ߦλΠϛϯά • ctxͷΩϟϯηϧ࣌ • ࣮ߦॱ൪ • FIFO͔ͭฒߦ࣮ߦ • ొ࿥͞Εͨؔ਺ͷ࣮ߦ׬ྃʢpanicҎ֎ʣ • ʢdonegroup.WaitͰʣ଴ͭ 35 ఏҊख๏ donegroup.Cleanup
36. ఏҊख๏ 36 ൺֱʢ࠶ʣ
37. 37 ఏҊख๏ ֤ػೳͷൺֱ defer func (*Server) RegisterOnShutdown func (*T) Cleanup func AfterFunc (ctx Context, f func()) (func() bool) dongroup.Cleanup ొ࿥͢Δ ؔ਺γάωνϟ func() func() func() func() func() error ొ࿥Մೳͳείʔϓ ର৅ͱ͢Δؔ਺಺ *Server͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ *T͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ ctx͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ ctx͕౉͍ͬͯΕ͹Ͳ͜Ͱ΋ ࣮ߦλΠϛϯά ݺͼग़͠ݩͷ ؔ਺ͷऴΘΓ Server.Shutdown࣮ߦ࣌ ݺͼग़͠ݩͷ ςετؔ਺ͷऴΘΓ ctxͷΩϟϯηϧ࣌ ctxͷΩϟϯηϧ࣌ ࣮ߦॱ൪ LIFO FIFO LIFO FIFO FIFO ஞ࣍/ฒߦ ஞ࣍ ฒߦ ஞ࣍ ฒߦ ฒߦ ొ࿥͞Εͨؔ਺ͷ ࣮ߦ׬ྃ ʢpanicҎ֎ʣ ଴ͭ ଴ͨͳ͍ ଴ͭ ଴ͨͳ͍ ʢdongroup.WaitͰʣ ଴ͭ ൃදऀ͕૬ରతʹྑ͍ಛੑͱߟ͍͑ͯΔ ※
38. 38 ೚ҙͷॲཧϒϩοΫͷ࣮ߦ׬ྃΛอূ donegroup.Awaiter
39. • donegroup.AwaiterͰcontext.Contextʹؔ࿈͚ͮΔɻ • completed()͕࣮ߦ͞ΕΔ·Ͱ donegroup.Wait ͸ॲཧΛϒϩοΫ͢Δ • donegroup.Awaiter͔Βcompleted·Ͱͷॲཧ͕ऴྃ͢Δ·Ͱ଴ͭ 39 ೚ҙͷॲཧϒϩοΫͷ࣮ߦ׬ྃΛอূ completed͕࣮ߦ͞ΕΔ·Ͱdonegroup.Wait͸଴ͭ
40. 40 ύοέʔδ΍ؔ਺Λ·͍ͨͩ errgroup func (*Group) GoϥΠΫͳؔ਺ donegroup.Go
41. • donegroup.AwaiterͱҟͳΓɺgoroutineͰฒߦ࣮ߦ͞ΕΔ • errgroup func (*Group) Goͷڍಈͱ΄΅ಉ͡ • ݸਓతʹ͸errgroupʹ׳Ε͍ͯΔͷͰ޷ΈͷγϯλοΫε • context.ContextΛ௨ͯ͡ύοέʔδ΍ؔ਺Λ·͙ͨ͜ͱ͕Ͱ͖Δ 41 ύοέʔδ΍ؔ਺Λ·͍ͨͩ errgroup func (*Group) GoϥΠΫͳؔ਺ donegroup.GoͰฒߦ࣮ߦΛ։࢝ͯ͠donegroup.WaitͰ࣮ߦ׬ྃΛ଴ͭ
42. 42 donegroupͷίΞ࣮૷
43. • context.WithCancelCause ͔Βಘͨ஋Λ context.Value + sync.WaitGroup ͰՃ޻ • context.Contextͷ਌ࢠؔ܎ʹ΋ରԠ 43 donegroupͷίΞ࣮૷ context.Value + sync.WaitGroup
44. 44 ͓ΘΓʹ
45. 45 ͓ΘΓʹ ͓ΘΓʹ • ຊηογϣϯʹ͓͚ΔʮΫϦʔϯΞοϓʯΛఆٛͨ͋͠ͱɺݴޠػߏ΍ඪ४ύοέʔδͰΫϦʔ ϯΞοϓΛϢʔεέʔεͷ1ͭʹ΋ͭػೳʹ͍ͭͯ͋ͨΒΊͯΈ͍ͯͬͨɻ • ࣍ʹɺΑΓෳࡶͳίʔυϕʔεΛ࣋ͭΞϓϦέʔγϣϯʹ͓͍ͯඞཁʹͳΔΫϦʔϯΞοϓॲཧ ʹ͍ͭͯߟ͑ཁ݅Λྻڍͨ͠ɻ • ্هཁ݅Λຬͨ͢ΞʔΩςΫνϟͱͯ͠ context.Value ͱ sync.WaitGroupʢ΍ errgroup.Groupʣ Λ૊Έ߹Θͤͨߏ੒ΛఏҊ͠ɺͦͷ۩ମత࣮૷ͱͯ͠ donegroup Λ঺հͨ͠ɻ • ʢվΊͯʣʮΫϦʔϯΞοϓॲཧʯʮάϨʔεϑϧγϟοτμ΢ϯʯͷ࣮૷͸։ൃͷݱ৔ʹ਺ଟ ͘ଘࡏ͢Δ͸ͣͳͷͰɺੋඇɺࠓճΛػձʹ஌ݟ͕GoίϛϡχςΟʹڞ༗͞Εͯ΄͍͠ʢੋඇ ஌Γ͍ͨʣɻ
46. 46 Thank you!