ホリネズミでもわかるGoroutine入門 / golang.tokyo#14

Transcript

1. ϗϦωζϛͰ΋Θ͔Δ Goroutineೖ໳ golang.tokyo#14 2018/04/16
2. ࣗݾ঺հ ‣ Name: ৿ᅳ ହฏ (Morikuni Taihei) ‣ GitHub: @morikuni ‣ Twitter: @inukirom ‣ ॴଐ ‣ גࣜձࣾϝϧΧϦ ‣ GoͰόοΫΤϯυ։ൃ
3. ຊ೔ͷ಺༰ ॳ৺ऀ͕GoroutineΛ࢖͑ΔΑ͏ʹͳΔͨΊͷ࿩ ‣ Goʹ͓͚Δฒߦॲཧʹ͍ͭͯ ‣ GoroutineΛ࢖ͬͨฒߦॲཧͷ஫ҙ఺ͱରࡦ ‣ ศརͳπʔϧͷ঺հ
4. Goʹ͓͚Δฒߦॲཧʹ͍ͭͯ
5. Goʹ૊Έࠐ·Ε͍ͯΔฒߦॲཧͷͨΊͷػೳ ‣ Goroutine ‣ chan ‣ select
6. Goroutineʹ͍ͭͯ
7. Goroutineͱ͸ ‣ GoΛ୅ද͢Δػೳͷ1ͭ ‣ ฒߦॲཧΛߦ͏ͨΊͷػೳ(CPU͕ෳ਺͋Ε͹ฒྻ࣮ߦ΋Ͱ͖Δ) ‣ ϓϩηε΍εϨουΑΓ΋ܰྔ ‣ mainؔ਺΋Goroutineͱ࣮ͯ͠ߦ͞Ε͍ͯΔ
8. Goroutineͷىಈ go Λ͚ͭͯؔ਺Λݺͼग़͢͜ͱͰGoroutine͕ىಈ͢Δ go Function() go obj.Method() go func() { fmt.Println("hello goroutine") }() go func(name string) { fmt.Println("hello "+name) }("morikuni") // Ҿ਺΋౉ͤΔ
9. ࣮ߦॱং͸อূ͞Εͳ͍ go func() { fmt.Println("goroutine 1") }() go func() { fmt.Println("goroutine 2") }() go func() { fmt.Println("goroutine 3") }() // goroutine 3 // goroutine 1 // goroutine 2
10. GoroutineͱίʔϧελοΫ ‣ Goroutine͕ىಈͨ࣌͠఺Ͱؔ਺ͷίʔϧελοΫ͕෼཭͢Δ ‣ panic͸GoroutineͷίʔϧελοΫΛ໭͍ͬͯ͘ ‣ ͭ·Γɺdefer & recover͸panic͕ൃੜͨ͠Goroutine಺Ͱ࢖͏ඞཁ͕͋Δ ‣ Devquiz࿮͸͜Ε͕ϋϚΓϙΠϯτ
11. GoroutineͱίʔϧελοΫ GoroutineΛ࢖Θͳ͍৔߹ͷίʔϧελοΫ func A() { B() } func B() { defer func(){}() C() } func C() { D() } func D() { panic("dead") }
12. GoroutineͱίʔϧελοΫ func A() { func B() { defer func(){}() func C() { func D() { panic("dead") } } } } ίʔϧελοΫ: A → B → C → D Bͷdefer͕࣮ߦ͞ΕΔ
13. GoroutineͱίʔϧελοΫ GoroutineΛ࢖ͬͨ৔߹ͷίʔϧελοΫ func A() { B() } func B() { defer func(){}() go C() // goͰݺͼग़͢ } func C() { D() } func D() { panic("dead") }
14. GoroutineͱίʔϧελοΫ func A() { func B() { defer func(){}() } } func C() { func D() { panic("dead") } } ίʔϧελοΫ: C → D Bͷdefer͸࣮ߦ͞Εͳ͍
15. chanʹ͍ͭͯ
16. chanͱ͸ ‣ Goroutineؒͷσʔλͷૹड৴΍ॲཧͷಉظΛߦ͏ ‣ FIFOͷΩϡʔ(࠷ॳʹೖΕͨ஋͕࠷ॳʹऔΓग़͞ΕΔ) ‣ ༰ྔΛ࣋ͭ ‣ 3छྨͷܕ͕͋Δ ‣ Read Write ‣ Read only ‣ Write only
17. chanͷ࢖͍ํ // chanͷ࡞੒ c := make(chan int) // chan΁ͷॻ͖ࠐΈ c <- 1 // chan͔ΒͷಡΈࠐΈ x := <- c for x := range c { ... } // chanΛด͡Δ close(c) ดͨ͡chan͔Β͸θϩ஋͕ಡΈࠐΊΔΑ͏ʹͳΓɺforϧʔϓ͕ऴྃ͢Δ ดͨ͡chanʹॻ͖ࠐΉͱpanic͢Δ
18. chanͷܕ c := make(chan int) // RW chan func Publish(c chan <- int) { // WO chan ... } func Subscribe(c <-chan int) { // RO chan ... } // RW chan͸WOͱROʹ୅ೖՄೳ Publish(c) Subscribe(c)
19. selectʹ͍ͭͯ
20. selectͱ͸ ‣ ෳ਺ͷchan΁ͷૢ࡞Λѻ͏ͨΊͷ΋ͷ ‣ ࣮ߦՄೳʹͳͬͨchan΁ͷૢ࡞Λ1͚࣮ͭͩߦ͢Δ ‣ defaultΛॻ͘͜ͱͰchanͷϒϩοΫΛ๷͛Δ
21. selectͷ࢖͍ํ select { case v := <-c1: fmt.Println(v) case c2 <- "hello": } c1͔ΒͷಡΈࠐΈ or c2΁ͷॻ͖ࠐΈͷͲͪΒ͔Λ࣮ߦ͢Δ ࣮ߦͰ͖Δ·ͰϒϩοΫ͢Δ
22. selectͷ࢖͍ํ select { case v := <-c1: fmt.Println(v) case c2 <- "hello": default: fmt.Println("not ready") } c1͔ΒͷಡΈࠐΈ΋c2΁ͷॻ͖ࠐΈ΋Ͱ͖ͳ͍৔߹ʹdefault͕࣮ߦ͞ΕΔ
23. ͜͜·Ͱ͕Goʹ͓͚Δฒߦॲཧͷجૅ
24. GoroutineΛ࢖ͬͨ ฒߦॲཧͷ஫ҙ఺ͱରࡦ
25. Waitॲཧ
26. Waitॲཧ func main() { go func() { fmt.Println("Hello World") }() } ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
27. Waitॲཧ $ go run main.go (ͳʹ΋ग़ྗ͞Εͳ͍) ͳͥʁ
28. Waitॲཧ Goroutineͷॲཧ͕࣮ߦ͞ΕΔલʹϓϩηε͕ऴྃͯ͠͠·͏͔Β func main() { go func() { fmt.Println("Hello World") }() }
29. Waitॲཧ ରࡦ1: Sleep͢Δ func main() { go func() { fmt.Println("Hello World") }() time.Sleep(time.Second) } ॲཧ͕ऴΘͬͨͷʹ଴ͪଓ͚Δ͜ͱʹͳΔ ॻ͖ࣺͯͷίʔυͰΑ͘࢖͏
30. Waitॲཧ ରࡦ2: channelΛ࢖͏ func main() { c := make(chan struct{}) go func() { defer func() { c <- struct{}{} }() fmt.Println("Hello World") }() <-c // close͞ΕΔ·Ͱ଴ͭ } chanͰॲཧͷ׬ྃΛ௨஌͢Δ deferͰඞ࣮ͣߦ͞ΕΔΑ͏ʹ͢Δ chan boolΛ࢖͏Α͏ͳྫ΋͋Δ͚Ͳɺstruct{}ͷ΄͏͕Ұൠత
31. Waitॲཧ ରࡦ3: sync.WaitGroupΛ࢖͏ func main() { var wg sync.WaitGroup wg.Add(1) // GoroutineΛݺͼग़͢લʹAdd͢Δ go func() { defer wg.Done() // GoroutineͷதͰdeferΛ࢖ͬͯDoneΛݺͼग़͢ fmt.Println("Hello World") }() wg.Wait() } WaitGroup͸಺෦ͷΧ΢ϯλ͕0ʹͳΔ·Ͱwait͢Δ Done() = Add(-1)ͳͷͰAddͨ͠਺͚ͩDone͢Δඞཁ͕͋Δ
32. Waitॲཧͷ·ͱΊ ‣ αϯϓϧίʔυͳͲ͸time.Sleepָ͕ ‣ جຊతʹsync.WaitGroupΛ࢖͍ͬͯΕ͹͍͍ ‣ chan͸ෳ਺ͷGoroutineͷ׬ྃΛ଴ͭͷʹ͸࢖͍ͮΒ͍ ‣ golang.org/x/sync/errgroupͱ͍͏WaitGroupʹΤϥʔॲཧΛ௥Ճͨ͠΋ ͷ΋͋Δ
33. Goroutineͱforϧʔϓ
34. Goroutineͱforϧʔϓ 1, 2, 3, 4, 5Λग़ྗ͍ͨ͠(ॱෆಉ) xs := []int{1, 2, 3, 4, 5} for _, x := range xs { go func() { fmt.Println(x) }() } time.Sleep(time.Second) ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
35. Goroutineͱforϧʔϓ $ go run main.go 5 5 5 5 5 ͳͥʁ
36. Goroutineͱforϧʔϓ x͸ϧʔϓຖʹ஋Λ্ॻ͖͞Εͯ࢖͍ճ͞ΕΔ͔Β xs := []int{1, 2, 3, 4, 5} for _, x := range xs { go func() { fmt.Println(x) }() } time.Sleep(time.Second)
37. Goroutineͱforϧʔϓ forͷ࣮૷Πϝʔδ func iterate(xs []int, x *int) func() { i := -1 return func() { i ++ *x = xs[i] } } xs := []int{1, 2, 3, 4, 5} var x int next := iterator(xs, &x) for i := 0; i < len(xs); i ++ { next() go func() { fmt.Println(x) }() }
38. Goroutineͱforϧʔϓ ରࡦ: Ұ౓ม਺ʹ୅ೖ͢Δ for _, x := range xs { go func(x int) { // ؔ਺ͷҾ਺Ͱ͏͚ͱΔ fmt.Println(x) }(x) } for _, x := range xs { x := x // ผͷม਺ʹ୅ೖ͢Δ go func() { fmt.Println(x) }() }
39. ڝ߹ঢ়ଶ
40. ڝ߹ঢ়ଶ xsͷ஋Λ2ഒͯ͠doubleʹॻ͖ࠐΈ͍ͨ(ॱෆಉ) xs := []int{1, 2, 3, 4, 5} var double []int for _, x := range xs { go func(x int) { double = append(double, x*2) }(x) } time.Sleep(time.Second) fmt.Println(double) ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
41. ڝ߹ঢ়ଶ $ go run main.go [2 10] $ go run main.go [4 2 6 8 10] $ go run main.go [4 2 8] ͳͥʁ
42. ڝ߹ঢ়ଶ doubleΛಡΈࠐΜͰɺॻ͖ࠐΉ·Ͱʹdouble͕ߋ৽͞Ε͍ͯΔ͔Β for _, x := range xs { go func(x int) { double = append(double, x*2) }(x) } ࣮ࡍʹ͸ಡΈࠐΈ͔Βॻ͖ࠐΈ·Ͱʹ͕͔͔͍࣌ؒͬͯΔ for _, x := range xs { go func(x int) { tmp := double tmp = append(tmp, x*2) double = tmp }(x) }
43. ڝ߹ঢ়ଶ R: double͔ΒಡΈࠐΜͩ஋ W: doubleʹॻ͖ࠐΉ஋
44. ڝ߹ঢ়ଶ ରࡦ1: ௚઀Ͱॲཧ͢Δ ‣ ฒߦॲཧΛ͢Δͱڝ߹͕͓͖ΔͷͰɺഉଞ੍ޚΛߦ͏ ‣ ഉଞ੍ޚΛҰ൪؆୯ʹ΍Δํ๏͸ɺ௚ྻͰॲཧ͢Δ͜ͱͳͷͰຊ౰ʹGoroutine ࢖͏ඞཁ͕͋Δ͔Λݕ౼ͯ͠ΈΔͱ͍͍
45. ڝ߹ঢ়ଶ ରࡦ2: sync.MutexΛ࢖͏ var mu sync.Mutex xs := []int{1, 2, 3, 4, 5} for _, x := range xs { go func(x int) { mu.Lock() defer mu.Unlock() double = append(double, x*2) }(x) } Ұ౓ʹmu.Lock()Ͱ͖Δͷ͸1Օॴ͚ͩ ଞͷՕॴ͸mu.Unlock()͞ΕΔ·ͰϒϩοΫ͞ΕΔ sync.RWMutexΛ࢖͏ͱɺಡΈࠐΈͱॻ͖ࠐΈͷLockΛ෼͚ͯಡΈࠐΈ͸ฒߦͰ ߦ͑Δ
46. ڝ߹ঢ়ଶ ओʹ਺஋ܥ: sync/atomicύοέʔδΛ࢖͏ var count int64 atomic.AddInt64(&count, 1) // countʹ1Λ଍͢ atomic.CompareAndSwapInt64(&count, 1, 10) // countͷ஋͕1ͳΒ10ʹ͢Δ v := atomic.LoadInt64(&count) // countͷ஋ΛಡΈࠐΉ atomic.StoreInt64(&count, 100) // countͷ஋Λ100ʹ͢Δ float64ͳͲ͸αϙʔτ͞Ε͍ͯͳ͍͕ؤுΕ͹࢖͑Δ (floatΛ಺෦తʹ͸uintͱͯ͠ѻ͍ɺmath.Float64frombitsͱ atomic.LoadUint64ͳͲΛ૊Έ߹ΘͤΔ)
47. ڝ߹ঢ়ଶͷ·ͱΊ ‣ ௚ྻॲཧ(Goroutine͔ͭΘͳ͍)Ͱ͸μϝ͔ݕ౼ ‣ ਺஋ܥ͸sync/atomic͕࢖͑ΔՄೳੑ͕͋Δ ‣ ͦΕҎ֎͸sync.Mutex, sync.RWMutexΛ࢖͑͹େମ໰୊ͳ͍
48. Goroutine leak
49. Goroutine leak websiteAͱwebsiteBͷͲͪΒ͔ͷ݁ՌΛग़ྗ͍ͨ͠ type result struct { Response *http.Response Err error } func fetch(url string, c chan <- result) { res, err := http.Get(url) c <- result{res, err} } ch := make(chan result) go fetch("https: //websiteA", ch) go fetch("https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ ͳʹ͕໰୊ʹͳΔͰ͠ΐ͏ʁ
50. Goroutine leak type result struct { Response *http.Response Err error } func fetch(url string, c chan <- result) { res, err := http.Get(url) c <- result{res, err} } ch := make(chan result) go fetch("https: //websiteA", ch) go fetch("https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ websiteAͱwebsiteBͷͲͪΒ͔ยํͷGoroutine͕leak͢Δ
51. Goroutine leak ରࡦ: context.ContextͱselectΛ࢖͏ func fetch(ctx context.Context, url string, result chan <- *http.Response) { res, err := http.Get(url) // ΄Μͱ͸Get΋contextΛ࢖ͬͨํ͕͍͍͚ͲྫͳͷͰׂѪ select { case c <- result{res, err} case <-ctx.Done(): // context͕׬͍ྃͯͨ͠ΒॲཧΛ΍ΊΔ } } ctx, cancel := context.WithCancel(context.Background()) defer cancel() // ׬ྃޙʹcancelͯ͠contextΛdoneঢ়ଶʹ͢Δ ch := make(chan *http.Response) go fetch(ctx, "https: //websiteA", ch) go fetch(ctx, "https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ websiteA͕׬ྃ → fmt.Println → defer cancel() → ctx.Done() → websiteBͷfetch͕ऴྃ → Goroutine͕ऴྃ
52. GoroutineΛ࢖ͬͨฒߦॲཧͷ஫ҙ఺ͱରࡦͷ·ͱΊ ‣ forͰ͸ม਺ʹ୅ೖ͢Δ ‣ sync.WaitGroupͰ׬ྃΛ଴ͭ ‣ sync.MutexͰഉଞ੍ޚΛ͢Δ ‣ ਺஋ܥ͸sync/atomic͕ศར ‣ context.ContextͰGoroutine leakΛճආ͢Δ
53. ศརͳπʔϧͷ঺հ
54. gotrace ‣ https://github.com/divan/gotrace ‣ Goroutineͱchanͷ΍ΓͱΓΛՄࢹԽͰ͖Δ
55. gotraceͷ࢖͍ํ gotraceίϚϯυΛΠϯετʔϧ͢Δ ͨͿΜmasterϒϥϯνͩͱಈ͔ͳ͍ $ go get -u https: //github.com/divan/gotrace $ cd $(GOPATH)/github.com/divan/gotrace $ git checkout -b go18 origin/go18 $ go install
56. gotraceͷ࢖͍ํ ՄࢹԽ͍ͨؔ͠਺ͷઌ಄ʹ͜ΕΛॻ͘ import "runtime/trace" f, err := os.Create("trace.out") if err != nil { log.Fatal(err) } defer f.Close() trace.Start(f) defer trace.Stop()
57. gotraceͷ࢖͍ํ gotraceͰՄࢹԽ͢Δ ύονΛ౰ͯͨGoΛ࢖͏ඞཁ͕͋ΔͷͰdockerΛ࢖͏ $ cd path/to/main # main.go͕͋ΔσΟϨΫτϦ $ docker run --rm -it \ -e GOOS=darwin \ -v (pwd):/src \ divan/golang:gotrace \ go build -o /src/binary /src/main.go $ ./binary $ gotrace ./trace.out
58. gotraceΛಈ͔ͯ͠ΈΔ func Publish(c chan <- int) { c <- 1 time.Sleep(time.Millisecond) c <- 2 time.Sleep(time.Millisecond) c <- 3 time.Sleep(time.Millisecond) } func Subscribe(c <-chan int) { for v := range c { fmt.Println(v) } } func main() { f, err := os.Create("trace.out") if err != nil { log.Fatal(err) } defer f.Close() trace.Start(f) defer trace.Stop() c := make(chan int, 0) go Publish(c) go Subscribe(c) time.Sleep(5 * time.Millisecond) }
59. gotraceΛಈ͔ͯ͠ΈΔ
60. Goroutine leakͱ͔ݟ͔ͭΔ͔΋?
61. ࠓ೔ͷ·ͱΊ ‣ Go͸ݴޠతʹฒߦॲཧΛαϙʔτ͍ͯ͠Δ ‣ ฒߦॲཧ͸೉͍͚͠Ͳ͋Δఔ౓ύλʔϯԽͰ͖Δ ‣ syncܥύοέʔδศར ‣ gotrace͸ָ͍͠(খฒײ)
62. ʕ◔ϖ◔ʔ < Goroutine΍͍͖ͬͯ