ホリネズミでもわかるGoroutine入門 / golang.tokyo#14

0f1d16574e6d15530b0e9e9b837d1d86?s=47 morikuni
April 16, 2018

ホリネズミでもわかるGoroutine入門 / golang.tokyo#14

0f1d16574e6d15530b0e9e9b837d1d86?s=128

morikuni

April 16, 2018
Tweet

Transcript

  1. ϗϦωζϛͰ΋Θ͔Δ Goroutineೖ໳ golang.tokyo#14 2018/04/16

  2. ࣗݾ঺հ ‣ Name: ৿ᅳ ହฏ (Morikuni Taihei) ‣ GitHub: @morikuni

    ‣ Twitter: @inukirom ‣ ॴଐ ‣ גࣜձࣾϝϧΧϦ ‣ GoͰόοΫΤϯυ։ൃ
  3. ຊ೔ͷ಺༰ ॳ৺ऀ͕GoroutineΛ࢖͑ΔΑ͏ʹͳΔͨΊͷ࿩ ‣ Goʹ͓͚Δฒߦॲཧʹ͍ͭͯ ‣ GoroutineΛ࢖ͬͨฒߦॲཧͷ஫ҙ఺ͱରࡦ ‣ ศརͳπʔϧͷ঺հ

  4. Goʹ͓͚Δฒߦॲཧʹ͍ͭͯ

  5. Goʹ૊Έࠐ·Ε͍ͯΔฒߦॲཧͷͨΊͷػೳ ‣ Goroutine ‣ chan ‣ select

  6. Goroutineʹ͍ͭͯ

  7. Goroutineͱ͸ ‣ GoΛ୅ද͢Δػೳͷ1ͭ ‣ ฒߦॲཧΛߦ͏ͨΊͷػೳ(CPU͕ෳ਺͋Ε͹ฒྻ࣮ߦ΋Ͱ͖Δ) ‣ ϓϩηε΍εϨουΑΓ΋ܰྔ ‣ mainؔ਺΋Goroutineͱ࣮ͯ͠ߦ͞Ε͍ͯΔ

  8. Goroutineͷىಈ go Λ͚ͭͯؔ਺Λݺͼग़͢͜ͱͰGoroutine͕ىಈ͢Δ go Function() go obj.Method() go func() {

    fmt.Println("hello goroutine") }() go func(name string) { fmt.Println("hello "+name) }("morikuni") // Ҿ਺΋౉ͤΔ
  9. ࣮ߦॱং͸อূ͞Εͳ͍ go func() { fmt.Println("goroutine 1") }() go func() {

    fmt.Println("goroutine 2") }() go func() { fmt.Println("goroutine 3") }() // goroutine 3 // goroutine 1 // goroutine 2
  10. GoroutineͱίʔϧελοΫ ‣ Goroutine͕ىಈͨ࣌͠఺Ͱؔ਺ͷίʔϧελοΫ͕෼཭͢Δ ‣ panic͸GoroutineͷίʔϧελοΫΛ໭͍ͬͯ͘ ‣ ͭ·Γɺdefer & recover͸panic͕ൃੜͨ͠Goroutine಺Ͱ࢖͏ඞཁ͕͋Δ ‣

    Devquiz࿮͸͜Ε͕ϋϚΓϙΠϯτ
  11. GoroutineͱίʔϧελοΫ GoroutineΛ࢖Θͳ͍৔߹ͷίʔϧελοΫ func A() { B() } func B() {

    defer func(){}() C() } func C() { D() } func D() { panic("dead") }
  12. GoroutineͱίʔϧελοΫ func A() { func B() { defer func(){}() func

    C() { func D() { panic("dead") } } } } ίʔϧελοΫ: A → B → C → D Bͷdefer͕࣮ߦ͞ΕΔ
  13. GoroutineͱίʔϧελοΫ GoroutineΛ࢖ͬͨ৔߹ͷίʔϧελοΫ func A() { B() } func B() {

    defer func(){}() go C() // goͰݺͼग़͢ } func C() { D() } func D() { panic("dead") }
  14. GoroutineͱίʔϧελοΫ func A() { func B() { defer func(){}() }

    } func C() { func D() { panic("dead") } } ίʔϧελοΫ: C → D Bͷdefer͸࣮ߦ͞Εͳ͍
  15. chanʹ͍ͭͯ

  16. chanͱ͸ ‣ Goroutineؒͷσʔλͷૹड৴΍ॲཧͷಉظΛߦ͏ ‣ FIFOͷΩϡʔ(࠷ॳʹೖΕͨ஋͕࠷ॳʹऔΓग़͞ΕΔ) ‣ ༰ྔΛ࣋ͭ ‣ 3छྨͷܕ͕͋Δ ‣

    Read Write ‣ Read only ‣ Write only
  17. chanͷ࢖͍ํ // chanͷ࡞੒ c := make(chan int) // chan΁ͷॻ͖ࠐΈ c

    <- 1 // chan͔ΒͷಡΈࠐΈ x := <- c for x := range c { ... } // chanΛด͡Δ close(c) ดͨ͡chan͔Β͸θϩ஋͕ಡΈࠐΊΔΑ͏ʹͳΓɺforϧʔϓ͕ऴྃ͢Δ ดͨ͡chanʹॻ͖ࠐΉͱpanic͢Δ
  18. chanͷܕ c := make(chan int) // RW chan func Publish(c

    chan <- int) { // WO chan ... } func Subscribe(c <-chan int) { // RO chan ... } // RW chan͸WOͱROʹ୅ೖՄೳ Publish(c) Subscribe(c)
  19. selectʹ͍ͭͯ

  20. selectͱ͸ ‣ ෳ਺ͷchan΁ͷૢ࡞Λѻ͏ͨΊͷ΋ͷ ‣ ࣮ߦՄೳʹͳͬͨchan΁ͷૢ࡞Λ1͚࣮ͭͩߦ͢Δ ‣ defaultΛॻ͘͜ͱͰchanͷϒϩοΫΛ๷͛Δ

  21. selectͷ࢖͍ํ select { case v := <-c1: fmt.Println(v) case c2

    <- "hello": } c1͔ΒͷಡΈࠐΈ or c2΁ͷॻ͖ࠐΈͷͲͪΒ͔Λ࣮ߦ͢Δ ࣮ߦͰ͖Δ·ͰϒϩοΫ͢Δ
  22. selectͷ࢖͍ํ select { case v := <-c1: fmt.Println(v) case c2

    <- "hello": default: fmt.Println("not ready") } c1͔ΒͷಡΈࠐΈ΋c2΁ͷॻ͖ࠐΈ΋Ͱ͖ͳ͍৔߹ʹdefault͕࣮ߦ͞ΕΔ
  23. ͜͜·Ͱ͕Goʹ͓͚Δฒߦॲཧͷجૅ

  24. GoroutineΛ࢖ͬͨ ฒߦॲཧͷ஫ҙ఺ͱରࡦ

  25. Waitॲཧ

  26. Waitॲཧ func main() { go func() { fmt.Println("Hello World") }()

    } ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
  27. Waitॲཧ $ go run main.go (ͳʹ΋ग़ྗ͞Εͳ͍) ͳͥʁ

  28. Waitॲཧ Goroutineͷॲཧ͕࣮ߦ͞ΕΔલʹϓϩηε͕ऴྃͯ͠͠·͏͔Β func main() { go func() { fmt.Println("Hello World")

    }() }
  29. Waitॲཧ ରࡦ1: Sleep͢Δ func main() { go func() { fmt.Println("Hello

    World") }() time.Sleep(time.Second) } ॲཧ͕ऴΘͬͨͷʹ଴ͪଓ͚Δ͜ͱʹͳΔ ॻ͖ࣺͯͷίʔυͰΑ͘࢖͏
  30. Waitॲཧ ରࡦ2: channelΛ࢖͏ func main() { c := make(chan struct{})

    go func() { defer func() { c <- struct{}{} }() fmt.Println("Hello World") }() <-c // close͞ΕΔ·Ͱ଴ͭ } chanͰॲཧͷ׬ྃΛ௨஌͢Δ deferͰඞ࣮ͣߦ͞ΕΔΑ͏ʹ͢Δ chan boolΛ࢖͏Α͏ͳྫ΋͋Δ͚Ͳɺstruct{}ͷ΄͏͕Ұൠత
  31. Waitॲཧ ରࡦ3: sync.WaitGroupΛ࢖͏ func main() { var wg sync.WaitGroup wg.Add(1)

    // GoroutineΛݺͼग़͢લʹAdd͢Δ go func() { defer wg.Done() // GoroutineͷதͰdeferΛ࢖ͬͯDoneΛݺͼग़͢ fmt.Println("Hello World") }() wg.Wait() } WaitGroup͸಺෦ͷΧ΢ϯλ͕0ʹͳΔ·Ͱwait͢Δ Done() = Add(-1)ͳͷͰAddͨ͠਺͚ͩDone͢Δඞཁ͕͋Δ
  32. Waitॲཧͷ·ͱΊ ‣ αϯϓϧίʔυͳͲ͸time.Sleepָ͕ ‣ جຊతʹsync.WaitGroupΛ࢖͍ͬͯΕ͹͍͍ ‣ chan͸ෳ਺ͷGoroutineͷ׬ྃΛ଴ͭͷʹ͸࢖͍ͮΒ͍ ‣ golang.org/x/sync/errgroupͱ͍͏WaitGroupʹΤϥʔॲཧΛ௥Ճͨ͠΋ ͷ΋͋Δ

  33. Goroutineͱforϧʔϓ

  34. Goroutineͱforϧʔϓ 1, 2, 3, 4, 5Λग़ྗ͍ͨ͠(ॱෆಉ) xs := []int{1, 2,

    3, 4, 5} for _, x := range xs { go func() { fmt.Println(x) }() } time.Sleep(time.Second) ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
  35. Goroutineͱforϧʔϓ $ go run main.go 5 5 5 5 5

    ͳͥʁ
  36. Goroutineͱforϧʔϓ x͸ϧʔϓຖʹ஋Λ্ॻ͖͞Εͯ࢖͍ճ͞ΕΔ͔Β xs := []int{1, 2, 3, 4, 5} for

    _, x := range xs { go func() { fmt.Println(x) }() } time.Sleep(time.Second)
  37. Goroutineͱforϧʔϓ forͷ࣮૷Πϝʔδ func iterate(xs []int, x *int) func() { i

    := -1 return func() { i ++ *x = xs[i] } } xs := []int{1, 2, 3, 4, 5} var x int next := iterator(xs, &x) for i := 0; i < len(xs); i ++ { next() go func() { fmt.Println(x) }() }
  38. Goroutineͱforϧʔϓ ରࡦ: Ұ౓ม਺ʹ୅ೖ͢Δ for _, x := range xs {

    go func(x int) { // ؔ਺ͷҾ਺Ͱ͏͚ͱΔ fmt.Println(x) }(x) } for _, x := range xs { x := x // ผͷม਺ʹ୅ೖ͢Δ go func() { fmt.Println(x) }() }
  39. ڝ߹ঢ়ଶ

  40. ڝ߹ঢ়ଶ xsͷ஋Λ2ഒͯ͠doubleʹॻ͖ࠐΈ͍ͨ(ॱෆಉ) xs := []int{1, 2, 3, 4, 5} var

    double []int for _, x := range xs { go func(x int) { double = append(double, x*2) }(x) } time.Sleep(time.Second) fmt.Println(double) ͳʹ͕ग़ྗ͞ΕΔͰ͠ΐ͏ʁ
  41. ڝ߹ঢ়ଶ $ go run main.go [2 10] $ go run

    main.go [4 2 6 8 10] $ go run main.go [4 2 8] ͳͥʁ
  42. ڝ߹ঢ়ଶ doubleΛಡΈࠐΜͰɺॻ͖ࠐΉ·Ͱʹdouble͕ߋ৽͞Ε͍ͯΔ͔Β for _, x := range xs { go

    func(x int) { double = append(double, x*2) }(x) } ࣮ࡍʹ͸ಡΈࠐΈ͔Βॻ͖ࠐΈ·Ͱʹ͕͔͔͍࣌ؒͬͯΔ for _, x := range xs { go func(x int) { tmp := double tmp = append(tmp, x*2) double = tmp }(x) }
  43. ڝ߹ঢ়ଶ R: double͔ΒಡΈࠐΜͩ஋ W: doubleʹॻ͖ࠐΉ஋

  44. ڝ߹ঢ়ଶ ରࡦ1: ௚઀Ͱॲཧ͢Δ ‣ ฒߦॲཧΛ͢Δͱڝ߹͕͓͖ΔͷͰɺഉଞ੍ޚΛߦ͏ ‣ ഉଞ੍ޚΛҰ൪؆୯ʹ΍Δํ๏͸ɺ௚ྻͰॲཧ͢Δ͜ͱͳͷͰຊ౰ʹGoroutine ࢖͏ඞཁ͕͋Δ͔Λݕ౼ͯ͠ΈΔͱ͍͍

  45. ڝ߹ঢ়ଶ ରࡦ2: sync.MutexΛ࢖͏ var mu sync.Mutex xs := []int{1, 2,

    3, 4, 5} for _, x := range xs { go func(x int) { mu.Lock() defer mu.Unlock() double = append(double, x*2) }(x) } Ұ౓ʹmu.Lock()Ͱ͖Δͷ͸1Օॴ͚ͩ ଞͷՕॴ͸mu.Unlock()͞ΕΔ·ͰϒϩοΫ͞ΕΔ sync.RWMutexΛ࢖͏ͱɺಡΈࠐΈͱॻ͖ࠐΈͷLockΛ෼͚ͯಡΈࠐΈ͸ฒߦͰ ߦ͑Δ
  46. ڝ߹ঢ়ଶ ओʹ਺஋ܥ: sync/atomicύοέʔδΛ࢖͏ var count int64 atomic.AddInt64(&count, 1) // countʹ1Λ଍͢

    atomic.CompareAndSwapInt64(&count, 1, 10) // countͷ஋͕1ͳΒ10ʹ͢Δ v := atomic.LoadInt64(&count) // countͷ஋ΛಡΈࠐΉ atomic.StoreInt64(&count, 100) // countͷ஋Λ100ʹ͢Δ float64ͳͲ͸αϙʔτ͞Ε͍ͯͳ͍͕ؤுΕ͹࢖͑Δ (floatΛ಺෦తʹ͸uintͱͯ͠ѻ͍ɺmath.Float64frombitsͱ atomic.LoadUint64ͳͲΛ૊Έ߹ΘͤΔ)
  47. ڝ߹ঢ়ଶͷ·ͱΊ ‣ ௚ྻॲཧ(Goroutine͔ͭΘͳ͍)Ͱ͸μϝ͔ݕ౼ ‣ ਺஋ܥ͸sync/atomic͕࢖͑ΔՄೳੑ͕͋Δ ‣ ͦΕҎ֎͸sync.Mutex, sync.RWMutexΛ࢖͑͹େମ໰୊ͳ͍

  48. Goroutine leak

  49. Goroutine leak websiteAͱwebsiteBͷͲͪΒ͔ͷ݁ՌΛग़ྗ͍ͨ͠ type result struct { Response *http.Response Err

    error } func fetch(url string, c chan <- result) { res, err := http.Get(url) c <- result{res, err} } ch := make(chan result) go fetch("https: //websiteA", ch) go fetch("https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ ͳʹ͕໰୊ʹͳΔͰ͠ΐ͏ʁ
  50. Goroutine leak type result struct { Response *http.Response Err error

    } func fetch(url string, c chan <- result) { res, err := http.Get(url) c <- result{res, err} } ch := make(chan result) go fetch("https: //websiteA", ch) go fetch("https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ websiteAͱwebsiteBͷͲͪΒ͔ยํͷGoroutine͕leak͢Δ
  51. Goroutine leak ରࡦ: context.ContextͱselectΛ࢖͏ func fetch(ctx context.Context, url string, result

    chan <- *http.Response) { res, err := http.Get(url) // ΄Μͱ͸Get΋contextΛ࢖ͬͨํ͕͍͍͚ͲྫͳͷͰׂѪ select { case c <- result{res, err} case <-ctx.Done(): // context͕׬͍ྃͯͨ͠ΒॲཧΛ΍ΊΔ } } ctx, cancel := context.WithCancel(context.Background()) defer cancel() // ׬ྃޙʹcancelͯ͠contextΛdoneঢ়ଶʹ͢Δ ch := make(chan *http.Response) go fetch(ctx, "https: //websiteA", ch) go fetch(ctx, "https: //websiteB", ch) fmt.Println( <-ch) // ࠷ॳʹฦ͖ͬͯͨํ͚ͩΛ࢖͏ websiteA͕׬ྃ → fmt.Println → defer cancel() → ctx.Done() → websiteBͷfetch͕ऴྃ → Goroutine͕ऴྃ
  52. GoroutineΛ࢖ͬͨฒߦॲཧͷ஫ҙ఺ͱରࡦͷ·ͱΊ ‣ forͰ͸ม਺ʹ୅ೖ͢Δ ‣ sync.WaitGroupͰ׬ྃΛ଴ͭ ‣ sync.MutexͰഉଞ੍ޚΛ͢Δ ‣ ਺஋ܥ͸sync/atomic͕ศར ‣

    context.ContextͰGoroutine leakΛճආ͢Δ
  53. ศརͳπʔϧͷ঺հ

  54. gotrace ‣ https://github.com/divan/gotrace ‣ Goroutineͱchanͷ΍ΓͱΓΛՄࢹԽͰ͖Δ

  55. gotraceͷ࢖͍ํ gotraceίϚϯυΛΠϯετʔϧ͢Δ ͨͿΜmasterϒϥϯνͩͱಈ͔ͳ͍ $ go get -u https: //github.com/divan/gotrace $

    cd $(GOPATH)/github.com/divan/gotrace $ git checkout -b go18 origin/go18 $ go install
  56. gotraceͷ࢖͍ํ ՄࢹԽ͍ͨؔ͠਺ͷઌ಄ʹ͜ΕΛॻ͘ import "runtime/trace" f, err := os.Create("trace.out") if err

    != nil { log.Fatal(err) } defer f.Close() trace.Start(f) defer trace.Stop()
  57. gotraceͷ࢖͍ํ gotraceͰՄࢹԽ͢Δ ύονΛ౰ͯͨGoΛ࢖͏ඞཁ͕͋ΔͷͰdockerΛ࢖͏ $ cd path/to/main # main.go͕͋ΔσΟϨΫτϦ $ docker

    run --rm -it \ -e GOOS=darwin \ -v (pwd):/src \ divan/golang:gotrace \ go build -o /src/binary /src/main.go $ ./binary $ gotrace ./trace.out
  58. gotraceΛಈ͔ͯ͠ΈΔ func Publish(c chan <- int) { c <- 1

    time.Sleep(time.Millisecond) c <- 2 time.Sleep(time.Millisecond) c <- 3 time.Sleep(time.Millisecond) } func Subscribe(c <-chan int) { for v := range c { fmt.Println(v) } } func main() { f, err := os.Create("trace.out") if err != nil { log.Fatal(err) } defer f.Close() trace.Start(f) defer trace.Stop() c := make(chan int, 0) go Publish(c) go Subscribe(c) time.Sleep(5 * time.Millisecond) }
  59. gotraceΛಈ͔ͯ͠ΈΔ

  60. Goroutine leakͱ͔ݟ͔ͭΔ͔΋?

  61. ࠓ೔ͷ·ͱΊ ‣ Go͸ݴޠతʹฒߦॲཧΛαϙʔτ͍ͯ͠Δ ‣ ฒߦॲཧ͸೉͍͚͠Ͳ͋Δఔ౓ύλʔϯԽͰ͖Δ ‣ syncܥύοέʔδศར ‣ gotrace͸ָ͍͠(খฒײ)

  62. ʕ◔ϖ◔ʔ < Goroutine΍͍͖ͬͯ