contextによるキャンセル処理 / Go Context cancel

Transcript

1. contextʹΑΔΩϟϯηϧॲཧ Yoshima Takatada / @shiimaxx Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
2. About me • Yoshima Takatada / @shiimaxx • h1ps:/ /shiimaxx.com • גࣜձࣾϋʔτϏʔπ • ٕज़։ൃࣨ ΤϯδχΞ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
3. h"ps:/ /github.com/gopherdojo/dojo1/tree/master/kadai3 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
4. ՝୊3-2 ෼ׂμ΢ϯϩʔυΛߦ͏ • ! RangeΞΫηεΛ༻͍Δ • ! ͍͔ͭ͘ͷΰϧʔνϯͰμ΢ϯϩʔυͯ͠Ϛʔδ͢Δ • ! ΤϥʔॲཧΛ޻෉͢Δ • ! golang.org/x/sync/errgourpύοέʔδͳͲΛ࢖ͬͯΈΔ • " Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
5. Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ eg, ctx := errgroup.WithContext(context.TODO()) tempFiles := make([]*os.File, c.Parallel) for p := 0; p < c.Parallel; p++ { s := p * chunkSize e := s + (chunkSize - 1) if p == c.Parallel-1 { e += surplus } i := p eg.Go(func() error { return rangeGet(ctx, url, s, e, i, tempFiles) }) } h"ps:/ /github.com/gopherdojo/dojo1/blob/kadai3-2-shiimaxx/kadai3/shiimaxx/range-access/gurl/gurl.go#L79-L92 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
6. Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ʁ func rangeGet(ctx context.Context, url string, s, e, i int, tempFiles []*os.File) error { client := &http.Client{} req, err := http.NewRequest("GET", url, nil) if err != nil { return err } req.Header.Add("Range", fmt.Sprintf("bytes=%d-%d", s, e)) resp, err := client.Do(req) if err != nil { return err } defer resp.Body.Close() reader, err := ioutil.ReadAll(resp.Body) if err != nil { return err } tempFile, err := ioutil.TempFile("./", "temp") if err != nil { return err } if err := ioutil.WriteFile(tempFile.Name(), reader, 0644); err != nil { return err } tempFiles[i] = tempFile return nil } Ωϟϯηϧॲཧ͕Ͳ͜ʹ΋ͳ͍……ͦ΋ͦ΋Ҿ਺Ͱड͚औͬͨctxΛ࢖͍ͬͯͳ͍…… h"ps:/ /github.com/gopherdojo/dojo1/blob/kadai3-2-shiimaxx/kadai3/shiimaxx/range-access/gurl/gurl.go#L35-L62 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
7. Ϟνϕʔγϣϯ • ՝୊3-2 ෼ׂμ΢ϯϩʔυΛ࣮૷͠Α͏ • ! Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ • Ωϟϯηϧॲཧ -> contextΛ࢖͏ͱ͍͏Πϝʔδ͸͋ͬͨ • Ωϟϯηϧॲཧʹ͓͍ͯcontext͕ԿΛͯ͘͠ΕΔͷ͔ΛͪΌΜͱ ཧղ͍ͯ͠ͳ͔ͬͨ • contextΛཧղͯ͠࢖͑ΔΑ͏ʹͳΖ͏ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
8. contextͷऔಘ ctx, cancel := context.WithCancel(context.Background()) • func Background() Context • ۭίϯςΩετΛฦ͢ • emptyCtxܕ • func WithCancel(parent Context) (ctx Context, cancel CancelFunc) • ContextΠϯλʔϑΣΠεΛҾ਺ʹͱΓɺcontextͱΩϟϯηϧ༻ͷؔ਺Λฦ͢ • ฦ͞Εͨcontextͷdoneνϟϯωϧ͸ɺΩϟϯηϧ༻ͷؔ਺͕ݺͼग़͞Εͨͱ͖ɺ΋͘͠͸਌contex ͷdoneνϟϯωϧ͕Ϋϩʔζͨ͠ͱ͖ʹΫϩʔζ͞ΕΔ • cancelCtxܕ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
9. emptyCtx // An emptyCtx is never canceled, has no values, and has no deadline. It is not // struct{}, since vars of this type must have distinct addresses. type emptyCtx int func (*emptyCtx) Deadline() (deadline time.Time, ok bool) { return } func (*emptyCtx) Done() <-chan struct{} { return nil } func (*emptyCtx) Err() error { return nil } func (*emptyCtx) Value(key interface{}) interface{} { return nil } • h#ps:/ /github.com/golang/go/blob/release-branch.go1.10/src/context/context.go#L167-L195 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
10. cancelCtx // A cancelCtx can be canceled. When canceled, it also cancels any children // that implement canceler. type cancelCtx struct { Context mu sync.Mutex // protects following fields done chan struct{} // created lazily, closed by first cancel call children map[canceler]struct{} // set to nil by the first cancel call err error // set to non-nil by the first cancel call } func (c *cancelCtx) Done() <-chan struct{} { c.mu.Lock() if c.done == nil { c.done = make(chan struct{}) } d := c.done c.mu.Unlock() return d } func (c *cancelCtx) Err() error { c.mu.Lock() defer c.mu.Unlock() return c.err } func (c *cancelCtx) String() string { return fmt.Sprintf("%v.WithCancel", c.Context) } • h#ps:/ /github.com/golang/go/blob/release-branch.go1.10/src/context/context.go#L314-L343 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
11. cancelCtx // cancel closes c.done, cancels each of c's children, and, if // removeFromParent is true, removes c from its parent's children. func (c *cancelCtx) cancel(removeFromParent bool, err error) { if err == nil { panic("context: internal error: missing cancel error") } c.mu.Lock() if c.err != nil { c.mu.Unlock() return // already canceled } c.err = err if c.done == nil { c.done = closedchan } else { close(c.done) } for child := range c.children { // NOTE: acquiring the child's lock while holding parent's lock. child.cancel(false, err) } c.children = nil c.mu.Unlock() if removeFromParent { removeChild(c.Context, c) } } • h#ps:/ /github.com/golang/go/blob/release-branch.go1.10/src/context/context.go#L345-L372 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
12. Ωϟϯηϧͷ଴ػ // ΩϟϯηϧΛ଴ػ <-ctx.Done() • func (c *cancelCtx) Done() <-chan struct{} • cancelCtx͕ϑΟʔϧυʹ࣋ͭdoneνϟϯωϧΛ make(chan struct{})ͰॳظԽͯ͠ฦ͢ • ͜ͷνϟϯωϧΛड৴͢Δ·ͰϒϩοΫ͞ΕΔ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
13. cancelCtx.Done() func (c *cancelCtx) Done() <-chan struct{} { c.mu.Lock() if c.done == nil { c.done = make(chan struct{}) } d := c.done c.mu.Unlock() return d } • c.doneΛdʹ୅ೖͨ͋͠ͱɺdΛreturn͍ͯ͠Δ • chan͸ࢀরܕͳͷͰࢀরΛฦ͍ͯ͠Δ • h*ps:/ /github.com/golang/go/blob/release-branch.go1.10/src/context/context.go#L325-L333 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
14. Ωϟϯηϧͷ࣮ߦ // contextΛऔಘ ctx, cancel := context.WithCancel(context.Background()) : // Ωϟϯηϧ cancel() • cancel͸context.WithCancel()ͷೋͭ໨ͷฦΓ஋ͱͯ͠ड͚औ͍ͬͯΔ • context.WithCancelͷ2ͭΊͷฦΓ஋͸ func() { c.cancel(true, Canceled) } • cancelCtxʹ࣮૷͞Ε͍ͯΔcancelϝιουͷ࣮૷ΛݟΔ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
15. cancelCtx.cancel() if c.done == nil { c.done = closedchan } else { close(c.done) } for child := range c.children { // NOTE: acquiring the child's lock while holding parent's lock. child.cancel(false, err) } • c.done == nilͰ͸ͳ͍৔߹ɺc.done͕Ϋϩʔζ͞ΕΔ • c.doneΛΫϩʔζ͢Δͱ<- ctx.Done()Ͱ଴ػ͍ͯͨ͠ՕॴͰड৴͢Δ • ͭ·ΓνϟϯωϧΛΫϩʔζ͢Δ͜ͱͰΩϟϯηϧΛ௨஌͍ͯ͠Δ • ࣗ਎ͷࢠcontext(c.children)΋Ϋϩʔζ͍ͯ͠Δ h"ps:/ /github.com/golang/go/blob/release-branch.go1.10/src/context/context.go#L357-L365 Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
16. context • context͸Ωϟϯηϧͷ఻ൖΛߦ͏ • Ωϟϯηϧʹ൐͏ॲཧͷதஅɺϦιʔε։์ͳͲ͸ద੾ʹ࣮૷ ͢Δඞཁ͕͋Δ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
17. ՝୊3-2 ෼ׂμ΢ϯϩʔυΛߦ͏ • ! RangeΞΫηεΛ༻͍Δ • ! ͍͔ͭ͘ͷΰϧʔνϯͰμ΢ϯϩʔυͯ͠Ϛʔδ͢Δ • ! ΤϥʔॲཧΛ޻෉͢Δ • ! golang.org/x/sync/errgourpύοέʔδͳͲΛ࢖ͬͯΈΔ • " Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
18. Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ • ෼ׂμ΢ϯϩʔυͷͨΊෳ਺ͷgorou&neΛىಈͯ͠HTTPϦΫ ΤετΛ࣮ߦ͢Δ • HTTPϦΫΤετͰ1ͭͰ΋Τϥʔ͕ൃੜͨ͠৔߹͸ଞͷ͢΂ͯ ͷϦΫΤετΛΩϟϯηϧ͢Δ • golang.org/x/sync/errgourpΛ࢖͏ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
19. golang.org/x/sync/errgourp • func WithContext(ctx context.Context) (*Group, context.Context) • Ωϟϯηϧ༻ͷؔ਺ΛcancelϑΟʔϧυʹࢦఆͨ͠Groupߏ଄ମͷΞυϨεͱcontextΛฦ͢ • context͸context.WithCancel()Ͱऔಘ͍ͯ͠Δ • Ωϟϯηϧ͕࣮ߦͰ͖Δ • func (g *Group) Go(f func() error) • Ҿ਺ʹ౉ͨؔ͠਺͕ΤϥʔΛฦͨ͠৔߹ʹࣗ਎ͷcancelϑΟʔϧυʹ΋ͭΩϟϯηϧ༻ͷؔ਺Λ࣮ߦ͢Δ • ͭ·Γ໌ࣔతʹcancel()Λݺͼग़͢ඞཁ͸ͳ͍ • GoϝιουͰ࣮ߦ͢Δؔ਺಺ͰΩϟϯηϧ௨஌Λड͚औΕΔΑ͏ʹ͢Δඞཁ͕͋Δ h"ps:/ /github.com/golang/sync/blob/master/errgroup/errgroup.go Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
20. Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ func rangeGet(ctx context.Context, url string, s, e, i int, tempFiles []*os.File) error { client := &http.Client{} req, err := http.NewRequest("GET", url, nil) if err != nil { return err } req.Header.Add("Range", fmt.Sprintf("bytes=%d-%d", s, e)) req.WithContext(ctx) resp, err := client.Do(req) if err != nil { return err } : func (c *Client) Get(url string) error { : eg, ctx := errgroup.WithContext(context.Background()) tempFiles := make([]*os.File, c.Parallel) : eg.Go(func() error { return rangeGet(ctx, url, s, e, i, tempFiles) }) } : Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
21. Ωϟϯηϧ͕ൃੜͨ͠৔߹ͷ࣮૷Λߦ͏ • ࣗ෼ͷ৔߹ɺGet()ͷeg.Go()Ͱݺͼग़͢rangeGet()಺Ͱ HTTPϦΫΤετΛ࣮ߦ͢Δ࣮૷ʹͳ͍ͬͯΔ • ඪ४ϥΠϒϥϦͷnet/h*pͰ͸ɺhttp.Requestʹ WithContext()ϝιουͰcontextΛ౉͢͜ͱ͕Ͱ͖Δ • rangeGet()ʹ౉ͨ͠contextΛreq.WithContext(ctx)ͰϦ ΫΤετʹඥ෇͚Δ͜ͱͰΩϟϯηϧΛड৴Ͱ͖ΔΑ͏ʹͨ͠ Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx
22. ·ͱΊ • context͸Ωϟϯηϧͷ఻ൖΛߦ͏ • Ωϟϯηϧʹ൐͏ॲཧͷதஅɺϦιʔε։์ͳͲ͸ద੾ʹ࣮૷͢ Δඞཁ͕͋Δ • <- ctx.Done()Ͱड৴ͨ͠৔߹ͷॲཧΛॻ͘ • ϥΠϒϥϦ͕contextΛαϙʔτ͍ͯ͠Δ৔߹͸ͦΕΛར༻͢Δ • Ԟ͕ਂͦ͏ͳͷͰ·ͩ·ͩ͜Ε͔Β Gopherಓ৔#1 LTେձ 2018/05/28 - Yoshima Takatada / @shiimaxx