帯域制限とx/time/rate

http://connpass.com/event/39794/

Transcript

1. ଳҬ੍ݶͱ x/time/rate ΈΜGoϦϦύˏVOYAGE ajito
2. ౻ݪ ढ़Ұ࿠ @fujiwara github.com/fujiwara sfujiwara.hatenablog.com ٕज़෦
3. 3ষΛࣥච͠·ͨ͠ ײ૝Λblog౳Ͱ͍͚ͨͩΔͱ େมتͼ·͢!
4. ಛूΛࣥච͠·ͨ͠ ײ૝Λ(ུ
5. GoͰଳҬ੍ݶ͕͍ͨ͠ • Ϧιʔε࢖͍͗͢๷ࢭ • StretcherͰμ΢ϯϩʔυଳҬ੍ݶ • ϕϯνϚʔΫ • Fluentd in_tail ϕϯνϚʔΫͷͨΊͷ go-dummer • (ࣾ಺)ISUCON ϕϯνϚʔΧʔͰଳҬΤϛϡϨʔγϣϯ • io.Reader, io.Writer ΠϯλʔϑΣʔεͩͱศར
6. ͭ͘Γ·ͨ͠ (4݄) github.com/fujiwara/shapeio import "github.com/fujiwara/shapeio" r := shapeio.NewReader(reader) // io.Reader r.SetRateLimit(1024 * 10) // 10KB/sec // ͋ͱ͸ී௨ʹ io.Reader ͱͯ͠࢖͏ Writer΋͋Γ·͢
7. ͦͷޙ
8. golang.org/x/time/rate !? ʮgolang.org/x/time/rateͰ଎౓੍ݶΛߦ͏ʯ http://qiita.com/lufia/items/29bf1aeb0a0fe69d16f0 ศརͦ͏
9. x/time/rate Ͱॻ͖௚͠ before (ଳҬ੍ޚͰ଴ͭ෦෼) func (l *limiter) wait(n int, err error) (int, error) { if n == 0 || err != nil { return n, err } elapsed := time.Since(l.startedAt) l.bytes += int64(n) rate := float64(l.bytes) / elapsed.Seconds() if rate < l.bytesPerSec { return n, nil } d := time.Duration(float64(l.bytes)/l.bytesPerSec*float64(time.Second) - float64(elapsed)) time.Sleep(d) // reset shaping window l.startedAt = time.Now() l.bytes = 0 return n, nil }
10. x/time/rate Ͱॻ͖௚͠ after func (s *Writer) Write(p []byte) (int, error) { if s.limiter == nil { return s.w.Write(p) } n, err := s.w.Write(p) if err != nil { return n, err } // ଴ͭͱ͜Ζ͸͚ͩ͜͜ if err := s.limiter.WaitN(s.ctx, n); err != nil { return n, err } return n, err }
11. x/time/rate 1ඵؒʹཷ·Δtokenͷ਺ͱɺ࠷େͰཷΊΔ਺Λࢦఆ limiter := rate.NewLimiter(rate.Limit(m), burst) nݸͷtokenΛফඅ͢Δ(ཷ·ͬͯͳ͚Ε͹ཷ·Δ·Ͱ଴ͭ) err := limiter.WaitN(ctx, n) nݸͷtoken͕ཷ·ͬͯΔ͔֬ೝ(ཷ·ͬͯΕ͹ফඅͯ͠true) ok := limiter.AllowN(time.Now(), n)
12. x/time/rate ಋೖͷ෭࢈෺ context ͕࢖͑ΔΑ͏ʹ ctx, cancel := context.WithTimeout(context.Background(), 3*time.Seconds) r := shapeio.NewReaderWithContext(reader, ctx) shapeio.SetRateLimit(1024 * 1024) io.Copy(w, r) ... ଳҬ੍ݶͭͭ͠λΠϜΞ΢τ΍Ωϟϯηϧ΋
13. ·ͱΊ x/time/rate ศརʂ shapeio ΋ΑΖ͘͠