帯域制限とx/time/rate

http://connpass.com/event/39794/

Transcript

1. ଳҬ੍ݶͱ x/time/rate
   ΈΜGoϦϦύˏVOYAGE ajito
   

   View Slide

2. ౻ݪ ढ़Ұ࿠
   @fujiwara
   github.com/fujiwara
   sfujiwara.hatenablog.com
   ٕज़෦
   

   View Slide

3. 3ষΛࣥච͠·ͨ͠
   ײ૝Λblog౳Ͱ͍͚ͨͩΔͱ
   େมتͼ·͢!
   

   View Slide

4. ಛूΛࣥච͠·ͨ͠
   ײ૝Λ(ུ
   

   View Slide

5. GoͰଳҬ੍ݶ͕͍ͨ͠
   • Ϧιʔε࢖͍͗͢๷ࢭ
   • StretcherͰμ΢ϯϩʔυଳҬ੍ݶ
   • ϕϯνϚʔΫ
   • Fluentd in_tail ϕϯνϚʔΫͷͨΊͷ go-dummer
   • (ࣾ಺)ISUCON ϕϯνϚʔΧʔͰଳҬΤϛϡϨʔγϣϯ
   • io.Reader, io.Writer ΠϯλʔϑΣʔεͩͱศར
   

   View Slide

6. ͭ͘Γ·ͨ͠ (4݄)
   github.com/fujiwara/shapeio
   import "github.com/fujiwara/shapeio"
   r := shapeio.NewReader(reader) // io.Reader
   r.SetRateLimit(1024 * 10) // 10KB/sec
   // ͋ͱ͸ී௨ʹ io.Reader ͱͯ͠࢖͏
   Writer΋͋Γ·͢
   

   View Slide

7. ͦͷޙ
   

   View Slide

8. golang.org/x/time/rate !?
   ʮgolang.org/x/time/rateͰ଎౓੍ݶΛߦ͏ʯ
   http://qiita.com/lufia/items/29bf1aeb0a0fe69d16f0
   ศརͦ͏
   

   View Slide

9. x/time/rate Ͱॻ͖௚͠
   before (ଳҬ੍ޚͰ଴ͭ෦෼)
   func (l *limiter) wait(n int, err error) (int, error) {
   if n == 0 || err != nil {
   return n, err
   }
   elapsed := time.Since(l.startedAt)
   l.bytes += int64(n)
   rate := float64(l.bytes) / elapsed.Seconds()
   if rate < l.bytesPerSec {
   return n, nil
   }
   d := time.Duration(float64(l.bytes)/l.bytesPerSec*float64(time.Second) - float64(elapsed))
   time.Sleep(d)
   // reset shaping window
   l.startedAt = time.Now()
   l.bytes = 0
   return n, nil
   }
   

   View Slide

10. x/time/rate Ͱॻ͖௚͠
    after
    func (s *Writer) Write(p []byte) (int, error) {
    if s.limiter == nil {
    return s.w.Write(p)
    }
    n, err := s.w.Write(p)
    if err != nil {
    return n, err
    }
    // ଴ͭͱ͜Ζ͸͚ͩ͜͜
    if err := s.limiter.WaitN(s.ctx, n); err != nil {
    return n, err
    }
    return n, err
    }
    

    View Slide

11. x/time/rate
    1ඵؒʹཷ·Δtokenͷ਺ͱɺ࠷େͰཷΊΔ਺Λࢦఆ
    limiter := rate.NewLimiter(rate.Limit(m), burst)
    nݸͷtokenΛফඅ͢Δ(ཷ·ͬͯͳ͚Ε͹ཷ·Δ·Ͱ଴ͭ)
    err := limiter.WaitN(ctx, n)
    nݸͷtoken͕ཷ·ͬͯΔ͔֬ೝ(ཷ·ͬͯΕ͹ফඅͯ͠true)
    ok := limiter.AllowN(time.Now(), n)
    

    View Slide

12. x/time/rate ಋೖͷ෭࢈෺
    context ͕࢖͑ΔΑ͏ʹ
    ctx, cancel := context.WithTimeout(context.Background(), 3*time.Seconds)
    r := shapeio.NewReaderWithContext(reader, ctx)
    shapeio.SetRateLimit(1024 * 1024)
    io.Copy(w, r)
    ...
    ଳҬ੍ݶͭͭ͠λΠϜΞ΢τ΍Ωϟϯηϧ΋
    

    View Slide

13. ·ͱΊ
    x/time/rate ศརʂ
    shapeio ΋ΑΖ͘͠
    

    View Slide