Upgrade to Pro — share decks privately, control downloads, hide ads and more …

帯域制限とx/time/rate

FUJIWARA Shunichiro
September 16, 2016
570

 帯域制限とx/time/rate

FUJIWARA Shunichiro

September 16, 2016
Tweet

More Decks by FUJIWARA Shunichiro

Transcript

  1. ଳҬ੍ݶͱ x/time/rate
    ΈΜGoϦϦύˏVOYAGE ajito

    View Slide

  2. ౻ݪ ढ़Ұ࿠
    @fujiwara
    github.com/fujiwara
    sfujiwara.hatenablog.com
    ٕज़෦

    View Slide

  3. 3ষΛࣥච͠·ͨ͠
    ײ૝Λblog౳Ͱ͍͚ͨͩΔͱ
    େมتͼ·͢!

    View Slide

  4. ಛूΛࣥච͠·ͨ͠
    ײ૝Λ(ུ

    View Slide

  5. GoͰଳҬ੍ݶ͕͍ͨ͠
    • Ϧιʔε࢖͍͗͢๷ࢭ
    • StretcherͰμ΢ϯϩʔυଳҬ੍ݶ
    • ϕϯνϚʔΫ
    • Fluentd in_tail ϕϯνϚʔΫͷͨΊͷ go-dummer
    • (ࣾ಺)ISUCON ϕϯνϚʔΧʔͰଳҬΤϛϡϨʔγϣϯ
    • io.Reader, io.Writer ΠϯλʔϑΣʔεͩͱศར

    View Slide

  6. ͭ͘Γ·ͨ͠ (4݄)
    github.com/fujiwara/shapeio
    import "github.com/fujiwara/shapeio"
    r := shapeio.NewReader(reader) // io.Reader
    r.SetRateLimit(1024 * 10) // 10KB/sec
    // ͋ͱ͸ී௨ʹ io.Reader ͱͯ͠࢖͏
    Writer΋͋Γ·͢

    View Slide

  7. ͦͷޙ

    View Slide

  8. golang.org/x/time/rate !?
    ʮgolang.org/x/time/rateͰ଎౓੍ݶΛߦ͏ʯ
    http://qiita.com/lufia/items/29bf1aeb0a0fe69d16f0
    ศརͦ͏

    View Slide

  9. x/time/rate Ͱॻ͖௚͠
    before (ଳҬ੍ޚͰ଴ͭ෦෼)
    func (l *limiter) wait(n int, err error) (int, error) {
    if n == 0 || err != nil {
    return n, err
    }
    elapsed := time.Since(l.startedAt)
    l.bytes += int64(n)
    rate := float64(l.bytes) / elapsed.Seconds()
    if rate < l.bytesPerSec {
    return n, nil
    }
    d := time.Duration(float64(l.bytes)/l.bytesPerSec*float64(time.Second) - float64(elapsed))
    time.Sleep(d)
    // reset shaping window
    l.startedAt = time.Now()
    l.bytes = 0
    return n, nil
    }

    View Slide

  10. x/time/rate Ͱॻ͖௚͠
    after
    func (s *Writer) Write(p []byte) (int, error) {
    if s.limiter == nil {
    return s.w.Write(p)
    }
    n, err := s.w.Write(p)
    if err != nil {
    return n, err
    }
    // ଴ͭͱ͜Ζ͸͚ͩ͜͜
    if err := s.limiter.WaitN(s.ctx, n); err != nil {
    return n, err
    }
    return n, err
    }

    View Slide

  11. x/time/rate
    1ඵؒʹཷ·Δtokenͷ਺ͱɺ࠷େͰཷΊΔ਺Λࢦఆ
    limiter := rate.NewLimiter(rate.Limit(m), burst)
    nݸͷtokenΛফඅ͢Δ(ཷ·ͬͯͳ͚Ε͹ཷ·Δ·Ͱ଴ͭ)
    err := limiter.WaitN(ctx, n)
    nݸͷtoken͕ཷ·ͬͯΔ͔֬ೝ(ཷ·ͬͯΕ͹ফඅͯ͠true)
    ok := limiter.AllowN(time.Now(), n)

    View Slide

  12. x/time/rate ಋೖͷ෭࢈෺
    context ͕࢖͑ΔΑ͏ʹ
    ctx, cancel := context.WithTimeout(context.Background(), 3*time.Seconds)
    r := shapeio.NewReaderWithContext(reader, ctx)
    shapeio.SetRateLimit(1024 * 1024)
    io.Copy(w, r)
    ...
    ଳҬ੍ݶͭͭ͠λΠϜΞ΢τ΍Ωϟϯηϧ΋

    View Slide

  13. ·ͱΊ
    x/time/rate ศརʂ
    shapeio ΋ΑΖ͘͠

    View Slide