Interfaces will Save the Future: Rate Limiter in Golang

Sphinx, a high-performance rate limiter built in Go as a case study:

* Collaborating on a code base by defining interfaces
* How interfaces allow trivially adding dynamic configuration reload in a few lines of code
* Providing multiple, swappable backends with different functionality
* Where to use Go’s benchmarking library, and where it’s not enough

Transcript

1. Interfaces Will Save the Future Case Study: Sphinx Rate Limiter 21 Jan 2015 Mohit Gupta & Alex Zylman Clever
2. What is Sphinx? HTTP rate limiter Based on leakybucket algorithm (supporting both Redis and in-memory stores) Supports matching on request headers and IP address Transparent limiting through X-Rate-Limit headers Low request latencies (~15 ms)
3. Limiting requests Sphinx allows you to set any number of limits, where a limit defines: How many requests to allow in an interval Which requests this limit applies to (e.g. only a specific path, IP, or header) What part of the request you're ratelimiting by (e.g. IP, user)
4. ♥ interfaces ♥
5. ♥ interfaces in Go ♥ Interfaces are sets of methods Any struct with the set of methods implements the interface type LimitKey interface { Key(common.Request) (string, error) Type() string } type ipLimitKey struct{} func (ilk ipLimitKey) Type() string { return "ip" } func (ilk ipLimitKey) Key(request common.Request) (string, error) { if _, ok := request["remoteaddr"]; !ok { return "", EmptyKeyError{ilk, "No remoteaddr key in request"} } return "ip:" + request["remoteaddr"].(string), nil }
6. ♥ interfaces in Go♥ Improved collaboration Easier testing Swappable behaviors
7. Collaboration through interfaces
8. Collaboration through interfaces "Who works on what" and "where do you start" is a hard problem Desire: Working on different parts simultaneously Flexibility in deciding what to work on Not be blocked by your teammates
9. Collaboration through interfaces: Example With defined interfaces, you can use mock implementations // A RateLimiter adds a request to a rate limiting bucket and returns the result. type RateLimiter interface { Add(request common.Request) ([]Status, error) } // A Status contains the result of adding a request to our limiting buckets. type Status struct { Capacity uint Reset time.Time Remaining uint Name string }
10. Collaboration through interfaces: Example import ( "github.com/stretchr/testify/mock" ) type MockRateLimiter struct { *mock.Mock } func (r *MockRateLimiter) Add(request common.Request) ([]Status, error) { args := r.Mock.Called(request) return args.Get(0).([]Status), args.Error(1) } // Verify at compile time that MockRateLimiter implements the RateLimiter interface var _ RateLimiter = &MockRateLimiter{}
11. Collaboration through interfaces: Example func DoSomethingWithRateLimiter(limiter RateLimiter) error { _, err := limiter.Add(common.Request{"path": "/debug"}) return err } var tests = []struct{ error }{ {error: nil}, {error: errors.New("garbage")}, } func TestDoingSomethingWithRateLimiter(t *testing.T) { for _, test := range tests { limiter := &MockRateLimiter{Mock: new(mock.Mock)} limiter.On("Add", common.Request{"path": "/debug"}).Return([]Status{}, test.error) assert.Equal(t, DoSomethingWithRateLimiter(limiter), test.error) limiter.AssertExpectations(t) } }
12. Easier testing
13. Easier testing Have different implementations of the same interface Desire: - Different implementations stay in sync - Bugs fixed in one implementation are fixed in all
14. Easier testing: Example Write tests against the interface so each implementation can reuse them A test case for a bug will ensure all implementations are bug-free type Storage interface { Create(name string, capacity uint, rate time.Duration) (Bucket, error) }
15. Easier testing: Example func CreateTest(s Storage) func(*testing.T) { return func(t *testing.T) { now := time.Now() bucket, err := s.Create("testbucket", 100, time.Minute) if err != nil { t.Fatal(err) } if capacity := bucket.Capacity(); capacity != 100 { t.Fatalf("expected capacity of %d, got %d", 100, capacity) } e := float64(1 * time.Second) // margin of error if error := float64(bucket.Reset().Sub(now.Add(time.Minute))); math.Abs(error) > e { t.Fatalf("expected reset time close to %s, got %s", now.Add(time.Minute), bucket.Reset()) } } }
16. Easier testing: Example Tests for the memory implementation: func TestCreate(t *testing.T) { CreateTest(New())(t) } Tests for the redis implementation: func TestCreate(t *testing.T) { s, err := New("tcp", os.Getenv("REDIS_URL")) assert.Nil(t, err) CreateTest(s)(t) }
17. Swappable behaviors
18. Swappable behaviors Desire: Change behavior with just config type Handler interface { ServeHTTP(ResponseWriter, *Request) } func NewHTTPLimiter(rateLimiter ratelimiter.RateLimiter, proxy http.Handler) http.Handler { return &httpRateLimiter{rateLimiter: rateLimiter, proxy: proxy} } func NewHTTPLogger(rateLimiter ratelimiter.RateLimiter, proxy http.Handler) http.Handler { return &httpRateLogger{rateLimiter: rateLimiter, proxy: proxy} }
19. Plugins / components Support matching and limiting on different parts of the request Desire: Easily add Matchers and Keys without diving into the whole system Each component responsible for it's own configuration
20. Plugins / components: Example // A Matcher determines if a common.Request matches its requirements. type Matcher interface { Match(common.Request) bool } // A MatcherFactory creates a Matcher based on a config. type MatcherFactory interface { Type() string Create(config interface{}) (Matcher, error) } type pathMatcherConfig struct { MatchAny []string `yaml:"match_any"` } type pathMatcher struct { Paths []*regexp.Regexp } type pathMatcherFactory struct{} func (pmf pathMatcherFactory) Type() string { return "paths" } func (pmf pathMatcherFactory) Create(config interface{}) (Matcher, error) { ...
21. Bonus
22. Bonus: Adding dynamic configuration reload type daemon struct { handler http.Handler proxy config.Proxy } +func (d *daemon) ServeHTTP(rw http.ResponseWriter, req *http.Request) { + d.handler.ServeHTTP(rw, req) +} + func (d *daemon) Start() { log.Printf("Listening on %s", d.proxy.Listen) - log.Fatal(http.ListenAndServe(d.proxy.Listen, d.handler)) + log.Fatal(http.ListenAndServe(d.proxy.Listen, d)) return } func (d *daemon) LoadConfig(newConfig config.Config) error { d.proxy = newConfig.Proxy target, _ := url.Parse(d.proxy.Host) // already tested for invalid Host proxy := httputil.NewSingleHostReverseProxy(target) ...
23. Thank you 21 Jan 2015 Mohit Gupta & Alex Zylman @mohitgupta (http://twitter.com/mohitgupta) @amzylman (http://twitter.com/amzylman) Clever https://clever.com (https://clever.com) https://github.com/Clever (https://github.com/Clever)