From Zero to 400 webhooks/second in 24 hours

A brief intro about the main three things I believe make Go so good. for the cloud: simplicity, a great standard library, and performance.

Transcript

1. From Zero to 400 Req/s in 24 hours
2. Go: the language for the cloud
7. and many companies
8. Why?
9. concurrency!
10. concurrency!
11. Go is great to build servers and CLIs
12. my top 3 reasons
13. 1. Simplicity 2. Great stdlib 3. Performance
14. Why am I here?
19. Explore & Learn
20. Work with new people
21. FOCUS What do you want to learn?
22. Webhook Dispatcher Kubernetes, Prometheus, Go, Google Cloud
24. Simplicity
25. easy to develop
26. it is a small language
27. 272 pages
28. 1.368 pages just C++?
29. 400 pages
31. teach the whole language in 2 days
32. easy to deploy
33. static binaries without dependencies
34. FROM golang:1.8 as builder WORKDIR /go/src/github.com/gocardless/whebhook-dispatcher ADD . . RUN CGO_ENABLED=0 GOOS=linux \ go install -a -installsuffix cgo \ github.com/gocardless/whebhook-dispatcher/cmd/... FROM alpine:latest RUN apk --no-cache add ca-certificates WORKDIR /root/ COPY --from=builder /go/bin/wh-* /go/bin/demo-webhook /bin/ CMD ["wh-receiver"]
35. easy to operate
38. Great stdlib
39. net/http
40. http.HandleFunc( "/webhooks/send", func(w http.ResponseWriter, req *http.Request) { w.WriteHeader(http.StatusCreated) }, ) http.ListenAndServe(":80", http.DefaultServeMux)
41. http.ListenAndServeTLS
42. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc( "/webhooks/send", func(w http.ResponseWriter, req *http.Request) { err := r.Receive( req.Header.Get("X-WH-Endpoint-Key"), forwardedHeaders(req.Header), req.Body, ) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) } w.WriteHeader(http.StatusCreated) }, ) http.ListenAndServe(":80", http.DefaultServeMux)
43. production-ready HTTP 1 & 2 server
44. io
45. package io type Reader interface { Read(p []byte) (n int, err error) } type Writer interface { Write(p []byte) (n int, err error) }
46. os.File net.Conn http.Request http.ResponseWriter gzip.Reader gzip.Writer hash.Hash ...
47. // calculate hash of the request body h := sha1.New() io.Copy(h, request.Body) h.Sum(nil)
48. // calculate hash and decode json var user User h := sha1.New() r := io.TeeReader(request.Body, h) json.NewDecoder(r).Decode(&user) h.Sum(nil)
49. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc( "/webhooks/send", func(w http.ResponseWriter, req *http.Request) { err := r.Receive( req.Header.Get("X-WH-Endpoint-Key"), forwardedHeaders(req.Header), req.Body, ⚠ ) // ... }, ) http.ListenAndServe(":80", http.DefaultServeMux)
50. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc( "/webhooks/send", func(w http.ResponseWriter, req *http.Request) { err := r.Receive( req.Header.Get("X-WH-Endpoint-Key"), forwardedHeaders(req.Header), io.LimitReader(req.Body, 10 * 1024), ! ✅ ) // ... }, ) http.ListenAndServe(":80", http.DefaultServeMux)
51. context
52. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc( "/webhooks/send", func(w http.ResponseWriter, req *http.Request) { err := r.Receive( req.Context(),❓" req.Header.Get("X-WH-Endpoint-Key"), forwardedHeaders(req.Header), io.LimitReader(req.Body, 10 * 1024), ) // ... }, ) http.ListenAndServe(":80", http.DefaultServeMux)
53. func (s *receiver) Receive( ctx context.Context, ! ✅ key string, headers http.Header, body io.Reader, ) error { e, err := s.endpoints.FindByKey(ctx, key) ✅ if err != nil { return err } wh, err := webhooks.New(e.ID) if err != nil { return err } wh.Reader = body wh.Headers = headers err = s.webhooks.Put(ctx, wh) ✅ if err != nil { return err } err = s.queue.Enqueue(ctx, wh) ✅ if err != nil { return err } return nil }
54. context cancelation
55. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc("/webhooks/send", func(w http.ResponseWriter, req *http.Request) { ctx, cancel := context.WithTimeout(req.Context(), 20 * time.Second) defer cancel() err := r.Receive( ctx, req.Header.Get("X-WH-Endpoint-Key"), forwardedHeaders(req.Header), io.LimitReader(req.Body, 10 * 1024), ) // ... } http.ListenAndServe(":80", http.DefaultServeMux)
56. distributed tracing
58. r := receiving.NewService(endpointsRepo, webhooksRepo, webhooksQueue) http.HandleFunc("/webhooks/send", func(w http.ResponseWriter, req *http.Request) { // ... } http.ListenAndServe( ":80", nethttp.Middleware( opentracing.GlobalTracer(), http.DefaultServeMux, ), ✅ )
63. func (s *dispatcher) Dispatch(ctx context.Context, e *endpoints.Endpoint, id string) error { wh, err := s.webhooks.Get(ctx, id) if err != nil { return err } defer wh.Close() ctx, cancel := context.WithTimeout(ctx, 10 * time.Second) defer cancel() req, err := http.NewRequest(http.MethodPost, e.URL, wh.Body) if err != nil { return err } req = req.WithContext(ctx) res, err := s.client.Do(req) if err != nil { return err } defer res.Body.Close() if res.StatusCode != http.StatusOK { return errors.Errorf("invalid status response: %s", res.Status) } return nil }
64. Performance
65. Memory and CPU efficient
66. Easy to optimise
67. 100 webhooks/second Dispatcher: ~30MB memory
68. ~100 webhooks/second Dispatcher: ~30MB memory Receiver: ~900MB !
69. time to profile!
70. go test -cpuprofile cpu.out go test -memprofile mem.out go tool pprof
72. import _ "net/http/pprof"
75. $ go-torch -u http://my-service:8080
77. // package cloud.google.com/go/storage type Writer struct { // ChunkSize controls the maximum number of bytes of the object that the // Writer will attempt to send to the server in a single request. Objects // smaller than the size will be sent in a single request, while larger // objects will be split over multiple requests. The size will be rounded up // to the nearest multiple of 256K. If zero, chunking will be disabled and // the object will be uploaded in a single request. // // ChunkSize will default to a reasonable value. Any custom configuration // must be done before the first Write call. ChunkSize int // ... }
78. Default chunk size: ~8MB
79. w.ChunkSize = 0
80. +400 webhooks/second Dispatcher: < 100MB memory Receiver: < 100MB !
81. Try Go!
82. 1. Simplicity 2. Great stdlib 3. Performance
83. amazing inclusive community
84. We are hiring! bit.ly/join-gocardless
85. Questions? @ernesto_jimenez ernesto@gocardless.com