3 secrets to build web APIs in Go

3 secrets to build web APIs in Go

Based on his experience with Splice, Matt Aimonetti explores 3 critical elements that made a big difference when building a startup on top of Go.
Talk given at Gopher SummerFest and Google I/O

C69521d6e22fc0bbd69337ec8b1698df?s=128

Matt Aimonetti

June 26, 2014
Tweet

Transcript

  1. 3 secrets to build web APIs in Go Matt Aimonetti

    - splice.com - @mattetti
  2. Matt Aimonetti splice: Go, Ruby, JS, Obj-C, C#, C++, C

    golangbootcamp.com
  3. Splice

  4. Splice

  5. None
  6. None
  7. None
  8. Architecture clients APIs / core logic web rendering

  9. Architecture Web API: Synchronization File upload File download Notifications Obj-C

    C# C Go
  10. Architecture file parsing/analysis (sessions, media) deduplication/storage intermediate representation

  11. Architecture internal/private API (application logic) “public” API (states retrieval/change)

  12. Architecture HTTP Message Bus Long running async tasks

  13. Go vs Ruby 10-20% slower to write 20-100x faster to

    run 60-80% less maintenance required batteries included
  14. • Deployment • Routing • Monitoring

  15. None
  16. Deployment Zero downtime deployments bitbucket.org/splice/go.grace careful with goroutines

  17. Deployment Version Control CI Storage

  18. Deployment

  19. Deployment 2014-06-23-183000/api 2014-06-23-204500/api current/api symlink

  20. Deployment 2014-06-23-183000/api 2014-06-23-204500/api current/api $ kill -USR2 {api PID} symlink

  21. None
  22. Routing var V42Routes = Endpoints{ // create Ableton Live session

    { Verb: "POST", Path: "/v42/live/sessions", Handler: handlers.AlsUpload, RequiredParams: handlers.AlsUploadReqParams, Auth: splice.ClientAuth, }, // create Logic session { Verb: "POST", Path: "/v42/logic/sessions", Handler: handlers.LogicUpload, RequiredParams: handlers.LogicUploadReqParams, Auth: splice.ClientAuth, }, }
  23. Routing package router type Endpoint struct { Verb string Path

    string RequiredParams []string Auth func(*splice.ReqEnv, bool) bool Handler splice.Handler }
  24. Routing type Endpoints []Endpoint // Activate all the endpoint API

    routes. func (es Endpoints) Activate(p *pat.PatternServeMux) { for _, e := range es { // [..] set authentication calls // [..] check param requirements handler := splice.ReqWrap(e.Handler, e.Auth) p.Add(e.Verb, e.Path, handler) // [..] collect all CORS verbs accepted for this endpoint so we can create OPTIONS routes. } }
  25. Routing func ReqWrap(handler *Handler, checkAuthFn func(*ReqEnv, bool) bool) http.Handler {

    ! return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { start := time.Now() defer reportRequest(req, start, handler.Metrics) reqUuid := uuid.GenUUID() req := &ReqEnv{ ReqUuid: reqUuid, Request: r, Response: &statusResponseWriter{w, 0}, } // [..] set headers (content type, req ID, build) if !checkAuthFn(req, checkAuth) { return } ! ! !
  26. Routing ! ! ! // We must return a 400

    and stop here if there was a problem // parsing the request. err := req.ParseParams() if err != nil { // […]return 400 return } // convert a panic into a 500 defer req.handlePanic() handler.Handle(req) }) }
  27. None
  28. Monitoring Monitor all the things! runtime.ReadMemStats http://golang.org/pkg/runtime/#MemStats

  29. Lessons learned low memory usage

  30. Lessons learned detect issues

  31. Lessons learned detect issues

  32. Low memory usage use streaming io.Reader/io.Writer io.Copy Encoder/Decoder instead of

    ReadAll and Unmarshal
  33. Monitoring Monitor all the things! Per endpoint: • # of

    requests • Status codes • Response times
  34. None
  35. @mattetti http://matt.aimonetti.net