Handling slow requests in your Go web server

(delivered at dotGo Paris, 2017 https://youtu.be/0qKqVSvB1G0)

A common pattern when serving web requests that produce long-running processes is to send a work message to a queue, such as beanstalkd, and then handle the work in async fashion.
In Go we can
1) implement the queue within the same Go process.
or 2) forgo the queue idea completely.

Transcript

1. Handling slow requests in your Go web server Jaime Silvela - dotGo Paris
2. Slow requests are trouble package foo type SlowJob struct { // fields } func doSlowJob(j Job) { // takes minutes }
3. Slow requests are trouble type handler struct { // stuff } func (h handler) ServeHTTP(…) { job := parseJob(r) doSlowJob(job) // render, encode… } package foo type SlowJob struct { // fields } func doSlowJob(j Job) { // takes minutes }
4. Slow requests are trouble • Frustrated users • Reloads • Proxy timeouts • …
5. Job queue Slow Request Web Server Background Worker
6. Job queue Web Server Background Worker Slow Request
7. Job queue Web Server Background Worker Response
8. Job queue Web Server Background Worker
9. Problems with the solution • Infrastructure deployment and maintenance • Your slow jobs now execute
 in a separate process • Serialization and deserialization
10. Problems with the solution package foo type SlowJob struct { file io.Reader // > 100MB other Stuff } func storeDB(j SlowJob) { … } • Infrastructure deployment and maintenance • Your slow jobs now execute
 in a separate process • Serialization and deserialization
11. Solution 1: queue with channels • Add a channel to our handler • Identify jobs with a ticket number, track progress in DB • Send jobs to channel, return
 quickly • A goroutine (or more) checks 
 channel for jobs
12. Solution 1: queue with channels type SlowReq struct { job SlowJob ticket int } type slowHandlerQ struct { queue chan SlowReq db *memcache.Client } • Add a channel to our handler • Identify jobs with a ticket number, track progress in DB • Send jobs to channel, return
 quickly • A goroutine (or more) checks 
 channel for jobs
13. Solution 1: queue with channels func newSlowHandlerQ(db *memcache.Client) slowHandlerQ { queue := make(chan SlowReq) go func(q <-chan SlowReq) { for { req := <-q result := doSlowJob(req.job) publish(req.ticket, result, db) } }(queue) return slowHandlerQ{…} }
14. Solution 1: queue with channels func newSlowHandlerQ(db *memcache.Client) slowHandlerQ { queue := make(chan SlowReq) go func(q <-chan SlowReq) { for { req := <-q result := doSlowJob(req.job) publish(req.ticket, result, db) } }(queue) return slowHandlerQ{…} }
15. Solution 1: queue with channels func (h slowHandlerQ) ServeHTTP(…) { job := // parse request params into SlowJob ticket, _ := getTicket(h.db) h.queue <- SlowReq{ job: job, ticket: int(ticket), } w.Write([]byte(fmt.Sprintf("ticket: %d", ticket))) }
16. Solution 1: queue with channels func (h slowHandlerQ) ServeHTTP(…) { job := // parse request params into SlowJob ticket, _ := getTicket(h.db) h.queue <- SlowReq{ job: job, ticket: int(ticket), } w.Write([]byte(fmt.Sprintf("ticket: %d", ticket))) }
17. Problems • The channel send is synchronous, and may block/wait • We can add a time limit to the send, drop if limit exceeded • We can make the channel buffered • We can add more worker goroutines • … we have a new dimensioning problem in our server
18. Solution 2: delegate • Handler delegates the work
 to a goroutine, returns quickly • Identify jobs with a ticket
 number, track progress in DB
19. Solution 2: delegate type slowHandlerD struct { db *memcache.Client } • Handler delegates the work
 to a goroutine, returns quickly • Identify jobs with a ticket
 number, track progress in DB
20. Solution 2: delegate func (h slowHandlerD) ServeHTTP(…) { job := // parse request params into SlowJob ticket, _ := getTicket(h.db) go func(j SlowJob, ticket int) { result := doSlowJob(j) publish(ticket, result, h.db) }(job, int(ticket)) w.Write([]byte(fmt.Sprintf("ticket: %d", ticket))) }
21. Solution 2: delegate func (h slowHandlerD) ServeHTTP(…) { job := // parse request params into SlowJob ticket, _ := getTicket(h.db) go func(j SlowJob, ticket int) { result := doSlowJob(j) publish(ticket, result, h.db) }(job, int(ticket)) w.Write([]byte(fmt.Sprintf("ticket: %d", ticket))) }
22. Graceful shutdown type slowHandlerD struct { db *memcache.Client wg *sync.Waitgroup } func (h slowHandlerD) ServeHTTP(…) { … h.wg.Add(1) go func(j SlowJob, ticket int) { defer h.wg.Done()
23. Other improvements • Cancellation • Monitoring in-transit jobs • Saving state quickly to recover after restart 
 (c.f. 12 factor app)
24. You can handle slow requests entirely in your Go server
25. • https://pixabay.com/en/box-cardboard-cardboard-box-package-1673579/ • https://github.com/golang-samples/gopher-vector