Slide 1
Slide 1 text
Handling slow requests
in your Go web server
Jaime Silvela - dotGo Paris
Slide 2
Slide 2 text
Slow requests are trouble
package foo
type SlowJob struct {
// fields
}
func doSlowJob(j Job) {
// takes minutes
}
Slide 3
Slide 3 text
Slow requests are trouble
type handler struct {
// stuff
}
func (h handler) ServeHTTP(…) {
job := parseJob(r)
doSlowJob(job)
// render, encode…
}
package foo
type SlowJob struct {
// fields
}
func doSlowJob(j Job) {
// takes minutes
}
Slide 4
Slide 4 text
Slow requests are trouble
• Frustrated users
• Reloads
• Proxy timeouts
• …
Slide 5
Slide 5 text
Job queue
Slow Request
Web Server Background Worker
Slide 6
Slide 6 text
Job queue
Web Server Background Worker
Slow Request
Slide 7
Slide 7 text
Job queue
Web Server Background Worker
Response
Slide 8
Slide 8 text
Job queue
Web Server Background Worker
Slide 9
Slide 9 text
Problems with the solution
• Infrastructure deployment
and maintenance
• Your slow jobs now execute
in a separate process
• Serialization and
deserialization
Slide 10
Slide 10 text
Problems with the solution
package foo
type SlowJob struct {
file io.Reader // > 100MB
other Stuff
}
func storeDB(j SlowJob) {
…
}
• Infrastructure deployment
and maintenance
• Your slow jobs now execute
in a separate process
• Serialization and
deserialization
Slide 11
Slide 11 text
Solution 1: queue with
channels
• Add a channel to our handler
• Identify jobs with a ticket
number, track progress in DB
• Send jobs to channel, return
quickly
• A goroutine (or more) checks
channel for jobs
Slide 12
Slide 12 text
Solution 1: queue with
channels
type SlowReq struct {
job SlowJob
ticket int
}
type slowHandlerQ struct {
queue chan SlowReq
db *memcache.Client
}
• Add a channel to our handler
• Identify jobs with a ticket
number, track progress in DB
• Send jobs to channel, return
quickly
• A goroutine (or more) checks
channel for jobs
Slide 13
Slide 13 text
Solution 1: queue with
channels
func newSlowHandlerQ(db *memcache.Client) slowHandlerQ {
queue := make(chan SlowReq)
go func(q <-chan SlowReq) {
for {
req := <-q
result := doSlowJob(req.job)
publish(req.ticket, result, db)
}
}(queue)
return slowHandlerQ{…}
}
Slide 14
Slide 14 text
Solution 1: queue with
channels
func newSlowHandlerQ(db *memcache.Client) slowHandlerQ {
queue := make(chan SlowReq)
go func(q <-chan SlowReq) {
for {
req := <-q
result := doSlowJob(req.job)
publish(req.ticket, result, db)
}
}(queue)
return slowHandlerQ{…}
}
Slide 15
Slide 15 text
Solution 1: queue with
channels
func (h slowHandlerQ) ServeHTTP(…) {
job := // parse request params into SlowJob
ticket, _ := getTicket(h.db)
h.queue <- SlowReq{
job: job,
ticket: int(ticket),
}
w.Write([]byte(fmt.Sprintf("ticket: %d", ticket)))
}
Slide 16
Slide 16 text
Solution 1: queue with
channels
func (h slowHandlerQ) ServeHTTP(…) {
job := // parse request params into SlowJob
ticket, _ := getTicket(h.db)
h.queue <- SlowReq{
job: job,
ticket: int(ticket),
}
w.Write([]byte(fmt.Sprintf("ticket: %d", ticket)))
}
Slide 17
Slide 17 text
Problems
• The channel send is synchronous, and may block/wait
• We can add a time limit to the send, drop if limit
exceeded
• We can make the channel buffered
• We can add more worker goroutines
• … we have a new dimensioning problem in our server
Slide 18
Slide 18 text
Solution 2: delegate
• Handler delegates the work
to a goroutine, returns quickly
• Identify jobs with a ticket
number, track progress in DB
Slide 19
Slide 19 text
Solution 2: delegate
type slowHandlerD struct {
db *memcache.Client
}
• Handler delegates the work
to a goroutine, returns quickly
• Identify jobs with a ticket
number, track progress in DB
Slide 20
Slide 20 text
Solution 2: delegate
func (h slowHandlerD) ServeHTTP(…) {
job := // parse request params into SlowJob
ticket, _ := getTicket(h.db)
go func(j SlowJob, ticket int) {
result := doSlowJob(j)
publish(ticket, result, h.db)
}(job, int(ticket))
w.Write([]byte(fmt.Sprintf("ticket: %d", ticket)))
}
Slide 21
Slide 21 text
Solution 2: delegate
func (h slowHandlerD) ServeHTTP(…) {
job := // parse request params into SlowJob
ticket, _ := getTicket(h.db)
go func(j SlowJob, ticket int) {
result := doSlowJob(j)
publish(ticket, result, h.db)
}(job, int(ticket))
w.Write([]byte(fmt.Sprintf("ticket: %d", ticket)))
}
Slide 22
Slide 22 text
Graceful shutdown
type slowHandlerD struct {
db *memcache.Client
wg *sync.Waitgroup
}
func (h slowHandlerD) ServeHTTP(…) {
…
h.wg.Add(1)
go func(j SlowJob, ticket int) {
defer h.wg.Done()
Slide 23
Slide 23 text
Other improvements
• Cancellation
• Monitoring in-transit jobs
• Saving state quickly to recover after restart
(c.f. 12 factor app)
Slide 24
Slide 24 text
You can handle slow requests
entirely
in your Go server
Slide 25
Slide 25 text
• https://pixabay.com/en/box-cardboard-cardboard-box-package-1673579/
• https://github.com/golang-samples/gopher-vector