Understanding net/http: building a new router for GOV.UK 10 July 2013 Nick Stenning Government Digital Service

GOV.UK Hello, I'm from the government.

GOV.UK

GOV.UK

GOV.UK (behind the scenes)

Internal APIs, small pieces Behind the scenes we have: 34 Rack apps (mostly Rails, some Sinatra) Flask Django Scala/Play Loads of nginx Varnish HTTP everywhere .

Routing Existing solution leaves something to be desired... ahem... sub vcl_recv { # Routing if (req.url ~ "^/autocomplete(\?.*)?$|^/preload-autocomplete(\?.*)?$|^/sitemap[^/]*.xml(\?.*)?$") { <%= set_backend('search') %> } else if (req.url ~ "^/when-do-the-clocks-change([/?.].*)?$|^/bank-holidays([/?.].*)?$|^/gwyliau-banc( <%= set_backend('calendars') %> } else if (req.url ~ "^/(<%= @smartanswers.join("|") %>)([/?.].*)?$") { <%= set_backend('smartanswers') %> } else if (req.url ~ "^/stylesheets|^/javascripts|^/images|^/templates|^/favicon\.ico(\?.*)?$|^/humans\ <%= set_backend('static') %> } else if (req.url ~ "^/service-manual([/?.].*)?$|^/designprinciples([/?.].*)?$") { <%= set_backend('designprinciples') %> ... } else if (req.url ~ "^/__canary__$") { <%= set_backend('canary_frontend') %> <%# This matches on any subpath of the slugs, the bare slugs should fall through to content in Frontend } else if (req.url ~ "^/pay-foreign-marriage-certificates/(.+)$|^/deposit-foreign-marriage/(.+)$|^/pay- <%= set_backend('transaction_wrappers') %> } else { <%= set_backend('frontend') %> } }

Hello, go package main import "fmt" func main () { fmt.Println("Hello, world!") } Run

Hello, HTTP package main import ( "fmt" "log" "net/http" ) const listenAddr = ":4000" func sayHello(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "Hello, world!") } func main() { log.Println("Listening on", listenAddr) http.HandleFunc("/", sayHello) http.ListenAndServe(listenAddr, nil) } Run

HandlerFunc and DefaultServeMux func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) { DefaultServeMux.HandleFunc(pattern, handler) } func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) { mux.Handle(pattern, HandlerFunc(handler)) } type HandlerFunc func(ResponseWriter, *Request) // ServeHTTP calls f(w, r). func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) { f(w, r) } func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) { ... if handler == nil { handler = DefaultServeMux } ... }

What's a ServeMux? func sayHello(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello!") } func sayGoodbye(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Bye!") } func main() { log.Println("Listening on", listenAddr) http.HandleFunc("/hello", sayHello) http.HandleFunc("/bye", sayGoodbye) log.Fatal(http.ListenAndServe(listenAddr, nil)) } Run

What's a ServeMux? (explicit) func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe() } func sayHello(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello!") } func sayGoodbye(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Bye!") } func main() { log.Println("Listening on", listenAddr) mux := http.NewServeMux() mux.HandleFunc("/hello", sayHello) mux.HandleFunc("/bye", sayGoodbye) log.Fatal(http.ListenAndServe(listenAddr, mux)) } Run

What's a ServeMux? http.ListenAndServe takes a Handler . func ListenAndServe(addr string, handler Handler) error { ... } ServeMux is a type that satisfies the Handler interface. func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) { ... } ServeMux.Handle() registers other handlers at paths. func (mux *ServeMux) Handle(pattern string, handler Handler) { ... } It's a request router.

Handler Everything is a Handler. But what's one of those? type Handler interface { ServeHTTP(ResponseWriter, *Request) } Well, anything. Anything that has an appropriate ServeHTTP method.

Turtles all the way down

Interfaces all the way down Interfaces, interfaces everywhere! func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe() } type Handler interface { ServeHTTP(ResponseWriter, *Request) } type ResponseWriter interface { Header() Header Write([]byte) (int, error) WriteHeader(int) } type Writer interface { Write([]byte) (int, error) }

The start of a better router The mapping between URLs and backend services can be modelled as a prefix tree (or trie)

The start of a better router package trie type trieChildren map[string]*Trie type Trie struct { Leaf bool Entry interface{} Children trieChildren } func (t *Trie) Get(path []string) (entry interface{}, ok bool) { ... } func (t *Trie) Set(path []string, value interface{}) { ... } github.com/nickstenning/trie (https://github.com/nickstenning/trie)

The routing table is data Treat routing information as data, which can be updated on the fly by applications as they deploy. They can register themselves: { "id": "whitehall", "url": "http://whitehall.internal" } And their routes: { "type": "exact", "path": "/airport-rights", "application_id": "frontend" } { "type": "prefix", "path": "/government", "application_id": "whitehall" }

Combining ServeMux and Trie... package triemux func (mux *Mux) Handle(path string, prefix bool, handler http.Handler) { mux.mu.Lock() defer mux.mu.Unlock() mux.trie.Set(splitpath(path), muxEntry{prefix, handler}) } func (mux *Mux) ServeHTTP(w http.ResponseWriter, r *http.Request) { handler, ok := mux.lookup(r.URL.Path) if !ok { http.NotFound(w, r) return } handler.ServeHTTP(w, r) } github.com/nickstenning/router/triemux (https://github.com/nickstenning/router/tree/master/triemux)

A simple TrieMux example mux := triemux.NewMux() googUrl, _ := url.Parse("http://google.com") aaplUrl, _ := url.Parse("http://apple.com") goog := httputil.NewSingleHostReverseProxy(googUrl) aapl := httputil.NewSingleHostReverseProxy(aaplUrl) // register a prefix route pointing to the Google backend (all requests to // "/google" will go to this backend) mux.Handle("/google", true, goog) // register an exact (non-prefix) route pointing to the Apple backend mux.Handle("/apple", false, aapl) log.Println("Listening on :8088") log.Fatalln(http.ListenAndServe(":8088", mux)) Run

Finally, combining TrieMux and a Database type Router struct { mux *triemux.Mux mongoUrl string mongoDbName string } func (rt *Router) ServeHTTP(w http.ResponseWriter, r *http.Request) { rt.mux.ServeHTTP(w, r) }

Atomic reloading func (rt *Router) ReloadRoutes() { // save a reference to the previous mux in case we have to restore it oldmux := rt.mux defer func() { if r := recover(); r != nil { log.Println("router: recovered from panic in ReloadRoutes:", r) rt.mux = oldmux log.Println("router: original routes have been restored") } }() ... sess, err := mgo.Dial(rt.mongoUrl) ... newmux := triemux.NewMux() apps := loadApplications(db.C("applications"), newmux) loadRoutes(db.C("routes"), newmux, apps) ... rt.mux = newmux log.Printf("router: reloaded routes") }

Is it any good? Small request bodies (lower bound): Transactions: 91610 hits Availability: 100.00 % Elapsed time: 59.55 secs Data transferred: 6.97 MB Response time: 0.01 secs Transaction rate: 1538.37 trans/sec Throughput: 0.12 MB/sec Concurrency: 19.44 Large request bodies (lower bound): Transactions: 13821 hits Availability: 100.00 % Elapsed time: 59.98 secs Data transferred: 3548.23 MB Response time: 0.09 secs Transaction rate: 230.43 trans/sec Throughput: 59.16 MB/sec Concurrency: 19.98

The unreasonable effectiveness of Go This was about three days' work, with no attempt at optimization. Contributing factors in Go's effectiveness: Dealing with error conditions up front The defer, recover() pattern Interfaces (is a bare pointer really the right answer?) Composition of simple components (Trie, TrieMux, Router) Rich standard library

If you only remember two things Interfaces, interfaces, interfaces Do the hard work to make it simple www.gov.uk/designprinciples#fourth (https://www.gov.uk/designprinciples#fourth)

Thank you Nick Stenning Government Digital Service https://whiteink.com/ (https://whiteink.com/) @nickstenning (http://twitter.com/nickstenning)