manage millions of concurrent connections • How to write a webserver in Go? • How to handle persistent connections? • What limitations arise in scale? • How to handle persistent connections efficiently? ◦ OS limitations ◦ Hardware limitations
connections on the Listener l, creating a // new service goroutine for each. The service goroutines read requests and // then call srv.Handler to reply to them. func (srv *Server) Serve(l net.Listener) error { // ... for { rw, e := l.Accept() // ... c := srv.newConn(rw) c.setState(c.rwc, StateNew) // before Serve can return go c.serve(ctx) } }
connections on the Listener l, creating a // new service goroutine for each. The service goroutines read requests and // then call srv.Handler to reply to them. func (srv *Server) Serve(l net.Listener) error { // ... for { rw, e := l.Accept() // ... c := srv.newConn(rw) c.setState(c.rwc, StateNew) // before Serve can return go c.serve(ctx) } } func hello(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "Hello Gophercon!" ) }
a full-duplex persistent connection between a client and server that both parties can start sending data at any time, with low overhead and latency GET ws://websocket.example.com/ HTTP/1.1 Connection: Upgrade Host: websocket.example.com Upgrade: websocket Client Server HTTP/1.1 101 WebSocket Protocol Handshake Connection: Upgrade Upgrade: WebSocket
a file descriptor • The OS needs memory to manage each open file • Memory is a limited resource • Maximum number of open files can be changed via ulimits
processes • The kernel enforces the soft limit for the corresponding resource • The hard limit acts as a ceiling for the soft limit • Unprivileged process can only raise up to the hard limit • Privileged process can make any arbitrary change • RLIMIT_NOFILE is the resource enforcing max number of open files
data in the format expected by the pprof visualization tool. • Analyze heap memory: go tool pprof http://localhost:6060/debug/pprof/heap • Analyze goroutines: go tool pprof http://localhost:6060/debug/pprof/goroutine import _ "net/http/pprof" go func() { if err := http.ListenAndServe ("localhost:6060" , nil); err != nil { log.Fatalf("Pprof failed: %v" , err) } }()
if we must: • Ulimit: Increase the cap of NOFILE resource • Epoll (Async I/O): Reduce the high load of goroutines • Gobwas - More performant ws library to reduce buffer allocations • Conntrack table - Increase the cap of total concurrent connections in the OS