Linux port sharding

A86a88130238a0d7f01b5612b9c72499?s=47 Joe Walnes
November 05, 2015

Linux port sharding

The mysterious SO_REUSEPORT option introduced in Linux kernel 3.9.

Example code here: https://github.com/joewalnes/port-sharding

A86a88130238a0d7f01b5612b9c72499?s=128

Joe Walnes

November 05, 2015
Tweet

Transcript

  1. Linux Port Sharding @joewalnes

  2. Wat?

  3. Linux Kernel 3.9 introduced new socket option SO_REUSEPORT

  4. Allows multiple processes to listen on same port Linux Kernel

    Process Clients Process Process Kernel distributes incoming connections
  5. Why? Spread socket event loop across multiple CPU cores requests

    / sec NGINX, 36 core AWS instance Source: https:/ /www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/
  6. But it has another trick… Rolling upgrades

  7. Rolling upgrades Linux Kernel MyServer version 1 Clients

  8. Rolling upgrades Linux Kernel MyServer version 1 Clients Let’s upgrade

    a server application
  9. Rolling upgrades Linux Kernel MyServer version 1 Clients MyServer version

    2 Start new version: accepts request alongside old
  10. Rolling upgrades Linux Kernel MyServer version 1 Clients MyServer version

    2 Signal old version to stop accepting new requests
  11. Rolling upgrades Linux Kernel Clients MyServer version 2 Old version

    terminates when done with requests
  12. from socket import * port = 1234 server = socket(AF_INET,

    SOCK_STREAM) server.bind(('', port)) server.listen(0) while True: client, addr = server.accept() data = client.recv(1024) # echo data back client.send(data) client.close() server.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1) require "socket" port = 1234 server = Socket.new Socket::AF_INET, Socket::SOCK_STREAM server.bind Addrinfo.tcp "", port server.listen 0 loop do client, addr = server.accept data = client.recv 1024 # echo data back client.send data, 0 client.close end Ruby edition Python edition Using it server.setsockopt Socket::SOL_SOCKET, Socket::SO_REUSEPORT, true
  13. @joewalnes Caveats: Don’t confuse this with the BSD SO_REUSEPORT option

    of exactly the same name (it won’t work on OSX) Also it’s not SO_REUSEADDR Sharing is not very smart: kernel still passes to process even when slow at accepting connections More on SO_REUSEPORT → https:/ /lwn.net/Articles/542629/ Try it → https:/ /github.com/joewalnes/port-sharding