Asynchronous networking @ GopherCon 2018

An introduction to the net package, livecoding a TLS-aware TCP proxy.

Read the code! → https://github.com/FiloSottile/mostly-harmless/tree/master/talks/asyncnet

Transcript

1. ASYNCHRONOUS NETWORKING
   GOPHERCON, DENVER — 28 AUGUST 2018
   Filippo Valsorda
   Google
   @FiloSottile
   

   View Slide

2. Threading servers
   Thread
   Request 2
   Request 1
   Thread
   Thread
   Request 3
   

   View Slide

3. The event loop
   Request 1
   Thread
   Request 1
   Request 2
   Request 3
   Request 3
   Request 2
   

   View Slide

4. The event loop
   Request 1
   Thread
   Request 1
   Request 2
   Request 3
   Request 3
   Pop from event queue
   Monitor for events
   Request 2
   

   View Slide

5. Goroutines
   go
   Request 2
   Request 1
   go
   Request 1
   go
   Request 3
   Block
   Block
   Request 3
   Request 2
   Block
   

   View Slide

6. Accepting
   connections
   SECTION ONE
   

   View Slide

7. Let's write code!
   

   View Slide

8. View Slide

9. ACCEPTING CONNECTIONS — RECAP
   01. ACCEPTING CONNECTIONS
   l, err := net.Listen("tcp", "localhost:4242")
   if err != nil {
   log.Fatal(err)
   }
   Open a net.Listener.
   Never block in the Accept loop.
   for {
   conn, err := l.Accept()
   if err != nil {
   log.Fatal(err)
   }
   go serviceConn(conn)
   }
   Call Accept in a loop, and
   start new goroutines for
   each Conn.
   

   View Slide

10. A simple
    proxy
    SECTION TWO
    

    View Slide

11. Proxying a connection
    02. A SIMPLE PROXY
    Proxy
    gophercon.com:443
    localhost:4242
    Browser
    
    

    View Slide

12. "splice(2) moves data between two file
    descriptors without copying between kernel
    address space and user address space."
    

    View Slide

13. A SIMPLE PROXY — RECAP
    02. A SIMPLE PROXY
    upstream, err := net.Dial("tcp", addr)
    if err != nil {
    log.Println(err)
    return
    }
    defer upstream.Close()
    Connect to upstream.
    go io.Copy(upstream, conn)
    _, err = io.Copy(conn, upstream)
    Copy both directions.
    Close cancels both io.Copy.
    

    View Slide

14. Parsing TLS
    SECTION THREE
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. PARSING A TLS CLIENT HELLO — RECAP
    03. PARSING TLS
    conn.SetDeadline(time.Now().Add(30 * time.Second))
    var buf bytes.Buffer
    if _, err := io.CopyN(&buf, conn, 1+2+2); err != nil {
    return
    }
    length := binary.BigEndian.Uint16(buf.Bytes()[3:5])
    if _, err := io.CopyN(&buf, conn, int64(length));
    err != nil {
    return
    }
    Set a timeout and

    read the message.
    ch, ok := ParseClientHello(buf.Bytes())
    if ok {
    log.Printf("Received connection for %q!", ch.SNI)
    }
    Parse TLS.
    

    View Slide

20. Pre-read connection
    03. PARSING TLS
    Client Hello
    Already Read()
    up to here
    rest of the connection
    Would only proxy this part
    In bytes.Buffer
    

    View Slide

21. View Slide

22. PROXYING A PRE-READ CONNECTION — RECAP
    03. PARSING TLS
    type prefixConn struct {
    io.Reader
    net.Conn
    }
    func (c prefixConn) Read(b []byte) (int, error) {
    return c.Reader.Read(b)
    }
    Make a net.Conn
    wrapper.
    proxyConn(prefixConn{
    Reader: io.MultiReader(&buf, conn),
    Conn: conn,
    }, "gophercon.com:https")
    Use MultiReader.
    

    View Slide

23. Serving TLS
    SECTION FOUR
    

    View Slide

24. ! !
    tls.Conn is a
    net.Conn wrapper
    04. SERVING TLS
    

    View Slide

25. View Slide

26. SERVING A TLS CONNECTION — RECAP
    04. SERVING TLS
    c := tls.Server(prefixConn{
    Reader: io.MultiReader(&buf, conn),
    Conn: conn,
    }, config)
    Wrap the net.Conn with
    tls.Server.
    proxyConn(c, "gophercon.com:http")
    Service the wrapped
    plaintext connection.
    tls.Conn is a net.Conn wrapper. It takes care of handshake and de/encryption.
    

    View Slide

27. Thank you!
    https://git.io/fAYGy
    Filippo Valsorda
    Google
    @FiloSottile
    

    View Slide