It doesn't have to be REST: WebSockets in Go

The speed of Go should make us reconsider well established paradigms. When developing APIs it is practically impossible not to mention REST yet many applications would benefit from real-time interactivity. There are different WebSocket implementation in the Go ecosystem. This talk will give you headspace to consider WebSockets as an alternative communication protocol for your application by giving an overview of the current state of the art of WebSockets in Go.

Transcript

1. It doesn’t have
   to be REST
   WebSockets in Go
   

   View Slide

2. @konradreiche
   

   View Slide

3. View Slide

4. HTTP 0.9
   HTTP 1.0
   HTTP 1.1
   HTTP/2
   GraphQL
   WebSocket
   1991 1996 1999 2000 2005 2008 2013 2015 2016 2018
   REST
   React
   High React
   Adoption
   Towards Real-Time
   Rendering & Updating
   AJAX
   Expected
   Real-Time
   Singularity
   

   View Slide

5. REST
   GET
   DELETE
   POST
   PUT
   Resources
   ?
   

   View Slide

6. REST (Representational State Transfer)
   - Defined by Roy T. Fielding in his dissertation:
   “Architectural Styles and the Design of Network-based Software Architectures”
   - Fielding: there are two perspectives on the process of architectural design
   - Build the architecture from familiar components until needs are satisfied
   - Start with system needs; incrementally add constraints when needed
   - First one emphasizes creativity and unbound vision
   - Second one emphasizes restraint and understanding the system context
   

   View Slide

7. View Slide

8. Client-Server
   

   View Slide

9. Client-Server
   Stateless
   

   View Slide

10. Client-Server
    Stateless
    Cache
    

    View Slide

11. Client-Server
    Stateless
    Cache
    Uniform Interface
    

    View Slide

12. Client-Server
    Stateless
    Cache
    Uniform Interface
    Layered System
    

    View Slide

13. Client-Server
    Stateless
    Cache
    Uniform Interface
    Layered System
    Code on Demand
    

    View Slide

14. WebSockets
    

    View Slide

15. Chats Games
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. GET ws://localhost:4000/ HTTP/1.1
    Origin: http://localhost:4000
    Host: localhost:4000
    Connection: Upgrade
    Upgrade: websocket
    Sec-WebSocket-Key: zy6Dy9mSAIM7GJZNf9rI1A==
    Sec-WebSocket-Version: 13
    Open Handshake
    Request
    

    View Slide

20. HTTP/1.1 101 Switching Protocols
    Connection: Upgrade
    Upgrade: websocket
    Sec-WebSocket-Accept: EDJa7WCAQQzMCYNJM42Syuo9SqQ=
    Open Handshake
    Response
    

    View Slide

21. Server
    ws://…
    Client
    Messages
    Text/Binary
    

    View Slide

22. Server
    ws://…
    Client
    Messages
    Text/Binary
    - Message-oriented protocol
    - Messages are split into frames
    - Payload can be binary or text
    - Supports fragmentation
    

    View Slide

23. Server
    ws://…
    Client
    Messages
    Text/Binary
    - Message-oriented protocol
    - Messages are split into frames
    - Payload can be binary or text
    - Supports fragmentation
    

    View Slide

24. package main
    import (
    "net/http"
    )
    func main() {
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

25. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func main() {
    http.Handle("/websocket", websocket.Handler(nil))
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

26. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func Handle(ws *websocket.Conn) {
    // TODO
    }
    func main() {
    http.Handle("/websocket", websocket.Handler(Handle))
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

27. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func Handle(ws *websocket.Conn) {
    // TODO
    }
    func main() {
    http.Handle("/websocket", websocket.Handler(Handle))
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    io.Reader
    io.Writer
    

    View Slide

28. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func Handle(ws *websocket.Conn) {
    ws.Write([]byte("I am alive"))
    }
    func main() {
    http.Handle("/websocket", websocket.Handler(Handle))
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

29. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func main() {
    origin := "http://localhost/"
    _, err := websocket.Dial("ws://localhost:4000/websocket", "", origin)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

30. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func main() {
    origin := "http://localhost/"
    ws, err := websocket.Dial("ws://localhost:4000/websocket", "", origin)
    if err != nil {
    panic(err)
    }
    _, err := ws.Write([]byte("Hello? Anybody here?"))
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

31. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func main() {
    origin := "http://localhost/"
    ws, err := websocket.Dial("ws://localhost:4000/websocket", "", origin)
    if err != nil {
    panic(err)
    }
    _, err := ws.Write([]byte("Hello? Anybody here?"))
    if err != nil {
    panic(err)
    }
    msg := make([]byte, 512)
    _, err = ws.Read(msg)
    // …
    

    View Slide

32. https://godoc.org/?q=websocket
    

    View Slide

33. github.com/gorilla/websocket
    3958 stars, 2393 imports
    golang.org/x/net/websocket
    1412 imports
    

    View Slide

34. View Slide

35. github.com/gorilla/websocket golang.org/x/net
    RFC 6455 Features
    Passes Autobahn Test Suite Yes No
    Receive fragmented message Yes No
    Send close message Yes No
    Send pings and receive pongs Yes No
    Get the type of a received data message Yes Yes
    Other Features
    Read message using io.Reader Yes No?
    Write message using io.WriterCloser Yes No?
    

    View Slide

36. Message 1
    

    View Slide

37. Frame 1
    Message 1
    

    View Slide

38. Frame 1 Frame 2
    Message 1
    

    View Slide

39. Frame 1 Frame 2 Frame 3
    Message 1
    

    View Slide

40. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    

    View Slide

41. package main
    import (
    "net/http"
    "golang.org/x/net/websocket"
    )
    func main() {
    origin := "http://localhost/"
    ws, err := websocket.Dial("ws://localhost:4000/websocket", "", origin)
    if err != nil {
    panic(err)
    }
    _, err := ws.Write([]byte("Hello?"))
    if err != nil {
    panic(err)
    }
    msg := make([]byte, 512)
    _, err = ws.Read(msg)
    // …
    

    View Slide

42. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    

    View Slide

43. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    Read() / Write()
    golang.org/x/net/websocket
    

    View Slide

44. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    Read() / Write() Read() / Write()
    golang.org/x/net/websocket
    

    View Slide

45. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    Read() / Write() Read() / Write() Read() / Write()
    golang.org/x/net/websocket
    

    View Slide

46. Frame 1 Frame 2 Frame 3 Frame 1
    Message 1
    Frame 2
    Message 2
    github.com/gorilla/websocket
    Read() / Write() Read() / Write()
    Read() / Write() Read() / Write() Read() / Write()
    golang.org/x/net/websocket
    

    View Slide

47. package main
    import (
    "log"
    "net/http"
    )
    func handleUpgrade(w http.ResponseWriter, r *http.Request) {
    // TODO
    }
    func main() {
    http.HandleFunc("/websocket", handleUpgrade)
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

48. package main
    import (
    "log"
    "net/http"
    )
    var upgrader = websocket.Upgrader{}
    func handleUpgrade(w http.ResponseWriter, r *http.Request) {
    // TODO
    }
    func main() {
    http.HandleFunc("/websocket", handleUpgrade)
    err := http.ListenAndServe(":4000", nil)
    if err != nil {
    panic(err)
    }
    }
    

    View Slide

49. package main
    import (
    "log"
    "net/http"
    )
    var upgrader = websocket.Upgrader{}
    func handleUpgrade(w http.ResponseWriter, r *http.Request) {
    conn, err := upgrader.Upgrade(w, r, nil)
    if err != nil {
    panic(err)
    }
    defer conn.Close()
    // TODO
    }
    func main() {
    http.HandleFunc("/websocket", handleUpgrade)
    // …
    

    View Slide

50. package main
    import (
    "log"
    "net/http"
    )
    var upgrader = websocket.Upgrader{}
    func handleUpgrade(w http.ResponseWriter, r *http.Request) {
    conn, err := upgrader.Upgrade(w, r, nil)
    if err != nil {
    panic(err)
    }
    defer conn.Close()
    handleConnection(conn)
    }
    func main() {
    http.HandleFunc("/websocket", handleUpgrade)
    // …
    

    View Slide

51. func handleConnection(conn *websocket.Conn) {
    for {
    messageType, message, err := c.ReadMessage()
    if err != nil {
    break
    }
    log.Println(message)
    err = c.WriteMessage(messageType, message)
    if err != nil {
    break
    }
    }
    }
    

    View Slide

52. View Slide

53. View Slide

54. https://github.com/gobwas/ws
    - Efficient low-level WebSocket library
    - Allow users to reuse I/O buffers between connections
    - Export efficient low-level interface for working with the protocol
    Zero-Copy Upgrade
    - Skips the need to use HTTP for upgrading
    - ws.Upgrade() accepts io.ReadWriter (net.Conn)
    

    View Slide

55. Backend Clients
    API
    Database
    HTTP
    

    View Slide

56. Backend Clients
    API
    Database
    HTTP
    Polling
    

    View Slide

57. Push
    Backend Clients
    API
    Database
    WebSocket
    Bus
    

    View Slide

58. Push
    Backend Clients
    API
    Database
    WebSocket
    Bus
    Subscribe
    Publish
    State change
    

    View Slide

59. Push
    Backend Clients
    API
    Database
    WebSocket
    Bus
    Subscribe
    Publish
    State change
    

    View Slide

60. Push
    Backend Clients
    API
    Database
    WebSocket
    {"cmd”:"listUsers"}
    

    View Slide

61. Push
    Backend Clients
    API
    Database
    WebSocket
    "CONNECT"
    "LIST_USERS"
    

    View Slide

62. type MessageEndpoint func(ctx context.Context, m Message, w *Writer) error
    type Router struct {
    endpoints map[string]MessageEndpoint
    writer map[string]*Writer
    }
    func (r *Router) HandleConnection(conn *websocket.Conn) {
    w := r.NewWriter(conn)
    for {
    _, message, err := conn.ReadMessage()
    if err != nil {
    break
    }
    request := string(message)
    r.writer[w.ID] = w
    r.Route(context.Background(), w, request)
    }
    delete(r.writer, r.ID(conn))
    }
    

    View Slide

63. type MessageEndpoint func(ctx context.Context, m Message, w *Writer) error
    type Router struct {
    endpoints map[string]MessageEndpoint
    writer map[string]*Writer
    }
    func (r *Router) HandleConnection(conn *websocket.Conn) {
    w := r.NewWriter(conn)
    for {
    _, message, err := conn.ReadMessage()
    if err != nil {
    break
    }
    request := string(message)
    r.writer[w.ID] = w
    go r.Route(context.Background(), w, request)
    }
    delete(r.writer, r.ID(conn))
    }
    

    View Slide

64. type Writer struct {
    ID int
    conn *websocket.Conn
    }
    func (w *Writer) Write(data []byte) (int, error) {
    err := w.conn.WriteMessage(websocket.TextMessage, data)
    if err != nil {
    return 0, err
    }
    return len(data), nil
    }
    

    View Slide

65. type Writer struct {
    ID int
    conn *websocket.Conn
    }
    func (w *Writer) Write(data []byte) (int, error) {
    err := w.conn.WriteMessage(websocket.TextMessage, data)
    if err != nil {
    return 0, err
    }
    return len(data), nil
    }
    panic: concurrent write to websocket connection
    

    View Slide

66. type Writer struct {
    ID int
    conn *websocket.Conn
    }
    func (w *Writer) Write(data []byte) (int, error) {
    err := w.conn.WriteMessage(websocket.TextMessage, data)
    if err != nil {
    return 0, err
    }
    return len(data), nil
    }
    

    View Slide

67. type Writer struct {
    ID int
    mu sync.RWMutex
    conn *websocket.Conn
    }
    func (w *Writer) Write(data []byte) (int, error) {
    w.mu.Lock()
    defer w.mu.Unlock()
    err := w.conn.WriteMessage(websocket.TextMessage, data)
    if err != nil {
    return 0, err
    }
    return len(data), nil
    }
    

    View Slide

68. Summary
    - REST can be implemented using WebSockets
    - WebSockets allow us to implement a push architecture
    - Freedom in designing a sub-protocol tailored to your system’s needs
    - Also useful for connecting backend components
    - Gorilla WebSocket is a great implementation of RFC 6455
    - Rolling your own implementation can be costly
    - Bad compatibility to the outside world
    

    View Slide

69. Not Invented
    Here
    Proudly Found
    Elsewhere
    

    View Slide

70. Thank you.
    @konradreiche
    

    View Slide

71. Questions?
    @konradreiche
    

    View Slide