Upgrade to Pro — share decks privately, control downloads, hide ads and more …

HTTP Tunneling in Go

HTTP Tunneling in Go

HTTP Tunneling can be used in situations similar to the requirements of using VPN, for example, to be able to access a localhost server from outside. This talk will introduce the specific implementation of HTTP tunneling and give audiences a deep knowledge of TCP and HTTP level implementation in Go.

Specifically, this talk discusses the idea of HTTP Tunneling in Go.

Kazuki Higashiguchi

April 16, 2022
Tweet

More Decks by Kazuki Higashiguchi

Other Decks in Technology

Transcript

  1. Agenda HTTP(S) Client WebSocket server WebSocket client HTTP(S) Server 1.

    HTTP Tunneling 2. Establish bidirectional connection using WebSocket 3. HTTP Tunneling over WebSocket
  2. Autify Solution No-code AI-powered software testing automation services for Web/Mobile

    applications AI automatically maintenance test scripts No code unlocks automation at scale
  3. Agenda HTTP(S) Client WebSocket server WebSocket client HTTP(S) Server 1.

    HTTP Tunneling 2. Establish bidirectional connection using WebSocket 3. HTTP Tunneling over WebSocket
  4. HTTP tunneling • Create a network link between two computers

    in conditions of restricted network connectivity. e.g. firewalls, NATs, ACLs • Use a protocol of higher level (HTTP) to transport a lower level protocol (TCP)
  5. HTTP CONNECT method • Start bidirectional communications with the requested

    resource e.g. access HTTPS websites, HTTP proxy server HTTP(S) Client HTTP(S) Server Proxy (Gateway)
  6. HTTP tunneling with CONNECT Quoted from HTTP: The Definitive Guide

    / 8.5 Tunnels 1. Client sends CONNECT request 2. Gateway opens TCP connection to the server 3. Gateway returns HTTP ready message to the client 4. Start bidirectional communication of raw packets of data
  7. A case study: elazarl/goproxy elazarl/goproxy: An HTTP proxy library for

    Go $ curl -I -x http://localhost:8080 https://example.com HTTP/1.0 200 OK A basic sample code:
  8. Step 1: Take over the proxy connection http.Hijacker interface allows

    an HTTP handler to take over the connection.
  9. Step 2: Open TCP connections Opens TCP connection using net.Dial

    Returns HTTP ready message: 200 Connection established
  10. Streaming: TLS handshake over TCP connection • TLS handshake is

    done using the streaming • Application data is sent and receive after TLS handshake
  11. Key takeaways • HTTP Tunneling uses a protocol of higher

    level (HTTP) to transport a lower level protocol (TCP) • The basic flow of HTTP tunneling using the HTTP Connect method • http.Hijack and io.Copy are key functions to implement HTTP tunneling in Go
  12. Agenda HTTP(S) Client WebSocket server WebSocket client HTTP(S) Server 1.

    HTTP Tunneling 2. Establish bidirectional connection using WebSocket 3. HTTP Tunneling over WebSocket
  13. Establish bidirectional connections WebSocket server WebSocket client Handshake WebSocket Conn

    WebSocket Conn HTTP(S) Client WebSocket server WebSocket client HTTP(S) Server
  14. WebSockets • A mechanism for low-cost, full-duplex communication on Web

    • Designed to work over HTTP (compatible with the HTTP protocol) • Handshake -> Messages -> Close
  15. websocket.Conn implementation websocket.Conn has net.Conn in the unexported field. The

    return value of http.Hijack will be set to the field conn. https://github.com/gorilla/websocket/blob/eeb877e8a1fd26c449b7af3cb1d71879287d724c/server.go#L125
  16. Agenda HTTP(S) Client WebSocket server WebSocket client HTTP(S) Server 1.

    HTTP Tunneling 2. Establish bidirectional connection using WebSocket 3. HTTP Tunneling over WebSocket
  17. HTTP Tunneling over WebSocket HTTP(S) Client HTTP(S) Server WebSocket server

    WebSocket client WebSocket Conn WebSocket Conn TCP Conn TCP Conn $ curl -Lv -x http://websocket-server https://target.local WebSocket server and WebSocket client jointly act as a proxy(gateway). https://target.local http://websocket-server streaming streaming
  18. How HTTP Tunneling with WebSocket WebSocket server tells the address

    of destination to WebSocket client. WebSocket client open a TCP connection to the destination server.
  19. Step 1: Write messages to WebSocket connection Send a TCP

    handshake request with the destination address.
  20. Distinguish several types of messages This design transfers several kinds

    of message over a single WebSocket connection • Request to open TCP connections • Notify to establish TCP connections • Streaming raw packets of data • etc
  21. A case study: Chrome Devtools Protocol CDP defines the JSON

    protocol, which is used to communicate in the WebSocket connection. Request Members: id, method, params Response Members: id, result, (error)
  22. Define own protocol for WebSocket communication A small example of

    methods • Handshake: Request to open TCP connections • Completed: Notify to establish TCP connections • Communication: Streaming raw packets of data
  23. Step 2: Take over the proxy connection 2 Take over

    the connection using http.Hijacker.
  24. Notes on net.Conn returned from http.Hijack • http.Hijacker returns net.Conn

    and bufio.ReadWriter. [Code comment on http.Hijack] The returned bufio.Reader may contain unprocessed buffered data from the client. • Reading net.Conn alone may cause data to be dropped. • Read buffers from bufio.Reader.
  25. Wrap websocket.Conn to use in io.Copy: io.Writer Pack incoming packets

    to defined JSON format ンu��(gT��!"�. ��ҠF��0�Q��Yl/��L�E8/���c<��>���C� ��m$�X�Ax�Y%^��J�v�[�o�U�a���U�t �u��5��k��m��&|Ǜq����)�z7����?�… { "method": 3, "data": "ンu��(gT��!"�. ��ҠF��0�Q��Yl/��L�E8/���c<��>� ��C���m$�X�Ax�Y%^��J�v� [�o�U�a���U�t �u��5��k��m��&|Ǜq����) �z7����?�… " }
  26. Wrap websocket.Conn to use in io.Copy: io.Reader Read packets from

    a decoded JSON message ンu��(gT��!"�. ��ҠF��0�Q��Yl/��L�E8/���c<��>���C� ��m$�X�Ax�Y%^��J�v�[�o�U�a���U�t �u��5��k��m��&|Ǜq����)�z7����?�… { "method": 3, "data": "ンu��(gT��!"�. ��ҠF��0�Q��Yl/��L�E8/���c<��>� ��C���m$�X�Ax�Y%^��J�v� [�o�U�a���U�t �u��5��k��m��&|Ǜq����) �z7����?�… " }
  27. Open a TCP connection to the given destination Dial to

    the destination address conn, err := net.Dial("tcp", “autify.com:443”)
  28. Key takeaways • HTTP Tunneling over WebSocket • Define own

    protocol for WebSocket communication and create a wrap struct for streaming with io.Copy • Should read both net.Conn and bufio.ReadWriter returned from http.Hijacker
  29. Resources • RFC 6455 The WebSocket Protocol • IETF: Tunneling

    TCP based protocols through Web proxy servers • Mmdn web doc: Proxy servers and tunneling • Mmdn web doc: CONNECT • Wikipedia: HTTP Tunnel • Wikipedia: WebSocket • HTTP: The Definitive Guide / 8.5 Tunnels • Cloudflare: What happens in a TLS handshake? | SSL handshake • Chrome DevTools Protocol
  30. Thank you for your watching We are taking demo requests

    https://autify.com/ Autify for Web Autify for Mobile