Beyond HTTP, Breaking Free of The Web

Talk I gave at APIdays in Sydney 2015 and Microservices Meetup

Transcript

1. Beyond HTTP, Breaking
   Free of The Web
   Mark Wolfe
   

   View Slide

2. –From http://www.w3.org/DesignIssues/Architecture.html
   “The most fundamental specification of Web
   architecture, while one of the simpler, is that of the
   Universal Resource Identifier, or URI.”
   

   View Slide

3. –From https://tools.ietf.org/html/rfc2616
   “The Hypertext Transfer Protocol (HTTP) is an
   application protocol for distributed, collaborative,
   hypermedia information systems.”
   

   View Slide

4. – From http://en.wikipedia.org/wiki/Application_programming_interface
   “In computer programming, an application
   programming interface (API) is a set of routines,
   protocols, and tools for building software
   applications.”
   

   View Slide

5. APIs We Use != http
   DNS
   Used to resolve host names
   Also used for service discovery
   MDNS
   SPDY
   

   View Slide

6. Wireshark
   Network protocol analyser
   Our window into the wonderful world of protocols and
   APIs
   Available for Windows, OSX and Linux
   https://www.wireshark.org/
   The wire doesn’t lie
   

   View Slide

7. DNS?
   Protocol uses UDP or TCP for transport
   Consists of Questions and Answers
   Includes some information on authority
   Uses "intelligent retransmission”
   Small messages
   

   View Slide

8. DNS Continued..
   Binary protocol
   Fixed header
   Payload uses a compact self describing binary
   format
   TLV (type length value)
   

   View Slide

9. DNS Question
   

   View Slide

10. DNS Answer
    

    View Slide

11. DNS Exchange
    

    View Slide

12. HTTP Get Request
    

    View Slide

13. In Review
    Compact messages
    More likely to transit “wireless” networks
    Extensible self describing binary payload
    Established encoding so others can introspect
    without a definition!
    Fast
    

    View Slide

14. MDNS
    Uses UDP
    Over Multicast
    Designed for resolving names in adhoc networks
    Uses the existing DNS message structure, name
    syntax, and resource record types
    

    View Slide

15. Something Simpler?
    So lets look at something simpler than DNS
    What if we just put text in a UDP packet
    What would that look like?
    

    View Slide

16. Problem
    I want to send 10s of 1000s of metrics to a server for
    aggregation with minimal performance impact
    I want to keep it simple
    I want the server to be a staging area for data
    Perform Aggregation
    Flush to timeseries at regular intervals
    

    View Slide

17. statsd
    Developed by etsy
    Uses UDP as a transport
    Data is encoded using text
    Metrics are either a Gauge, Counter, Timer or Meter
    No reply necessary!
    

    View Slide

18. statsd protocol
    Metrics are generally of the form name>:|
    outside_temp|12.2|g
    get_home_request_time|123|ms
    get_home_request|1|c
    

    View Slide

19. In Review
    Very simple implementation
    Doesn’t block
    Fast
    Loss doesn’t really matter in the scheme of things
    Maths happens in the server
    

    View Slide

20. View Slide

21. The End
    @wolfeidau on twitter
    https://github.com/wolfeidau
    mark@wolfe.id.au
    http://wolfe.id.au
    

    View Slide

22. Links
    Good article on DNS message format
    https://www.wireshark.org/
    Read this book High Performance Browser Networking
    Etsy Statsd
    

    View Slide