Beyond HTTP, Breaking Free of The Web

Talk I gave at APIdays in Sydney 2015 and Microservices Meetup

1. Beyond HTTP, Breaking Free of The Web Mark Wolfe
2. –From http://www.w3.org/DesignIssues/Architecture.html “The most fundamental specification of Web architecture, while one of the simpler, is that of the Universal Resource Identifier, or URI.”
3. –From https://tools.ietf.org/html/rfc2616 “The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems.”
4. – From http://en.wikipedia.org/wiki/Application_programming_interface “In computer programming, an application programming interface (API) is a set of routines, protocols, and tools for building software applications.”
5. APIs We Use != http DNS Used to resolve host names Also used for service discovery MDNS SPDY
6. Wireshark Network protocol analyser Our window into the wonderful world of protocols and APIs Available for Windows, OSX and Linux https://www.wireshark.org/ The wire doesn’t lie
7. DNS? Protocol uses UDP or TCP for transport Consists of Questions and Answers Includes some information on authority Uses "intelligent retransmission” Small messages
8. DNS Continued.. Binary protocol Fixed header Payload uses a compact self describing binary format TLV (type length value)
9. DNS Question
10. DNS Answer
11. DNS Exchange
12. HTTP Get Request
13. In Review Compact messages More likely to transit “wireless” networks Extensible self describing binary payload Established encoding so others can introspect without a definition! Fast
14. MDNS Uses UDP Over Multicast Designed for resolving names in adhoc networks Uses the existing DNS message structure, name syntax, and resource record types
15. Something Simpler? So lets look at something simpler than DNS What if we just put text in a UDP packet What would that look like?
16. Problem I want to send 10s of 1000s of metrics to a server for aggregation with minimal performance impact I want to keep it simple I want the server to be a staging area for data Perform Aggregation Flush to timeseries at regular intervals
17. statsd Developed by etsy Uses UDP as a transport Data is encoded using text Metrics are either a Gauge, Counter, Timer or Meter No reply necessary!
18. statsd protocol Metrics are generally of the form <metric name>:<value>|<type> outside_temp|12.2|g get_home_request_time|123|ms get_home_request|1|c
19. In Review Very simple implementation Doesn’t block Fast Loss doesn’t really matter in the scheme of things Maths happens in the server
21. The End @wolfeidau on twitter https://github.com/wolfeidau mark@wolfe.id.au http://wolfe.id.au
22. Links Good article on DNS message format https://www.wireshark.org/ Read this book High Performance Browser Networking Etsy Statsd