Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Working with HTTP in Ruby: Tips, Tricks, and Te...

Working with HTTP in Ruby: Tips, Tricks, and Techniques.

Slides from "Barcelona on Rails" meetup (16.11.2017)

Avatar for Andrey Deryabin

Andrey Deryabin

November 19, 2017
Tweet

More Decks by Andrey Deryabin

Other Decks in Programming

Transcript

  1. HTTP 1.1 – most popular protocol in WEB – a

    stateless – request => response
  2. curl -v https: //google.com > GET / HTTP/1.1 > Host:

    google.com > User-Agent: curl/7.51.0 > Accept: */* > < HTTP/1.1 302 Found < Cache-Control: private < Content-Type: text/html; charset=UTF-8 < Location: https: // www.google.es/?gfe_rd=cr&dcr=0&ei=XIINWr2XIuis8wfFipPwDQ < Content-Length: 269 < <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>302 Moved </TITLE> </HEAD><BODY> <H1>302 Moved </H1> The document has moved <A HREF="https: // www.google.es/? gfe_rd=cr&amp;dcr=0&amp;ei=XIINWr2XIuis8wfFipPwDQ">here </A>. </BODY> </HTML> Method Resource Protocol Request Response Body Headers
  3. History – HTTP/1.1 (v1) - 1999 (RFCs 2616, 7230, 7231,

    7232, 7233, 7234 and 7235) – HTTP/2.0 -2015 (RFC 7540)
  4. HTTP/2 – Binary framing parsing and encoding – Query multiplexing

    – Headers Compression – Connection and stream management – And more and more
  5. Ruby HTTP Clients Net::HTTP RestClient Faraday Ethcon Curb Typhoeus HTTP.rb

    HTTPClient HTTParty Patron EM-HTTP-Request Excon
  6. Ruby HTTP Clients Faraday Ethcon Curb Typhoeus Patron Libcurl EM-HTTP-Request

    Excon Event machine HTTParty Net::HTTP RestClient HTTP.rb HTTPClient TCPSocket Net::HTTP
  7. Net::HTTP # request uri = URI.parse('http: //localhost:4567/json') http = Net

    ::HTTP.new(uri.host, uri.port) request = Net ::HTTP ::Post.new(uri.request_uri, 'Content-Type' => 'text/json') request.body = { ‘first_name' => 'Leo', ‘last_name' => 'Messi' }.to_json http.request(request) # response [201, { "Сontent-length" => "7", 'Connection' => "Barca" }, "Created"]
  8. Net::HTTP { "content-type" =>"text/html;charset=utf-8", "connection" =>"Barca, close", "x-xss-protection" =>"1; mode=block",

    "x-content-type-options" =>"nosniff", "x-frame-options" =>"SAMEORIGIN", "content-length" =>"7" } WHAT?
  9. Net::HTTP # [201, { "Сontent-length" => "7", 'Connection' => "Barca"

    }, "Created"] { "content-type" =>"text/html;charset=utf-8", "connection" =>"Barca, close", "x-xss-protection" =>"1; mode=block", "x-content-type-options" =>"nosniff", "x-frame-options" =>"SAMEORIGIN", "content-length" =>"7" } WHAT?
  10. RFC 2616 - "Hypertext Transfer Protocol -- HTTP/1.1", Section 4.2,

    "Message Headers" Each header field consists of a name followed by a colon (":") and the field value. Field names are case- insensitive
  11. # https: //github.com/mokevnin/ipgeobase/blob/master/lib/ipgeobase.rb require 'uri' require 'open-uri' module Ipgeobase URL

    = 'http: //ipgeobase.ru:7020/geo' autoload 'IpMetaData', 'ipgeobase/ip_meta_data' def self.lookup(ip, params = {}) uri = URI.parse(URL) uri.query = URI.encode_ www_form :ip => ip resp = open(uri, params).read() IpMetaData.parse(resp.to_s) end end
  12. Sniffer.config do logger: Logger.new($stdout), severity: Logger ::Severity ::DEBUG, # HTTP

    options to log log: { request_url: true, request_headers: true, request_body: true, request_method: true, response_status: true, response_headers: true, response_body: true, timing: true }, store: true, # save requests/responses to Sniffer.data enabled: false # Sniffer disabled by default end
  13. pry(main)> Sniffer.enable!; pry(main)> client = Elasticsearch ::Client.new; pry(main)> client.search q:

    'Barcelona on Rails'; D, [2017-11-16T01:19:45.455444 #45358] DEBUG -- : {"port": 9200,"host":"localhost","query":"/_search? q=Barcelona+on+Rails","rq_content_type":"application/ json","rq_user_agent":"Faraday v0.11.0","rq_accept_encoding":"gzip;q=1.0,deflate;q=0.6,identity ;q=0.3","rq_accept":"*/ *","rq_connection":"close","method":"GET","request_body":"","sta tus":200,"rs_content_type":"application/json; charset=UTF-8","rs_content_length":"124","timing": 1.6817439999431372,"response_body":"{\"took\": 989,\"timed_out\":false,\"_shards\":{\"total\":5,\"successful\": 5,\"failed\":0},\"hits\":{\"total\":0,\"max_score\":null, \"hits\":[]}}"} Example pry(main)> Sniffer.enable!; pry(main)> client = Elasticsearch ::Client.new; pry(main)> client.search q: 'Barcelona on Rails'; D, [2017-11-16T01:19:45.455444 #45358] DEBUG -- : {"port": 9200,"host":"localhost","query":"/_search? q=Barcelona+on+Rails","rq_content_type":"application/ json","rq_user_agent":"Faraday v0.11.0","rq_accept_encoding":"gzip;q=1.0,deflate;q=0.6,identity ;q=0.3","rq_accept":"*/ *","rq_connection":"close","method":"GET","request_body":"","sta tus":200,"rs_content_type":"application/json; charset=UTF-8","rs_content_length":"124","timing": 1.6817439999431372,"response_body":"{\"took\": 989,\"timed_out\":false,\"_shards\":{\"total\":5,\"successful\": 5,\"failed\":0},\"hits\":{\"total\":0,\"max_score\":null, \"hits\":[]}}"}
  14. Sniffer.data.first.request => #<Sniffer ::DataItem ::Request:0x00007fbe880cd700 @body="", @headers= {"content-type" =>"application/json", "user-agent"

    =>"Faraday v0.11.0", "accept- encoding" =>"gzip;q=1.0,deflate;q=0.6,identity;q=0.3", "accept" =>"*/*", "connection" =>"close"}, @host="localhost", @method="GET", @port=9200, @query="/_search?q=Barcelona+on+Rails"> Debug request