Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Deep Dive into Python Requests

214c694acb154321379cbc58dc91528c?s=47 Cory Benfield
September 20, 2014

A Deep Dive into Python Requests

In this not-actually-so-deep dive into requests we'll do a lightning sweep through some of requests' lesser-known features that you could be using to make your code faster, cleaner and more beautiful.

214c694acb154321379cbc58dc91528c?s=128

Cory Benfield

September 20, 2014
Tweet

Transcript

  1. A Deep Dive Into Requests

  2. Hi

  3. @lukasaoz @lukasa

  4. None
  5. Requests Core Contributor urllib3 Core Contributor Maintainer of 10+ others

  6. You Know This r  =  requests.get('http://www.google.com/')   print  r.content

  7. Let’s Learn Something New

  8. Sessions

  9. Sessions • Requests’ Persistence Layer • Manage cookies • Avoid

    boilerplate • Connection Pooling!
  10. Cookie Management >>>  import  requests   >>>  s  =  requests.Session()

      >>>  s.get('http://www.google.com/')   >>>  len(s.cookies)   2
  11. Avoid Boilerplate >>>  s  =  requests.Session()   >>>  s.auth  =

     ('Cory',  'LOLNO')   >>>  s.get('http://some-­‐protected-­‐resource.org/')
  12. Connection Pooling #  Two  TCP  Connections   >>>  requests.get('http://www.google.com/')  

    >>>  requests.get('http://www.google.com/')   #  Re-­‐use  the  same  TCP  connection   >>>  s  =  requests.Session()   >>>  s.get('http://www.google.com/')   >>>  s.get('http://www.google.com/')
  13. Transport Adapters • Change the way requests manages connections. •

    Useful for applying connection- scope changes. • Think: SSL version, protocol etc.
  14. Streaming #  Does  not  download  response  body   >>>  r

     =  requests.get(url,  stream=True)   #  Now,  iterate  over  the  body,  in  chunks...   >>>  [x  for  x  in  r.iter_content(1024)]   #  or  lines...   >>>  [x  for  x  in  r.iter_lines()]   #  Downloads  all  at  once   >>>  r.content
  15. SSL Client Certs >>>  r  =  requests.get(url,  cert=(    

                 'cert.crt',  ‘cert.key’          ))
  16. Unusual Auth • OAuth1 and 2: requests_oauthlib • Kerberos: requests_kerberos

    • NTLM: requests_ntlm
  17. Related Projects • requests-toolbelt: Utilities • trequests, treq, grequests, async-

    requests • requests-ftp • urllib3, hyper
  18. Thanks! @lukasaoz @lukasa