Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Deep Dive into Python Requests

Cory Benfield
September 20, 2014

A Deep Dive into Python Requests

In this not-actually-so-deep dive into requests we'll do a lightning sweep through some of requests' lesser-known features that you could be using to make your code faster, cleaner and more beautiful.

Cory Benfield

September 20, 2014
Tweet

More Decks by Cory Benfield

Other Decks in Programming

Transcript

  1. A Deep Dive
    Into Requests

    View full-size slide

  2. @lukasaoz
    @lukasa

    View full-size slide

  3. Requests Core Contributor
    urllib3 Core Contributor
    Maintainer of 10+ others

    View full-size slide

  4. You Know This
    r  =  requests.get('http://www.google.com/')  
    print  r.content

    View full-size slide

  5. Let’s Learn
    Something New

    View full-size slide

  6. Sessions
    • Requests’ Persistence Layer
    • Manage cookies
    • Avoid boilerplate
    • Connection Pooling!

    View full-size slide

  7. Cookie Management
    >>>  import  requests  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  len(s.cookies)  
    2

    View full-size slide

  8. Avoid Boilerplate
    >>>  s  =  requests.Session()  
    >>>  s.auth  =  ('Cory',  'LOLNO')  
    >>>  s.get('http://some-­‐protected-­‐resource.org/')

    View full-size slide

  9. Connection Pooling
    #  Two  TCP  Connections  
    >>>  requests.get('http://www.google.com/')  
    >>>  requests.get('http://www.google.com/')  
    #  Re-­‐use  the  same  TCP  connection  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  s.get('http://www.google.com/')

    View full-size slide

  10. Transport Adapters
    • Change the way requests
    manages connections.
    • Useful for applying connection-
    scope changes.
    • Think: SSL version, protocol etc.

    View full-size slide

  11. Streaming
    #  Does  not  download  response  body  
    >>>  r  =  requests.get(url,  stream=True)  
    #  Now,  iterate  over  the  body,  in  chunks...  
    >>>  [x  for  x  in  r.iter_content(1024)]  
    #  or  lines...  
    >>>  [x  for  x  in  r.iter_lines()]  
    #  Downloads  all  at  once  
    >>>  r.content

    View full-size slide

  12. SSL Client Certs
    >>>  r  =  requests.get(url,  cert=(  
                   'cert.crt',  ‘cert.key’  
           ))

    View full-size slide

  13. Unusual Auth
    • OAuth1 and 2:
    requests_oauthlib
    • Kerberos: requests_kerberos
    • NTLM: requests_ntlm

    View full-size slide

  14. Related Projects
    • requests-toolbelt: Utilities
    • trequests, treq, grequests, async-
    requests
    • requests-ftp
    • urllib3, hyper

    View full-size slide

  15. Thanks!
    @lukasaoz
    @lukasa

    View full-size slide