Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Deep Dive into Python Requests

Cory Benfield
September 20, 2014

A Deep Dive into Python Requests

In this not-actually-so-deep dive into requests we'll do a lightning sweep through some of requests' lesser-known features that you could be using to make your code faster, cleaner and more beautiful.

Cory Benfield

September 20, 2014
Tweet

More Decks by Cory Benfield

Other Decks in Programming

Transcript

  1. A Deep Dive
    Into Requests

    View Slide

  2. Hi

    View Slide

  3. @lukasaoz
    @lukasa

    View Slide

  4. View Slide

  5. Requests Core Contributor
    urllib3 Core Contributor
    Maintainer of 10+ others

    View Slide

  6. You Know This
    r  =  requests.get('http://www.google.com/')  
    print  r.content

    View Slide

  7. Let’s Learn
    Something New

    View Slide

  8. Sessions

    View Slide

  9. Sessions
    • Requests’ Persistence Layer
    • Manage cookies
    • Avoid boilerplate
    • Connection Pooling!

    View Slide

  10. Cookie Management
    >>>  import  requests  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  len(s.cookies)  
    2

    View Slide

  11. Avoid Boilerplate
    >>>  s  =  requests.Session()  
    >>>  s.auth  =  ('Cory',  'LOLNO')  
    >>>  s.get('http://some-­‐protected-­‐resource.org/')

    View Slide

  12. Connection Pooling
    #  Two  TCP  Connections  
    >>>  requests.get('http://www.google.com/')  
    >>>  requests.get('http://www.google.com/')  
    #  Re-­‐use  the  same  TCP  connection  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  s.get('http://www.google.com/')

    View Slide

  13. Transport Adapters
    • Change the way requests
    manages connections.
    • Useful for applying connection-
    scope changes.
    • Think: SSL version, protocol etc.

    View Slide

  14. Streaming
    #  Does  not  download  response  body  
    >>>  r  =  requests.get(url,  stream=True)  
    #  Now,  iterate  over  the  body,  in  chunks...  
    >>>  [x  for  x  in  r.iter_content(1024)]  
    #  or  lines...  
    >>>  [x  for  x  in  r.iter_lines()]  
    #  Downloads  all  at  once  
    >>>  r.content

    View Slide

  15. SSL Client Certs
    >>>  r  =  requests.get(url,  cert=(  
                   'cert.crt',  ‘cert.key’  
           ))

    View Slide

  16. Unusual Auth
    • OAuth1 and 2:
    requests_oauthlib
    • Kerberos: requests_kerberos
    • NTLM: requests_ntlm

    View Slide

  17. Related Projects
    • requests-toolbelt: Utilities
    • trequests, treq, grequests, async-
    requests
    • requests-ftp
    • urllib3, hyper

    View Slide

  18. Thanks!
    @lukasaoz
    @lukasa

    View Slide