A Deep Dive into Python Requests

In this not-actually-so-deep dive into requests we'll do a lightning sweep through some of requests' lesser-known features that you could be using to make your code faster, cleaner and more beautiful.

Transcript

1. A Deep Dive
   Into Requests
   

   View Slide

2. Hi
   

   View Slide

3. @lukasaoz
   @lukasa
   

   View Slide

4. View Slide

5. Requests Core Contributor
   urllib3 Core Contributor
   Maintainer of 10+ others
   

   View Slide

6. You Know This
   r  =  requests.get('http://www.google.com/')  
   print  r.content
   

   View Slide

7. Let’s Learn
   Something New
   

   View Slide

8. Sessions
   

   View Slide

9. Sessions
   • Requests’ Persistence Layer
   • Manage cookies
   • Avoid boilerplate
   • Connection Pooling!
   

   View Slide

10. Cookie Management
    >>>  import  requests  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  len(s.cookies)  
    2
    

    View Slide

11. Avoid Boilerplate
    >>>  s  =  requests.Session()  
    >>>  s.auth  =  ('Cory',  'LOLNO')  
    >>>  s.get('http://some-­‐protected-­‐resource.org/')
    

    View Slide

12. Connection Pooling
    #  Two  TCP  Connections  
    >>>  requests.get('http://www.google.com/')  
    >>>  requests.get('http://www.google.com/')  
    #  Re-­‐use  the  same  TCP  connection  
    >>>  s  =  requests.Session()  
    >>>  s.get('http://www.google.com/')  
    >>>  s.get('http://www.google.com/')
    

    View Slide

13. Transport Adapters
    • Change the way requests
    manages connections.
    • Useful for applying connection-
    scope changes.
    • Think: SSL version, protocol etc.
    

    View Slide

14. Streaming
    #  Does  not  download  response  body  
    >>>  r  =  requests.get(url,  stream=True)  
    #  Now,  iterate  over  the  body,  in  chunks...  
    >>>  [x  for  x  in  r.iter_content(1024)]  
    #  or  lines...  
    >>>  [x  for  x  in  r.iter_lines()]  
    #  Downloads  all  at  once  
    >>>  r.content
    

    View Slide

15. SSL Client Certs
    >>>  r  =  requests.get(url,  cert=(  
                   'cert.crt',  ‘cert.key’  
           ))
    

    View Slide

16. Unusual Auth
    • OAuth1 and 2:
    requests_oauthlib
    • Kerberos: requests_kerberos
    • NTLM: requests_ntlm
    

    View Slide

17. Related Projects
    • requests-toolbelt: Utilities
    • trequests, treq, grequests, async-
    requests
    • requests-ftp
    • urllib3, hyper
    

    View Slide

18. Thanks!
    @lukasaoz
    @lukasa
    

    View Slide