Cutting off the Internet: Testing Applications that Use Requests

A brief and opinionated view of testing applications and libraries that use requests by a core-developer of requests. You will receive an overview of testing with responses, vcr, httpretty, mock, and betamax. (Video: https://www.youtube.com/watch?v=YHbKxFcDltM)

Transcript

1. @sigmavirus24
   c u t t i n g
   Testing Applications that Use Requests
   o f f
   the
   Internet
   @sigmavirus24
   

   View Slide

2. @sigmavirus24
   requests
   flake8
   toolbelt
   urllib3
   twine
   github3.py
   chardet
   sqlobject
   python-guide
   OpenStack
   

   View Slide

3. View Slide

4. @sigmavirus24
   Why give
   this talk?
   

   View Slide

5. @sigmavirus24
   Why give
   this talk?
   

   View Slide

6. @sigmavirus24
   r e v i e w
   

   View Slide

7. @sigmavirus24
   Unit of
   Code
   

   View Slide

8. @sigmavirus24
   Unit Test
   

   View Slide

9. @sigmavirus24
   Collaborators
   

   View Slide

10. @sigmavirus24
    Isolation
    

    View Slide

11. @sigmavirus24
    class Bar(object):
    def __init__(self):
    self.foo = Foo()
    def baz(self, *args):
    a = self.foo.fiz()
    # other stuff
    

    View Slide

12. @sigmavirus24
    Integration
    Tests
    

    View Slide

13. @sigmavirus24
    def get_resource(
    session,
    resource,
    params=None,
    headers=None):
    

    View Slide

14. @sigmavirus24
    url = url_from(resource)
    kw = {‘headers’: headers,
    ‘params’: params}
    resp = session.get(url,
    **kw)
    if resp.ok:
    return resp
    resp.raise_for_status()
    

    View Slide

15. @sigmavirus24
    Too
    Simple to
    Test?
    

    View Slide

16. @sigmavirus24
    D O I T
    L I V E !
    

    View Slide

17. @sigmavirus24
    session = Session()
    response = get_resource(
    session, ‘users’)
    assert response.ok
    

    View Slide

18. @sigmavirus24
    Now do
    that 400
    times
    

    View Slide

19. @sigmavirus24
    Ratelimits
    

    View Slide

20. @sigmavirus24
    ✓✓✓✓✓✓FFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    

    View Slide

21. @sigmavirus24
    I m i t a t i o n
    i s t h e
    s i n c e r e s t
    f o r m o f
    F l a t t e r y
    

    View Slide

22. @sigmavirus24
    responses
    httpretty
    requests-mock
    mock
    

    View Slide

23. @sigmavirus24
    session = mock.Mock()
    get_resource(session,
    ‘users’)
    

    View Slide

24. @sigmavirus24
    (session.get.
    assert_called_once_with(
    ‘https://api.’
    ‘github.com/users’,
    headers=None,
    params=None,
    ))
    

    View Slide

25. @sigmavirus24
    $ py.test mock_ex.py
    mock_ex.py ✓
    

    View Slide

26. @sigmavirus24
    sess = mock.Mock()
    resp = mock.Mock()
    resp.ok = False
    get = sess.get
    get.return_value = resp
    

    View Slide

27. @sigmavirus24
    get_resource(session,
    ‘users’)
    rfs = resp.raise_for_status
    assert rfs.called is True
    

    View Slide

28. @sigmavirus24
    $ py.test mock_ex.py
    mock_ex.py ✓✓
    

    View Slide

29. @sigmavirus24
    Mock
    

    View Slide

30. @sigmavirus24
    Verbose
    

    View Slide

31. @sigmavirus24
    Verbose
    

    View Slide

32. @sigmavirus24
    P r e t e n d
    t o b e
    T h e
    N . S . A .
    

    View Slide

33. @sigmavirus24
    vcr.py
    betamax
    

    View Slide

34. @sigmavirus24
    vcr.py
    betamax
    

    View Slide

35. @sigmavirus24
    vcr.py
    

    View Slide

36. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0
    

    View Slide

37. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0
    

    View Slide

38. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0
    

    View Slide

39. @sigmavirus24
    $ py.test vcr_ex.py
    vcr_ex.py ✓
    

    View Slide

40. @sigmavirus24
    betamax
    

    View Slide

41. @sigmavirus24
    session = Session()
    rec = betamax.Betamax(
    session,
    cassette_library_dir=‘.’)
    

    View Slide

42. @sigmavirus24
    with rec.use_cassette(‘betamax’):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0
    

    View Slide

43. @sigmavirus24
    $ py.test betmax_ex.py
    betamax_ex.py ✓
    

    View Slide

44. @sigmavirus24
    self.session = Session()
    self.recorder = Betamax(
    session)
    self.recorder.use_cassette(
    cassette_name())
    self.recorder.start()
    

    View Slide

45. @sigmavirus24
    resp = get_resource(
    self.session,
    ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0
    

    View Slide

46. @sigmavirus24
    self.recorder.stop()
    

    View Slide

47. @sigmavirus24
    vcr.py
    betamax
    

    View Slide

48. @sigmavirus24
    vcr.py
    betamax
    

    View Slide

49. @sigmavirus24
    T h e
    r e a l i t y
    

    View Slide

50. @sigmavirus24
    Two part
    solution
    

    View Slide

51. @sigmavirus24
    Two part
    solution
    

    View Slide

52. @sigmavirus24
    Mocking
    mock, requests-mock,
    responses, httpretty
    

    View Slide

53. @sigmavirus24
    Integration
    betamax, vcr.py
    

    View Slide

54. @sigmavirus24
    T h a n k
    y o u
    

    View Slide