$30 off During Our Annual Pro Sale. View Details »

Cutting off the Internet: Testing Applications that Use Requests

Cutting off the Internet: Testing Applications that Use Requests

A brief and opinionated view of testing applications and libraries that use requests by a core-developer of requests. You will receive an overview of testing with responses, vcr, httpretty, mock, and betamax. (Video: https://www.youtube.com/watch?v=YHbKxFcDltM)

Ian Cordasco

April 14, 2015
Tweet

More Decks by Ian Cordasco

Other Decks in Programming

Transcript

  1. @sigmavirus24
    c u t t i n g
    Testing Applications that Use Requests
    o f f
    the
    Internet
    @sigmavirus24

    View Slide

  2. @sigmavirus24
    requests
    flake8
    toolbelt
    urllib3
    twine
    github3.py
    chardet
    sqlobject
    python-guide
    OpenStack

    View Slide

  3. View Slide

  4. @sigmavirus24
    Why give
    this talk?

    View Slide

  5. @sigmavirus24
    Why give
    this talk?

    View Slide

  6. @sigmavirus24
    r e v i e w

    View Slide

  7. @sigmavirus24
    Unit of
    Code

    View Slide

  8. @sigmavirus24
    Unit Test

    View Slide

  9. @sigmavirus24
    Collaborators

    View Slide

  10. @sigmavirus24
    Isolation

    View Slide

  11. @sigmavirus24
    class Bar(object):
    def __init__(self):
    self.foo = Foo()
    def baz(self, *args):
    a = self.foo.fiz()
    # other stuff

    View Slide

  12. @sigmavirus24
    Integration
    Tests

    View Slide

  13. @sigmavirus24
    def get_resource(
    session,
    resource,
    params=None,
    headers=None):

    View Slide

  14. @sigmavirus24
    url = url_from(resource)
    kw = {‘headers’: headers,
    ‘params’: params}
    resp = session.get(url,
    **kw)
    if resp.ok:
    return resp
    resp.raise_for_status()

    View Slide

  15. @sigmavirus24
    Too
    Simple to
    Test?

    View Slide

  16. @sigmavirus24
    D O I T
    L I V E !

    View Slide

  17. @sigmavirus24
    session = Session()
    response = get_resource(
    session, ‘users’)
    assert response.ok

    View Slide

  18. @sigmavirus24
    Now do
    that 400
    times

    View Slide

  19. @sigmavirus24
    Ratelimits

    View Slide

  20. @sigmavirus24
    ✓✓✓✓✓✓FFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

    View Slide

  21. @sigmavirus24
    I m i t a t i o n
    i s t h e
    s i n c e r e s t
    f o r m o f
    F l a t t e r y

    View Slide

  22. @sigmavirus24
    responses
    httpretty
    requests-mock
    mock

    View Slide

  23. @sigmavirus24
    session = mock.Mock()
    get_resource(session,
    ‘users’)

    View Slide

  24. @sigmavirus24
    (session.get.
    assert_called_once_with(
    ‘https://api.’
    ‘github.com/users’,
    headers=None,
    params=None,
    ))

    View Slide

  25. @sigmavirus24
    $ py.test mock_ex.py
    mock_ex.py ✓

    View Slide

  26. @sigmavirus24
    sess = mock.Mock()
    resp = mock.Mock()
    resp.ok = False
    get = sess.get
    get.return_value = resp

    View Slide

  27. @sigmavirus24
    get_resource(session,
    ‘users’)
    rfs = resp.raise_for_status
    assert rfs.called is True

    View Slide

  28. @sigmavirus24
    $ py.test mock_ex.py
    mock_ex.py ✓✓

    View Slide

  29. @sigmavirus24
    Mock

    View Slide

  30. @sigmavirus24
    Verbose

    View Slide

  31. @sigmavirus24
    Verbose

    View Slide

  32. @sigmavirus24
    P r e t e n d
    t o b e
    T h e
    N . S . A .

    View Slide

  33. @sigmavirus24
    vcr.py
    betamax

    View Slide

  34. @sigmavirus24
    vcr.py
    betamax

    View Slide

  35. @sigmavirus24
    vcr.py

    View Slide

  36. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0

    View Slide

  37. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0

    View Slide

  38. @sigmavirus24
    session = requests.Session()
    cassette = ‘vcrexample.yml’
    with vcr.use_cassette(cassette):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0

    View Slide

  39. @sigmavirus24
    $ py.test vcr_ex.py
    vcr_ex.py ✓

    View Slide

  40. @sigmavirus24
    betamax

    View Slide

  41. @sigmavirus24
    session = Session()
    rec = betamax.Betamax(
    session,
    cassette_library_dir=‘.’)

    View Slide

  42. @sigmavirus24
    with rec.use_cassette(‘betamax’):
    resp = get_resource(
    session, ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0

    View Slide

  43. @sigmavirus24
    $ py.test betmax_ex.py
    betamax_ex.py ✓

    View Slide

  44. @sigmavirus24
    self.session = Session()
    self.recorder = Betamax(
    session)
    self.recorder.use_cassette(
    cassette_name())
    self.recorder.start()

    View Slide

  45. @sigmavirus24
    resp = get_resource(
    self.session,
    ‘users’)
    assert resp.ok
    users = resp.json()
    assert len(users) > 0

    View Slide

  46. @sigmavirus24
    self.recorder.stop()

    View Slide

  47. @sigmavirus24
    vcr.py
    betamax

    View Slide

  48. @sigmavirus24
    vcr.py
    betamax

    View Slide

  49. @sigmavirus24
    T h e
    r e a l i t y

    View Slide

  50. @sigmavirus24
    Two part
    solution

    View Slide

  51. @sigmavirus24
    Two part
    solution

    View Slide

  52. @sigmavirus24
    Mocking
    mock, requests-mock,
    responses, httpretty

    View Slide

  53. @sigmavirus24
    Integration
    betamax, vcr.py

    View Slide

  54. @sigmavirus24
    T h a n k
    y o u

    View Slide