Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cutting off the Internet: Testing Applications ...

Cutting off the Internet: Testing Applications That Use Requests

requests is a very popular library for making HTTP requests in Python. Even though so many people use it, there's no thorough and exhaustive way to test applications that take advantage of it.

These slides are best read while listening to the presentation given at PyTennessee on 7 February 2015

Ian Cordasco

February 07, 2015
Tweet

More Decks by Ian Cordasco

Other Decks in Programming

Transcript

  1. 4 • github3.py • twine • urllib3 • sqlobject • betamax @sigmavirus24 (Ian the RFC

    Reader) www.rackspace.com • requests • Flake8 • chardet • requests-toolbelt • uritemplate.py
  2. class Bar(object): def __init__(self): self.foo = Foo() def baz(self, *args):

    a = self.foo.fiz() # do stuff with a A Collaborators Example… 14 www.rackspace.com
  3. url = url_from(resource) kw = {‘headers’: headers, ‘params’: params} resp

    = session.get(url, **kw) if resp.ok: return resp resp.raise_for_status() What We’re Testing 19 www.rackspace.com
  4. { "id": 3710711, "name": "github3.py", "full_name": "sigmavirus24/github3.py", "owner": { "login":

    "sigmavirus24", "id": 240830, "avatar_url": "https://avatars.githubusercontent.com/u/240830?v=3", "gravatar_id": "", "url": "https://api.github.com/users/sigmavirus24", "html_url": "https://github.com/sigmavirus24", "followers_url": "https://api.github.com/users/sigmavirus24/followers", "following_url": "https://api.github.com/users/sigmavirus24/following{/other_user}", "gists_url": "https://api.github.com/users/sigmavirus24/gists{/ gist_id}", "starred_url": "https://api.github.com/users/sigmavirus24/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/ users/sigmavirus24/subscriptions", "organizations_url": "https://api.github.com/users/sigmavirus24/orgs", "repos_url": "https:// api.github.com/users/sigmavirus24/repos", "events_url": "https://api.github.com/users/sigmavirus24/events{/privacy}", "received_events_url": "https://api.github.com/users/sigmavirus24/received_events", "type": "User", "site_admin": false }, "private": false, "html_url": "https://github.com/sigmavirus24/github3.py", "description": "Python library for interfacing with the GitHub APIv3", "fork": false, "url": "https://api.github.com/repos/sigmavirus24/github3.py", "forks_url": "https://api.github.com/repos/sigmavirus24/ github3.py/forks", "keys_url": "https://api.github.com/repos/sigmavirus24/github3.py/keys{/key_id}", "collaborators_url": "https:// api.github.com/repos/sigmavirus24/github3.py/collaborators{/collaborator}", "teams_url": "https://api.github.com/repos/sigmavirus24/ github3.py/teams", "hooks_url": "https://api.github.com/repos/sigmavirus24/github3.py/hooks", "issue_events_url": "https://api.github.com/ repos/sigmavirus24/github3.py/issues/events{/number}", "events_url": "https://api.github.com/repos/sigmavirus24/github3.py/events", "assignees_url": "https://api.github.com/repos/sigmavirus24/github3.py/assignees{/user}", "branches_url": "https://api.github.com/repos/ sigmavirus24/github3.py/branches{/branch}", "tags_url": "https://api.github.com/repos/sigmavirus24/github3.py/tags", "blobs_url": "https:// api.github.com/repos/sigmavirus24/github3.py/git/blobs{/sha}", "git_tags_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/ tags{/sha}", "git_refs_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/refs{/sha}", "trees_url": "https://api.github.com/ repos/sigmavirus24/github3.py/git/trees{/sha}", "statuses_url": "https://api.github.com/repos/sigmavirus24/github3.py/statuses/{sha}", "languages_url": "https://api.github.com/repos/sigmavirus24/github3.py/languages", "stargazers_url": "https://api.github.com/repos/ sigmavirus24/github3.py/stargazers", "contributors_url": "https://api.github.com/repos/sigmavirus24/github3.py/contributors", "subscribers_url": "https://api.github.com/repos/sigmavirus24/github3.py/subscribers", "subscription_url": "https://api.github.com/repos/ sigmavirus24/github3.py/subscription", "commits_url": "https://api.github.com/repos/sigmavirus24/github3.py/commits{/sha}", "git_commits_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/commits{/sha}", "comments_url": "https://api.github.com/repos/ sigmavirus24/github3.py/comments{/number}", "issue_comment_url": "https://api.github.com/repos/sigmavirus24/github3.py/issues/comments/ {number}", "contents_url": "https://api.github.com/repos/sigmavirus24/github3.py/contents/{+path}", "compare_url": "https://api.github.com/ repos/sigmavirus24/github3.py/compare/{base}...{head}", "merges_url": "https://api.github.com/repos/sigmavirus24/github3.py/merges", "archive_url": "https://api.github.com/repos/sigmavirus24/github3.py/{archive_format}{/ref}", "downloads_url": "https://api.github.com/ repos/sigmavirus24/github3.py/downloads", "issues_url": "https://api.github.com/repos/sigmavirus24/github3.py/issues{/number}", "pulls_url": "https://api.github.com/repos/sigmavirus24/github3.py/pulls{/number}", "milestones_url": "https://api.github.com/repos/ sigmavirus24/github3.py/milestones{/number}", "notifications_url": "https://api.github.com/repos/sigmavirus24/github3.py/notifications{? since,all,participating}", "labels_url": "https://api.github.com/repos/sigmavirus24/github3.py/labels{/name}", "releases_url": "https:// api.github.com/repos/sigmavirus24/github3.py/releases{/id}", "created_at": "2012-03-13T19:58:53Z", "updated_at": "2015-01-15T08:07:34Z", "pushed_at": "2015-01-08T04:36:37Z", "git_url": "git://github.com/sigmavirus24/github3.py.git", "ssh_url": "[email protected]:sigmavirus24/ github3.py.git", "clone_url": "https://github.com/sigmavirus24/github3.py.git", "svn_url": "https://github.com/sigmavirus24/github3.py", "homepage": "http://github3py.readthedocs.org/", "size": 8669, "stargazers_count": 268, "watchers_count": 268, "language": "Python", "has_issues": true, "has_downloads": true, "has_wiki": false, "has_pages": true, "forks_count": 91, "mirror_url": null, "open_issues_count": 22, "forks": 91, "open_issues": 22, "watchers": 268, "default_branch": "develop", "network_count": 91, "subscribers_count": 24 } Gigantic Resources 37 www.rackspace.com
  5. { "id": 3710711, "name": "github3.py", "full_name": "sigmavirus24/github3.py", "owner": { "login":

    "sigmavirus24", "id": 240830, "avatar_url": "https://avatars.githubusercontent.com/u/240830?v=3", "gravatar_id": "", "url": "https://api.github.com/users/sigmavirus24", "html_url": "https://github.com/sigmavirus24", "followers_url": "https://api.github.com/users/sigmavirus24/followers", "following_url": "https://api.github.com/users/sigmavirus24/following{/other_user}", "gists_url": "https://api.github.com/users/sigmavirus24/gists{/ gist_id}", "starred_url": "https://api.github.com/users/sigmavirus24/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/ users/sigmavirus24/subscriptions", "organizations_url": "https://api.github.com/users/sigmavirus24/orgs", "repos_url": "https:// api.github.com/users/sigmavirus24/repos", "events_url": "https://api.github.com/users/sigmavirus24/events{/privacy}", "received_events_url": "https://api.github.com/users/sigmavirus24/received_events", "type": "User", "site_admin": false }, "private": false, "html_url": "https://github.com/sigmavirus24/github3.py", "description": "Python library for interfacing with the GitHub APIv3", "fork": false, "url": "https://api.github.com/repos/sigmavirus24/github3.py", "forks_url": "https://api.github.com/repos/sigmavirus24/ github3.py/forks", "keys_url": "https://api.github.com/repos/sigmavirus24/github3.py/keys{/key_id}", "collaborators_url": "https:// api.github.com/repos/sigmavirus24/github3.py/collaborators{/collaborator}", "teams_url": "https://api.github.com/repos/sigmavirus24/ github3.py/teams", "hooks_url": "https://api.github.com/repos/sigmavirus24/github3.py/hooks", "issue_events_url": "https://api.github.com/ repos/sigmavirus24/github3.py/issues/events{/number}", "events_url": "https://api.github.com/repos/sigmavirus24/github3.py/events", "assignees_url": "https://api.github.com/repos/sigmavirus24/github3.py/assignees{/user}", "branches_url": "https://api.github.com/repos/ sigmavirus24/github3.py/branches{/branch}", "tags_url": "https://api.github.com/repos/sigmavirus24/github3.py/tags", "blobs_url": "https:// api.github.com/repos/sigmavirus24/github3.py/git/blobs{/sha}", "git_tags_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/ tags{/sha}", "git_refs_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/refs{/sha}", "trees_url": "https://api.github.com/ repos/sigmavirus24/github3.py/git/trees{/sha}", "statuses_url": "https://api.github.com/repos/sigmavirus24/github3.py/statuses/{sha}", "languages_url": "https://api.github.com/repos/sigmavirus24/github3.py/languages", "stargazers_url": "https://api.github.com/repos/ sigmavirus24/github3.py/stargazers", "contributors_url": "https://api.github.com/repos/sigmavirus24/github3.py/contributors", "subscribers_url": "https://api.github.com/repos/sigmavirus24/github3.py/subscribers", "subscription_url": "https://api.github.com/repos/ sigmavirus24/github3.py/subscription", "commits_url": "https://api.github.com/repos/sigmavirus24/github3.py/commits{/sha}", "git_commits_url": "https://api.github.com/repos/sigmavirus24/github3.py/git/commits{/sha}", "comments_url": "https://api.github.com/repos/ sigmavirus24/github3.py/comments{/number}", "issue_comment_url": "https://api.github.com/repos/sigmavirus24/github3.py/issues/comments/ {number}", "contents_url": "https://api.github.com/repos/sigmavirus24/github3.py/contents/{+path}", "compare_url": "https://api.github.com/ repos/sigmavirus24/github3.py/compare/{base}...{head}", "merges_url": "https://api.github.com/repos/sigmavirus24/github3.py/merges", "archive_url": "https://api.github.com/repos/sigmavirus24/github3.py/{archive_format}{/ref}", "downloads_url": "https://api.github.com/ repos/sigmavirus24/github3.py/downloads", "issues_url": "https://api.github.com/repos/sigmavirus24/github3.py/issues{/number}", "pulls_url": "https://api.github.com/repos/sigmavirus24/github3.py/pulls{/number}", "milestones_url": "https://api.github.com/repos/ sigmavirus24/github3.py/milestones{/number}", "notifications_url": "https://api.github.com/repos/sigmavirus24/github3.py/notifications{? since,all,participating}", "labels_url": "https://api.github.com/repos/sigmavirus24/github3.py/labels{/name}", "releases_url": "https:// api.github.com/repos/sigmavirus24/github3.py/releases{/id}", "created_at": "2012-03-13T19:58:53Z", "updated_at": "2015-01-15T08:07:34Z", "pushed_at": "2015-01-08T04:36:37Z", "git_url": "git://github.com/sigmavirus24/github3.py.git", "ssh_url": "[email protected]:sigmavirus24/ github3.py.git", "clone_url": "https://github.com/sigmavirus24/github3.py.git", "svn_url": "https://github.com/sigmavirus24/github3.py", "homepage": "http://github3py.readthedocs.org/", "size": 8669, "stargazers_count": 268, "watchers_count": 268, "language": "Python", "has_issues": true, "has_downloads": true, "has_wiki": false, "has_pages": true, "forks_count": 91, "mirror_url": null, "open_issues_count": 22, "forks": 91, "open_issues": 22, "watchers": 268, "default_branch": "develop", "network_count": 91, "subscribers_count": 24 } Gigantic Resources 38 www.rackspace.com
  6. session = requests.Session() response = get_resource(session, ‘/users’) assert response.ok Live

    Integration 45 www.rackspace.com Now write and run 500 more of these
  7. session = requests.Session() response = get_resource(session, ‘/users’) assert response.ok Live

    Integration 46 www.rackspace.com Now write and run 500 more of these
  8. session = requests.Session() with vcr.use_cassette(‘vcrexample.yml’): resp = my_module.get_resource( session, ‘/users’)

    assert response.ok users = resp.json() assert len(users) > 0 Example with vcr.py 51 www.rackspace.com
  9. session = requests.Session() with vcr.use_cassette(‘vcrexample.yml’): resp = my_module.get_resource( session, ‘/users’)

    assert response.ok users = resp.json() assert len(users) > 0 Example with vcr.py 52 www.rackspace.com
  10. session = requests.Session() with vcr.use_cassette(‘vcrexample.yml’): resp = my_module.get_resource( session, ‘/users’)

    assert response.ok users = resp.json() assert len(users) > 0 Example with vcr.py 53 www.rackspace.com
  11. session = requests.Session() recorder = betamax.Betamax( session, cassette_library_dir=‘.’) with recorder.use_cassette(‘betamaex’):

    resp = my_module.get_resource( session, ‘/users’) assert response.ok users = resp.json() assert len(users) > 0 Example with betamax 54 www.rackspace.com
  12. Example with betamax 55 www.rackspace.com session = requests.Session() recorder =

    betamax.Betamax( session, cassette_library_dir=‘.’) with recorder.use_cassette(‘betamaex’): resp = my_module.get_resource( session, ‘/users’) assert response.ok users = resp.json() assert len(users) > 0
  13. Example with betamax 56 www.rackspace.com session = requests.Session() recorder =

    betamax.Betamax( session, cassette_library_dir=‘.’) with recorder.use_cassette(‘betamaex’): resp = my_module.get_resource( session, ‘/users’) assert response.ok users = resp.json() assert len(users) > 0
  14. Example with betamax 57 www.rackspace.com session = requests.Session() recorder =

    betamax.Betamax( session, cassette_library_dir=‘.’) with recorder.use_cassette(‘betamaex’): resp = my_module.get_resource( session, ‘/users’) assert response.ok users = resp.json() assert len(users) > 0
  15. THANK YOU RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST

    | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM © RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM