$30 off During Our Annual Pro Sale. View Details »

Edwin Jung - Mocking and Patching Pitfalls

Edwin Jung - Mocking and Patching Pitfalls

Mocking and patching are powerful techniques for testing, but they can be easily abused, with negative effects on code quality, maintenance, and application architecture. These pain-points can be hard to verbalize, and consequently hard to address. If your unit tests are a PITA, but you cannot explain why, this talk may be for you.

Mocking as a technique has deep roots within OOD and TDD, going back 20+ years, but many Python developers know mocks and patches merely as a technique to isolate code under test. In the absence of knowledge around OOD and TDD, best practices around mocking are completely unknown, misunderstood, or ignored. Developers who use mocks and patches without doing TDD or OOD are susceptible to falling into many well-understood and documented traps.

This talk will draw a historical connection between the way mocks are taught today, and their origins in TDD, OOD, and Java. It will also demonstrate some pitfalls, and provide some guidance and alternatives to mocking and patching (e.g., dependency injection, test doubles, functional style).

https://us.pycon.org/2019/schedule/presentation/216/

PyCon 2019

May 03, 2019
Tweet

More Decks by PyCon 2019

Other Decks in Programming

Transcript

  1. Mocking and
    Patching Pitfalls
    PyCon 2019

    View Slide

  2. Mocking and Patching Pitfalls
    Misadventures In Mocking
    A People’s History of Mocking
    Mocking: Forms and Perversions
    Mocking: A Polemic
    Mock Hell
    PyCon 2019

    View Slide

  3. MOCK HELL
    PyCon 2019

    View Slide

  4. who i am
    • Staff Software Engineer,
    Platform @ Quid

    • Data engineering and
    microservices

    • Python is ~4th language
    email: [email protected]

    twitter: @ejjcatx

    github: xtaje

    View Slide

  5. who you are
    • Have used mocking or patching before.

    • Use the mock and patch APIs regularly, but
    are not sure why it’s sometimes painful.

    • Comfortable with mocks, but want to know
    more about different options.
    Advanced
    Beginner

    View Slide

  6. what you might learn
    • What are the styles of unit testing

    • Where did mocks come from?

    • What are alternatives to mocking?

    • Anti-patterns

    • Relationship between mocking and design
    Advanced
    Beginner

    View Slide

  7. 1. complex patch targets

    2. numerous mocks or patches

    3. mocks with brittle assertions

    4. mocks with complex setup

    5. deep mocks/recursive mocks/
    mocks returning mocks

    6. tests that test nothing

    7. using the debugger to reverse-
    engineer mocks

    8. mocks that prevent refactoring

    View Slide

  8. 1. Mock is a really cool library
    used to isolate code for
    testing.
    2. Here’s how to use
    MagicMock, patch, spec,
    autospec, side_effect, etc.
    3. Patching is tricky, use the
    right namespace target.
    the promise

    View Slide

  9. # my_module.py
    from db import db_read
    def total_value(item_id):
    items = db_read(item_id)
    return sum(items)
    code under test

    View Slide

  10. mock patch
    # test_my_module.py
    from unittest.mock import patch
    from my_module import total_value
    def test_total_value():
    with patch('my_module.db_read') as db_read:
    db_read.return_value = [100, 200]
    assert 300 == total_value(1234)
    db_read.assert_called_with(1234)

    View Slide

  11. total
    value
    db
    read

    View Slide

  12. total
    value
    test
    db
    read
    test

    View Slide

  13. total
    value
    test
    db
    read
    patch

    View Slide

  14. total
    value
    test
    mock
    substitute a mock

    View Slide

  15. total
    value
    test
    mock
    run

    View Slide

  16. test
    mock
    assert on the result
    total
    value

    View Slide

  17. test
    mock
    assert on the interaction
    total
    value

    View Slide

  18. real example: Guardian feed

    View Slide

  19. View Slide

  20. Given some topic of interest (e.g.
    “Brexit”),

    Find all relevant articles from a
    particular time period, and

    Feed those article urls to a scraper
    service.

    View Slide

  21. Guardian REST API
    GET

    https://content.guardianapis.com/search?

    from-date=2019-03-01&to-
    date=2019-03-31&page=1&q=brexit&api-
    key=

    View Slide

  22. {
    "response": {
    "status": "ok",
    “total": 1113,
    "startIndex": 1,
    "pageSize": 10,
    "currentPage": 1,
    "pages": 112,
    “results": [
    {
    "id": “world/2019/mar/21/thursday-briefing-mays-brexi...”,
    "type": "article",
    "webPublicationDate": "2019-03-21T06:30:38Z",
    "webTitle": "Thursday briefing: May's Brexit broadside",
    "webUrl": “https://www.theguardian.com/world/2019/m...”,
    },
    . . . and 9 more results
    ]
    }
    }

    View Slide

  23. if __name__ == "__main__":
    logging.basicConfig()
    logger.setLevel(logging.DEBUG)
    Feed("brexit").run()
    main.py

    View Slide

  24. import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------
    feed.py

    View Slide

  25. feed.py
    import redis
    import requests
    class Feed(object):
    def __init__(self,
    query,
    redis_host=None,
    page_size=None,
    date_range=None):
    self._query, self.page_size = query, page_size
    self.date_range = date_range or (None, None)
    self._redis_con = redis.Redis(redis_host or "localhost")
    self._session = requests.Session()
    + ---- < def format_request > ---------------------------------
    + ---- < def run > --------------------------------------------

    View Slide

  26. feed.py
    import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  27. feed.py
    import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > --------------------------------------
    def format_request(self, page):
    def param(key, val):
    return {key: val} if val is not None else {}
    params={"page": page,
    "query": self._query,
    "api-key": _APIKEY }
    params.update(param("page_size", self.page_size))
    # repeat for from-date, to-date
    return Request('GET', _BASE_URL, params=params).prepare()
    + ---- < def run > -------------------------------------------

    View Slide

  28. import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------
    feed.py

    View Slide

  29. import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > --------------------------------------
    + ---- < def format_request > --------------------------------
    def run(self):
    page, page_count = (1, 1)
    while page <= page_count:
    rsp = self._session.send(self.format_request(page))
    rsp.raise_for_status()
    response = rsp.json()['response']
    page_count = response['pages']
    results = response[‘results']
    def push(web_url):
    self._redis_con.rpush("scrape_urls", web_url)
    list(map(push, (r['webUrl'] for r in results)))
    page += 1
    feed.py

    View Slide

  30. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  31. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  32. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  33. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  34. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  35. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    Feed("brexit").run()

    View Slide

  36. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()

    View Slide

  37. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args

    View Slide

  38. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    mock_Redis.assert_called_with("localhost")
    expected_calls = [call('scrape_urls', 'http://article1'), ... ]
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list

    View Slide

  39. View Slide

  40. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    mock_Redis.assert_called_with("localhost")
    expected_calls = [call('scrape_urls', 'http://article1'), ... ]
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list
    How
    many
    mocks?

    A. 3

    B. 4

    C. __

    View Slide

  41. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    mock_Redis.assert_called_with("localhost")
    expected_calls = [call('scrape_urls', 'http://article1'), ... ]
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list
    How
    many
    mocks?

    A. 3

    B. 4

    C. 11

    View Slide

  42. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    mock_Redis.assert_called_with("localhost")
    expected_calls = [call('scrape_urls', 'http://article1'), ... ]
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list

    View Slide

  43. is this a violation of information
    hiding? encapsulation?
    @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)

    View Slide

  44. @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    "response": {
    "pages": 1,
    "results": [
    {"webUrl": "http://article1"},
    {"webUrl": "http://article2"}
    ]
    }
    }
    session_mock.return_value.send.return_value = mock_response
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    mock_Redis.assert_called_with("localhost")
    expected_calls = [call('scrape_urls', 'http://article1'), ... ]
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list

    View Slide

  45. def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    mock_response.json.return_value = {
    session_mock.return_value.send.return_value = mock_response
    request_mock.return_value.prepare.assert_called()
    assert expected_calls == mock_Redis.return_value.rpush.call_args_list
    law of demeter violation?

    View Slide

  46. what if you change
    imports?
    import redis
    import requests
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  47. from redis import Redis
    import requests
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------
    what if you change
    imports?

    View Slide

  48. broken
    @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    mock_response = MagicMock(spec=requests.Response)
    ...

    View Slide

  49. kwargs to args?
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    def format_request(self, page):
    def param(key, val):
    return {key: val} if val is not None else {}
    params={"page": page,
    "query": self._query,
    "api-key": _APIKEY }
    params.update(param("page_size", self.page_size))
    # repeat for from-date, to-date
    return Request('GET', _BASE_URL, params=params).prepare()
    + ---- < def run > --------------------------------------

    View Slide

  50. kwargs to args?
    class Feed(object):
    + ---- < def __init__ > ---------------------------------
    def format_request(self, page):
    def param(key, val):
    return {key: val} if val is not None else {}
    params={"page": page,
    "query": self._query,
    "api-key": _APIKEY }
    params.update(param("page_size", self.page_size))
    # repeat for from-date, to-date
    return Request('GET', _BASE_URL, params).prepare()
    + ---- < def run > --------------------------------------

    View Slide

  51. broken
    @patch("feed.redis.Redis", autospec=True)
    @patch("feed.requests.Session", autospec=True)
    @patch("feed.Request", autospec=True)
    def test_run(request_mock, session_mock, mock_Redis):
    # more code above ...
    Feed("brexit").run()
    request_mock.return_value.prepare.assert_called()
    params = {"page": 1, "api-key": ‘', 'query': 'brexit'}
    expected_call = mock.call("GET", _GUARDIAN_URL, params=params)
    assert expected_call == request_mock.call_args
    # more code below...

    View Slide

  52. 1. Mock is a really cool library
    used to isolate code for
    testing.
    2. Here’s how to use Mock,
    MagicMock, patch, spec,
    autospec, side_effect, etc.
    3. Patching is tricky, use the
    right namespace target.
    4. (And by the way, be careful
    about over-mocking.)
    5. (Mocks aren’t stubs)
    fine print

    View Slide

  53. • PyCon Cleveland 2018

    • EuroPython 2018

    • PyCon UK 2018

    • PyCon Australia 2017

    • PyCon CZ 2017

    • PyCon Israel 2017

    • EuroPython 2016

    • PyCon Portland 2016

    • PyCon UK 2015

    • PyCon SG 2015

    mock talk at PyCons

    View Slide

  54. PyCon Cleveland 2018: Demystifying the Patch Function

    mock talk at PyCons

    View Slide

  55. A: … Patches can definitely get out of
    hand …

    … you maybe need to look into why
    there is so much heavy mocking.
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  56. Q: Hello… that was a really
    good talk…
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  57. Q: I’m Michael Foord.
    I was the original author of mock…
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  58. Q: “One thing I would say is
    that…”
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  59. Q: “Patches are a sign of
    failure.”
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  60. Q: “The more you have
    to patch, the worse your
    code is.”
    “Q&A: Demystifying the Patch Function”
    -PyCon 2018

    View Slide

  61. ? !

    View Slide

  62. tests
    code
    reflect
    determines
    symbiosis

    View Slide

  63. where did mocks come
    from?

    View Slide

  64. View Slide

  65. Mock Objects is an
    extension to Test Driven
    Development that
    supports good Object-
    Oriented design …

    View Slide

  66. It turns out to be less
    interesting as a
    technique for isolating
    tests … than is widely
    thought.

    View Slide

  67. 1. TDD

    2. Object Oriented Design

    3. mocks are not a tool
    for isolation
    mocks in context

    View Slide

  68. 1. rapid cycles of small
    refactoring

    2. Object Oriented Design

    3. mocks are not a tool
    for isolation
    mocks in context

    View Slide

  69. 1. rapid, cycles of small
    refactoring

    2. object collaborations

    3. mocks are not a tool
    for isolation
    mocks in context

    View Slide

  70. 1. rapid, cycles of small
    refactoring

    2. object collaborations

    3. mocks are a tool for
    exploratory design and
    discovery
    mocks in context

    View Slide

  71. 1. rapid, cycles of small
    refactoring

    2. object collaborations

    3. mocks are a tool for
    exploratory design and
    discovery
    mocks in context

    View Slide

  72. contrary opinions

    View Slide

  73. Always choose interaction
    testing [mocking]… as the
    last option.

    This is very important… Not
    everyone agrees with this
    point of view ...

    View Slide

  74. Freeman and Pryce advocate
    what many call “the London
    school of TDD,” …

    View Slide

  75. I’m not fully disagreeing …
    but for maintainability …
    tests using mocks creates
    more trouble than it’s worth.

    View Slide

  76. Mocking frameworks are
    potent tools that let you
    wield incredible power.

    However, power nearly
    always corrupts.


    View Slide

  77. how to test without mocks?

    View Slide

  78. mocks aren’t stubs
    1. mocks are not stubs

    2. mocks != stubs

    3. stubs are not mocks

    View Slide

  79. test doubles
    name behavior
    mock
    stub
    fake
    dummy
    spy

    View Slide

  80. test doubles
    name behavior
    mock records calls to the object
    stub returns canned data, no logic
    fake implements fake version of production logic
    dummy does nothing
    spy records and delegates to the real thing

    View Slide

  81. # test_my_module.py
    from unittest.mock import patch
    from my_module import total_value
    def test_total_value():
    with patch('my_module.db_read') as db_read:
    db_read.return_value = [100, 200]
    assert 300 == total_value(1234)
    db_read.assert_called_with(1234)
    remember mock patch?
    mock
    stub

    View Slide

  82. alternative: fake patch
    # test_my_module.py
    from unittest.mock import patch
    from my_module import total_value
    def test_total_value():
    with patch('my_module.db_read'):
    assert 300 == total_value(1234)

    View Slide

  83. alternative: fake patch
    # test_my_module.py
    from unittest.mock import patch
    from my_module import total_value
    def test_total_value():
    def fake_db_read(item_id):
    with open(f'./fixtures/{item_id}.json') as fobj:
    return json.load(fobj)
    with patch('my_module.db_read'):
    assert 300 == total_value(1234)

    View Slide

  84. alternative: fake patch
    # test_my_module.py
    from unittest.mock import patch
    from my_module import total_value
    def test_total_value():
    def fake_db_read(item_id):
    with open(f'./fixtures/{item_id}.json') as fobj:
    return json.load(fobj)
    with patch('my_module.db_read', new=fake_db_read):
    assert 300 == total_value(1234)

    View Slide

  85. patch
    total
    value
    test
    db
    read

    View Slide

  86. define a fake
    total
    value
    test
    db
    read
    fake
    db_read

    View Slide

  87. substitute the fake
    total
    value
    test
    db_rea
    d
    fake
    db_read

    View Slide

  88. run
    total
    value
    test
    db_rea
    d
    fake
    db_read

    View Slide

  89. total
    value
    test
    db_rea
    d
    assert on result
    fake
    db_read

    View Slide

  90. alternative: fake injection
    # my_module.py
    from db import db_read
    def total_value(item_id):
    items = db_read(item_id)
    return sum(items)
    # my_module.py
    def total_value(item_id, db_read):
    items = db_read(item_id)
    return sum(items)

    View Slide

  91. alternative: fake injection
    def test_total_value2():
    def fake_db_read(item_id):
    with open(f'./fixtures/{item_id}.json') as fobj:
    return json.load(fobj)
    assert 300 == total_value2(1234, fake_db_read)

    View Slide

  92. total_value2 expects to be
    given a collaborator to fill a role
    total
    value

    View Slide

  93. test => define a fake
    fake
    total
    value
    test

    View Slide

  94. test => inject the fake
    fake
    total
    value
    test

    View Slide

  95. run
    fake
    total
    value
    test

    View Slide

  96. fake
    total
    value
    test
    assert on result

    View Slide

  97. app => create a real DB
    adapter
    db
    read
    total
    value
    app

    View Slide

  98. app => inject the adapter
    db
    read
    total
    value
    app

    View Slide

  99. db
    read
    total
    value
    app => run
    app

    View Slide

  100. tactical questions
    1.Which Test Doubles (mock or other)?

    2.Mockist or Classical? (aka London or Detroit)

    3.Patch or Inject?

    View Slide

  101. plan: Guardian feed

    View Slide

  102. Mock Roles, not Objects
    Tactics 3

    Dependency Injection
    Tactics 4

    Inject the Collaborator
    Tactics 5

    Go Functional
    Tactics 1

    Find a Seam

    Patch a Fake

    View Slide

  103. what are the roles in this
    collaboration?

    View Slide

  104. Master Student

    View Slide

  105. Master Student

    View Slide

  106. Villain
    Hero

    View Slide

  107. Hero Villain

    View Slide

  108. Father Son

    View Slide

  109. Feed
    test
    Session
    Request
    redispy
    redis
    server
    guardian
    what are the roles in this
    collaboration?

    View Slide

  110. Feed
    test
    Session
    Request
    redispy
    redis
    server
    guardian
    Parser
    Sink
    Source

    View Slide

  111. Mock Roles, not Objects
    Tactics 2

    Dependency Injection
    Tactics 3

    Inject the Collaborator
    Tactics 4

    Go Functional
    Tactics 1

    Find a Seam

    Patch a Fake

    View Slide

  112. Feed
    test
    guardian
    Session
    Request
    redispy
    redis
    server

    View Slide

  113. Feed
    test
    Session
    Request
    redispy
    redis
    server
    guardian

    View Slide

  114. seam
    “A seam is a place where
    you can alter behavior in
    your program without
    editing in that place.”

    View Slide

  115. Feed
    test
    Session
    Request
    redispy
    redis
    server
    guardian

    View Slide

  116. Feed
    test
    Session
    Request
    redispy
    redis
    server
    find a different “seam”
    urllib3
    requests
    internals
    TCP
    guardian

    View Slide

  117. Feed
    test
    Session
    Request
    redispy
    redis
    server
    patch a fake into a transport layer “seam”
    HTTMock
    guardian
    fakeguard
    ian

    View Slide

  118. @httmock.urlmatch(netloc=r’content.guardianapis.com')
    def fake_guardian(url, request):
    page, page_size = parse_query_params(url.query)
    article_ids = range(page_size*(page-1)+1, page_size*page+1)
    results = [{"webUrl": "http://article%d" % (i)}
    for i in article_ids]
    fake_response = {
    "response": {
    "currentPage": page,
    "pages": 2,
    "results": results
    }
    }
    return json.dumps(fake_response)

    View Slide

  119. from urllib.parse import parse_qs
    from httmock import urlmatch, HTTMock
    @mock.patch("feed.redis.Redis", autospec=True)
    def test_run(mock_Redis):
    with httmock.HTTMock(fake_guardian):
    Feed("brexit").run()
    mock_Redis.assert_called_with("localhost")
    expected_calls = [mock.call('scrape_urls', f'http://a{i}')
    for i in range(1, 21)]
    assert expected_calls ==
    mock_Redis.return_value.rpush.call_args_list)

    View Slide

  120. Mock Roles, not Objects
    Tactics 2

    Dependency Injection
    Tactics 3

    Inject the Collaborator
    Tactics 4

    Go Functional
    Tactics 1

    Find a Seam

    Patch a Fake

    View Slide

  121. Feed
    test
    Session
    Request
    mock
    redispy
    redis
    server
    fake
    guardian
    HTTMock

    View Slide

  122. Feed
    test
    Session
    Request
    redis
    server
    fake
    guardian
    HTTMock

    View Slide

  123. Feed
    test
    Session
    Request
    redis
    server
    fake
    redis
    fake
    guardian
    HTTMock

    View Slide

  124. Feed
    test
    Session
    Request
    fake
    redis
    redis
    server
    fake
    guardian
    HTTMock

    View Slide

  125. inject the redis connection
    class Feed(object):
    def __init__(self, query,
    redis_host=None,
    page_size=None,
    date_range=None):
    self._query = query
    self.page_size = page_size
    self.date_range = date_range or (None, None)
    self._redis_con = redis.Redis(redis_host or "localhost")
    self._session = requests.Session()
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  126. inject the redis connection
    class Feed(object):
    def __init__(self, query,
    redis_con,
    page_size=None,
    date_range=None):
    self._query = query
    self.page_size = page_size
    self.date_range = date_range or (None, None)
    self._redis_con = redis_con
    self._session = requests.Session()
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  127. from httmock import urlmatch, HTTMock
    from feed import Feed
    import fakeredis
    def test_run():
    fake_con = fakeredis.FakeStrictRedis()
    with HTTMock(fake_guardian):
    Feed("brexit", fake_con).run()
    expected = [f"http://article{i}" for i in range(1, 21)]
    received = [fake_con.lpop("scrape_urls").decode("utf-8")
    for _ in range(1,21)]
    self.assertEqual(expected, received)

    View Slide

  128. Mock Roles, not Objects
    Tactics 3

    Dependency Injection
    Tactics 4

    Inject the Collaborator
    Tactics 5

    Go Functional
    Tactics 1

    Find a Seam

    Patch a Fake

    View Slide

  129. inject the collaborator
    class Feed(object):
    def __init__(self, query,
    redis_con,
    page_size=None,
    date_range=None):
    self._query = query
    self.page_size = page_size
    self.date_range = date_range or (None, None)
    self._redis_con = redis_con
    self._session = requests.Session()
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  130. inject the collaborator
    class Feed(object):
    def __init__(self, query,
    sink,
    page_size=None,
    date_range=None):
    self._query = query
    self.page_size = page_size
    self.date_range = date_range or (None, None)
    self._sink = sink
    self._session = requests.Session()
    + ---- < def format_request > ---------------------------
    + ---- < def run > --------------------------------------

    View Slide

  131. class Feed(object):
    + ---- < def __init__ > ---------------------------------
    + ---- < def format_request > ---------------------------
    def run(self):
    page, page_count = (1, 1)
    while page <= page_count:
    rsp = self._session.send(self.format_request(page))
    rsp.raise_for_status()
    response = rsp.json()['response']
    logger.debug(response)
    page_count = response['pages']
    results = response['results']
    list(map(self._sink, (r['webUrl'] for r in results)))
    page += 1

    View Slide

  132. from httmock import urlmatch, all_requests, HTTMock
    from feed import Feed
    def test_run(self):
    received = []
    with HTTMock(fake_guardian):
    Feed("brexit", received.append).run()
    expected = [f"http://article{i}" for i in range(1, 21)]
    self.assertEqual(expected, received)

    View Slide

  133. if __name__ == "__main__":
    logging.basicConfig()
    logger.setLevel(logging.DEBUG)
    redis_con = redis.Redis("localhost")
    def sink(msg):
    redis_con.rpush("scrape_urls", msg)
    Feed("brexit", sink).run()

    View Slide

  134. Mock Roles, not Objects
    Tactics 3

    Dependency Injection
    Tactics 4

    Inject the Collaborator
    Tactics 5

    Go Functional
    Tactics 1

    Find a Seam

    Patch a Fake

    View Slide

  135. glue code
    def feed(query):
    s = requests.Session()
    first_page = s.send(initial_request(query))
    all_pages = map(s.send,
    page_requests(first_page.json(), query=query))
    web_urls = munge(all_pages)
    redis.Redis("localhost").rpush("scrape_urls", *web_urls)
    if __name__ == "__main__":
    feed("brexit")

    View Slide

  136. pure functions
    def flatmap(f, items):
    return itertools.chain.from_iterable(map(f, items))
    def param(key, val):
    return {key: val} if val is not None else {}
    def format_request(query, page, page_size = None, date_range=None):
    date_range = date_range or (None, None)
    params= { "page": page, "query": query, "api-key": _APIKEY }
    params.update(param("page_size", page_size))
    params.update(param("start_date", date_range[0]))
    params.update(param("end_date", date_range[1]))
    return Request('GET', _BASE_URL, params=params).prepare()
    def initial_request(query, page_size=None, date_range=None):
    return format_request(query, page=1, page_size=page_size, date_range=date_range)
    def page_requests(body, query):
    response = body['response']
    page_count = min(3, int(response['pages'])) # IRL needs flow control
    for page in range(1, page_count+1):
    params = {"page": page, "query": query, "api-key": _APIKEY}
    yield Request('GET', _BASE_URL, params=params).prepare()
    def munge(all_pages):
    bodies = map(lambda r: r.json(), all_pages)
    results = flatmap(lambda body: body['response']['results'], bodies)
    return map(lambda result: result['webUrl'], results)

    View Slide

  137. anti-patterns

    View Slide

  138. bootleg TDD

    View Slide

  139. bootleg toys

    View Slide

  140. bootleg toys

    View Slide

  141. • Aim for 100% test isolation, 100% coverage

    • Test Whenever, No Refactoring

    • Procedural Decomposition or Top-Down Design (no OOP)
    bootleg TDD

    View Slide

  142. • Aim for 100% test isolation, 100% coverage

    • Test Whenever, No Refactoring

    • Procedural Decomposition or Top-Down Design (no OOP)
    bootleg TDD

    View Slide

  143. procedural decomposition
    “In each step, one or several instructions of a
    program are decomposed into more detailed
    instructions”

    View Slide

  144. P

    View Slide

  145. P
    B
    A

    View Slide

  146. P
    B
    A
    C
    D
    E

    View Slide

  147. P
    B
    A
    C
    D
    E
    Done. Time to test.

    View Slide

  148. P
    B
    A
    test

    View Slide

  149. B
    A
    C
    D
    E
    test
    test

    View Slide

  150. C
    D
    E
    test
    test
    test

    View Slide

  151. C
    D
    E
    test
    test
    test
    “How do patch a database?”

    View Slide

  152. C
    D
    E
    test
    test
    test
    “. . . mock the db connection?”
    “. . . mock the query language?”

    View Slide

  153. inversion-lock

    View Slide

  154. Abstract
    Detailed
    1.Top down design => no dependency inversion
    2.Patching + mocking “locks in” design decisions
    Patching prevents understanding of dependency inversion
    Issues?

    View Slide

  155. dependency inversion
    total
    value
    client

    View Slide

  156. fake
    total
    value
    test
    test => inject a fake

    View Slide

  157. db
    read
    total
    value
    app => inject a real DB
    app

    View Slide

  158. total
    value
    test
    db
    read
    app
    fake
    clean

    View Slide

  159. total
    value
    test
    db
    read
    app
    fake
    ports-and-adapters

    View Slide

  160. total
    value
    test
    hexagonal
    db
    read
    app
    fake

    View Slide

  161. connections to “good”
    design?
    • Functional-Core/Imperative Shell

    • Ports-And-Adapters, Hexagonal

    • Clean, Onion

    View Slide

  162. the blob

    View Slide

  163. P

    View Slide

  164. P
    How do I test this without the database?
    test

    View Slide

  165. P
    Extreme Tactical Patching
    test

    View Slide

  166. issues
    • What kind of test is this?

    • Is this

    • pragmatic or

    • just enabling bad habits?

    View Slide

  167. patches as crosscuts

    View Slide

  168. propositions
    1.A test should test a single behavior.

    2.A patch is a violation of some encapsulation boundary.

    3.A test with many patches is violating multiple
    encapsulation boundaries.
    A test case with a lot of patches may represent a problem with
    cohesion or coupling.

    A test case with a lot of patches may represent a crosscutting
    concern.

    View Slide

  169. opinions
    • always be refactoring

    • consider other test doubles

    • patching

    • should be rare, and the last tool you use

    • mocks (if you use them)

    • should target roles and not objects

    • are not for just for test isolation

    View Slide

  170. thanks to

    View Slide

  171. Debates and Counter-opinions

    • Dependency Injection is Not a Virtue (DHH)

    • TDD is Dead (Martin Fowler, Kent Beck, DHH)

    • Why Most Unit Testing is a Waste (Coplien)

    • The TDD Apostate (Seeman)

    Resources

    • Mocks aren’t Stubs (Fowler)

    • Clean Code (Uncle Bob)

    • Working Effectively with Legacy Code (Feathers)

    • Growing Object Oriented Software Guided by Tests

    • The Art of Unit Testing (Osherove)
    email: [email protected]
    twitter: @ejjcatx

    View Slide