PyCon Siberia 2016. Не доверяйте тестам!

PyCon Siberia 2016. Не доверяйте тестам!

Каждый программист рано или поздно начинает писать тесты на свой код. В какой-то момент он начинает задумываться о том, насколько его тесты хороши. В своем докладе я расскажу о том, какие инструменты для проверки качества тестов существуют, как они работают и почему они обманывают нас.

15563b4bb24076f1801cd862f74ed3fe?s=128

Ivan Tsyganov

October 03, 2016
Tweet

Transcript

  1. 2.

    Обо мне ✤ Спикер PyCon Russia 2016, PiterPy#2 и PiterPy#3

    ✤ Люблю OpenSource ✤ Не умею frontend
  2. 3.

    ✤ 15 лет практического опыта на рынке ИБ ✤ Более

    650 сотрудников в 9 странах ✤ Каждый год находим более 200 уязвимостей нулевого дня ✤ Проводим более 200 аудитов безопасности в крупнейших компаниях мира ежегодно
  3. 4.
  4. 5.

    MaxPatrol ✤ Тестирование на проникновение (Pentest) ✤ Системные проверки (Audit)

    ✤ Соответствие стандартам (Compliance) ✤ Одна из крупнейших баз знаний в мире Система контроля защищенности и соответствия стандартам.
  5. 6.

    ✤ Тестирование на проникновение (Pentest) ✤ Системные проверки (Audit) ✤

    Соответствие стандартам (Compliance) ✤ Одна из крупнейших баз знаний в мире Система контроля защищенности и соответствия стандартам. ✤ Системные проверки (Audit) MaxPatrol
  6. 8.

    Зачем тестировать? ✤ Уверенность, что написанный код работает ✤ Ревью

    кода становится проще ✤ Гарантия, что ничего не сломалось при изменениях
  7. 10.

    Давайте писать тесты! def get_total_price(cart_prices): if len(cart_prices) == 0: return

    result = {'TotalPrice': sum(cart_prices)} if len(cart_prices) >= 2: result['Discount'] = result['TotalPrice'] * 0.25 return result['TotalPrice'] - result.get('Discount')
  8. 11.

    Плохой тест def get_total_price(cart_prices): if len(cart_prices) == 0: return result

    = {'TotalPrice': sum(cart_prices)} if len(cart_prices) >= 2: result['Discount'] = result['TotalPrice'] * 0.25 return result['TotalPrice'] - result.get('Discount') def test_get_total_price(): assert get_total_price([90, 10]) == 75
  9. 12.

    Неожиданные данные >>> balance = 1000 >>> >>> goods =

    [] >>> >>> balance -= get_total_price(goods) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for -=: 'int' and 'NoneType' >>>
  10. 14.
  11. 18.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount') def test_get_total_price(): assert get_total_price([90, 10]) == 75 Name Stmts Miss Cover Missing -------------------------------------------- target.py 7 1 85.71% 2
  12. 19.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount') def test_get_total_price(): assert get_total_price([90, 10]) == 75 Name Stmts Miss Cover Missing -------------------------------------------- target.py 7 1 85.71% 2 2 if len(cart_prices) == 0:
  13. 20.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount') def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Cover Missing -------------------------------------------- target.py 7 0 100.00%
  14. 21.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount') def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Cover Missing -------------------------------------------- target.py 7 0 100.00%
  15. 22.

    >>> get_total_price([90]) 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0:

    3 return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount')
  16. 23.

    >>> get_total_price([90]) Traceback (most recent call last): File "<stdin>", line

    1, in <module> File "<stdin>", line 9, in get_total_price TypeError: unsupported operand type(s) for -: 'int' and 'NoneType' >>> 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get('Discount')
  17. 24.
  18. 25.

    coverage.ini [report]
 show_missing = True
 precision = 2
 [run]
 branch

    = True py.test --cov-config=coverage.ini --cov=target test.py
  19. 26.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 7 0 4 1 90.91% 6 ->9
  20. 27.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 7 0 4 1 90.91% 6 ->9
  21. 28.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 assert get_total_price([90]) == 90 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 7 0 4 0 100.00%
  22. 29.
  23. 30.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 total_price = sum(cart_prices) 6 get_discount = lambda items, price: len(items) >= 2 and price * 0.25 7 8 return total_price-get_discount(cart_prices, total_price) 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0)
  24. 31.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 total_price = sum(cart_prices) 6 get_discount = lambda items, price: len(items) >= 2 and price * 0.25 7 8 return total_price-get_discount(cart_prices, total_price) def test_get_total_price(): assert get_total_price([90, 10]) == 75 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 6 1 4 1 80.00% 3, 2 ->3
  25. 32.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 total_price = sum(cart_prices) 6 get_discount = lambda items, price: len(items) >= 2 and price * 0.25 7 8 return total_price-get_discount(cart_prices, total_price) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 6 0 4 0 100.00%
  26. 33.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 total_price = sum(cart_prices) 6 get_discount = lambda items, price: len(items) >= 2 and price * 0.25 7 8 return total_price-get_discount(cart_prices, total_price) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 6 0 4 0 100.00% 6 get_discount = lambda items, price: len(items) >= 2 and price * 0.25
  27. 34.
  28. 37.

    coverage.parser.PythonParser ✤ Обходит все токены и отмечает «интересные» факты ✤

    Компилирует код. Обходит code-object и сохраняет номера строк
  29. 44.

    Обработка ноды class While(stmt): _fields = ( 'test', 'body', 'orelse',

    ) while i<10: print(i) i += 1 else: print('All done')
  30. 45.

    Выполненные строки sys.settrace(tracefunc) Set the system’s trace function, which allows

    you to implement a Python source code debugger in Python. Trace functions should have three arguments: frame, event, and arg. frame is the current stack frame. event is a string: 'call', 'line', 'return', 'exception', 'c_call', 'c_return', or 'c_exception'. arg depends on the event type.
  31. 46.

    PyTracer «call» event ✤ Сохраняем данные предыдущего контекста ✤ Начинаем

    собирать данные нового контекста ✤ Учитываем особенности генераторов
  32. 50.

    Зачем такие сложности? 1 for i in some_list: 2 if

    i == 'Hello': 3 print(i + ' World!') 4 elif i == 'Skip': 5 continue 6 else: 7 break 8 else: 9 print(r'¯\_(ツ)_/¯')
  33. 53.

    Что может пойти не так? 1 def make_dict(a,b,c): 2 return

    { 3 'a': a, 4 'b': b if a>1 else 0, 5 'c': [ 6 i for i in range(c) if i<(a*10) 7 ] 6 }
  34. 54.
  35. 55.

    Мутационное тестирование ✤ Берем тестируемый код ✤ Мутируем ✤ Тестируем

    мутантов нашими тестами ✤ Тест не упал -> плохой тест
  36. 56.

    Мутационное тестирование ✤ Берем тестируемый код ✤ Мутируем ✤ Тестируем

    мутантов нашими тестами ✤ Если тест не упал -> это плохой тест ✤ Тест не упал -> плохой тест
  37. 58.

    Идея def mul(a, b): return a * b def test_mul():

    assert mul(2, 2) == 4 def mul(a, b): return a ** b
  38. 59.

    Идея def mul(a, b): return a * b def test_mul():

    assert mul(2, 2) == 4 def mul(a, b): return a + b def mul(a, b): return a ** b
  39. 60.

    Идея def mul(a, b): return a * b def test_mul():

    assert mul(2, 2) == 4 assert mul(2, 3) == 6 def mul(a, b): return a + b def mul(a, b): return a ** b
  40. 63.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] / 0.25 8 …
  41. 64.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 9 return result['TotalPrice'] + result.get(‘Discount’, 0) …
  42. 65.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 2 if (not len(cart_prices) == 0): 3 return 0 …
  43. 66.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 2 if len(cart_prices) == 1: 3 return 0 …
  44. 67.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 2 if len(cart_prices) == 0: 3 return 1 …
  45. 68.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 5 result = {'': sum(cart_prices)} …
  46. 69.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) … 9 return result[‘some_key'] - result.get(‘Discount’, 0)
  47. 70.

    Мутации 1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3

    return 0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0)
  48. 71.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(self): self.assertEqual(get_total_price([90, 10]), 75) self.assertEqual(get_total_price( []), 0) self.assertEqual(get_total_price([90]), 90) [*] Mutation score [0.50795 s]: 96.4% - all: 28 - killed: 27 (96.4%) - survived: 1 (3.6%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%)
  49. 72.

    1 def get_total_price(cart_prices): 2 if len(cart_prices) == 0: 3 return

    0 4 5 result = {'TotalPrice': sum(cart_prices)} 6 if len(cart_prices) >= 2: 7 result['Discount'] = result['TotalPrice'] * 0.25 8 9 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(self): self.assertEqual(get_total_price([90, 10]), 75) self.assertEqual(get_total_price( []), 0) self.assertEqual(get_total_price([90]), 90) [*] Mutation score [0.50795 s]: 96.4% - all: 28 - killed: 27 (96.4%) - survived: 1 (3.6%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%) - survived: 1 (3.6%)
  50. 73.

    … ---------------------------------------------------------- 1: def get_total_price(cart_prices): 2: if len(cart_prices) == 0:

    ~3: pass 4: 5: result = {'TotalPrice': sum(cart_prices)} 6: if len(cart_prices) >= 2: 7: result['Discount'] = result['TotalPrice'] * 0.25 8: ---------------------------------------------------------- [0.00968 s] survived - [# 26] SDL target:5 : … [*] Mutation score [0.50795 s]: 96.4% - all: 28 - killed: 27 (96.4%) - survived: 1 (3.6%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%)
  51. 74.

    1 def get_total_price(cart_prices): 2 result = {'TotalPrice': sum(cart_prices)} 3 if

    len(cart_prices) >= 2: 4 result['Discount'] = result['TotalPrice'] * 0.25 5 6 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(self): self.assertEqual(get_total_price([90, 10]), 75) self.assertEqual(get_total_price( []), 0) self.assertEqual(get_total_price([90]), 90) [*] Mutation score [0.44658 s]: 100.0% - all: 23 - killed: 23 (100.0%) - survived: 0 (0.0%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%)
  52. 76.

    1 def get_total_price(cart_prices): 2 result = {'TotalPrice': sum(cart_prices)} 3 if

    len(cart_prices) >= 2: 4 result['Discount'] = result['TotalPrice'] * 0.25 5 6 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 assert get_total_price([90]) == 90 Name Stmts Miss Cover Missing -------------------------------------------- target.py 5 0 100.00%
  53. 77.

    1 def get_total_price(cart_prices): 2 result = {'TotalPrice': sum(cart_prices)} 3 if

    len(cart_prices) >= 2: 4 result['Discount'] = result['TotalPrice'] * 0.25 5 6 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 assert get_total_price([90]) == 90 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 5 0 2 0 100.00%
  54. 78.

    1 def get_total_price(cart_prices): 2 result = {'TotalPrice': sum(cart_prices)} 3 if

    len(cart_prices) >= 2: 4 result['Discount'] = result['TotalPrice'] * 0.25 5 6 return result['TotalPrice'] - result.get(‘Discount’, 0) def test_get_total_price(): assert get_total_price([90, 10]) == 75 assert get_total_price( []) == 0 assert get_total_price([90]) == 90 Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------- target.py 5 0 2 0 100.00%
  55. 82.
  56. 83.
  57. 84.

    Simple app app = Flask(__name__) @app.route('/get_total_discount', methods=['POST']) def get_total_discount(): cart_prices

    = json.loads(request.form['cart_prices']) result = {'TotalPrice': sum(cart_prices)} if len(cart_prices) >= 2: result['Discount'] = result['TotalPrice'] * 0.25 return jsonify(result['TotalPrice'] - result.get('Discount', 0)) flask_app.py
  58. 85.

    pip install pytest-flask @pytest.fixture def app(): from flask_app import app

    return app def test_get_total_discount(client): get_total_discount = lambda prices: client.post( '/get_total_discount', data=dict(cart_prices=json.dumps(prices)) ).json assert get_total_discount([90, 10]) == 75 assert get_total_discount( []) == 0 assert get_total_discount([90]) == 90 test_flask_app.py
  59. 86.

    pip install pytest-flask Name Stmts Miss Cover Missing ----------------------------------------------- flask_app.py

    9 0 100.00% py.test --cov-config=coverage.ini \ --cov=flask_app \ test_flask_app.py Name Stmts Miss Branch BrPart Cover Missing ------------------------------------------------------------- flask_app.py 9 0 2 0 100.00% py.test --cov-config=coverage_branch.ini \ --cov=flask_app \ test_flask_app.py
  60. 87.

    mutpy class FlaskTestCase(unittest.TestCase): def setUp(self): self.app = flask_app.app.test_client() def post(self,

    path, data): return json.loads(self.app.post(path, data=data).data.decode('utf-8')) def test_get_total_discount(self): get_total_discount = lambda prices: self.post( '/get_total_discount', data=dict(cart_prices=json.dumps(prices)) ) self.assertEqual(get_total_discount([90, 10]), 75) unittest_flask_app.py
  61. 88.

    mutpy [*] Mutation score [0.39122 s]: 100.0% - all: 27

    - killed: 1 (3.7%) - survived: 0 (0.0%) - incompetent: 26 (96.3%) - timeout: 0 (0.0%) mut.py --target flask_app --unit-test unittest_flask_app
  62. 89.

    mutpy [*] Mutation score [0.39122 s]: 100.0% - all: 27

    - killed: 1 (3.7%) - survived: 0 (0.0%) - incompetent: 26 (96.3%) - timeout: 0 (0.0%) mut.py --target flask_app --unit-test unittest_flask_app
  63. 90.

    mutpy def _matching_loader_thinks_module_is_package(loader, mod_name): #... raise AttributeError( ('%s.is_package() method is

    missing but is required by Flask of ' 'PEP 302 import hooks. If you do not use import hooks and ' 'you encounter this error please file a bug against Flask.') % loader.__class__.__name__)
  64. 91.

    mutpy def _matching_loader_thinks_module_is_package(loader, mod_name): #... raise AttributeError( ('%s.is_package() method is

    missing but is required by Flask of ' 'PEP 302 import hooks. If you do not use import hooks and ' 'you encounter this error please file a bug against Flask.') % loader.__class__.__name__) class InjectImporter: def __init__(self, module): # ... def find_module(self, fullname, path=None): # ... def load_module(self, fullname): # ... def install(self): # ... def uninstall(cls): # ...
  65. 92.

    mutpy class InjectImporter: def __init__(self, module): # ... def find_module(self,

    fullname, path=None): # ... def load_module(self, fullname): # ... def install(self): # ... def uninstall(cls): # … def is_package(self, fullname): # ...
  66. 93.

    mutpy [*] Mutation score [1.14206 s]: 100.0% - all: 27

    - killed: 25 (92.6%) - survived: 0 (0.0%) - incompetent: 2 (7.4%) - timeout: 0 (0.0%) mut.py --target flask_app --unit-test unittest_flask_app
  67. 94.
  68. 95.

    Simple app import json from django.http import HttpResponse def index(request):

    cart_prices = json.loads(request.POST['cart_prices']) result = {'TotalPrice': sum(cart_prices)} if len(cart_prices) >= 2: result['Discount'] = result['TotalPrice'] * 0.25 return HttpResponse(result['TotalPrice'] - result.get('Discount', 0)) django_root/billing/views.py
  69. 96.

    pip install pytest-django class TestCase1(TestCase): def test_get_total_price(self): get_total_price = lambda

    items: json.loads( self.client.post( '/billing/', data={'cart_prices': json.dumps(items)} ).content.decode('utf-8') ) self.assertEqual(get_total_price([90, 10]), 75) self.assertEqual(get_total_price( []), 0) self.assertEqual(get_total_price([90]), 90) django_root/billing/tests.py
  70. 97.

    pip install pytest-django Name Stmts Miss Cover Missing --------------------------------------------------- billing/views.py

    8 0 100.00% py.test --cov-config=coverage.ini \ --cov=billing.views \ billing/tests.py Name Stmts Miss Branch BrPart Cover Missing ----------------------------------------------------------------- billing/views.py 8 0 2 0 100.00% py.test --cov-config=coverage_branch.ini \ --cov=billing.views \ billing/tests.py
  71. 98.

    mutpy [*] Start mutation process: - targets: billing.views - tests:

    billing.tests [*] Tests failed: - error in setUpClass (billing.tests.TestCase1) - django.core.exceptions.ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings. mut.py --target billing.views --unit-test billing.tests
  72. 99.

    mutpy class Command(BaseCommand): def handle(self, *args, **options): operators_set = operators.standard_operators

    if options['experimental_operators']: operators_set |= operators.experimental_operators controller = MutationController( target_loader=ModulesLoader(options['target'], None), test_loader=ModulesLoader(options['unit_test'], None), views=[TextView(colored_output=False, show_mutants=True)], mutant_generator=FirstOrderMutator(operators_set) ) controller.run() django_root/mutate_command/management/commands/mutate.py
  73. 100.

    mutpy [*] Mutation score [1.07321 s]: 0.0% - all: 22

    - killed: 0 (0.0%) - survived: 22 (100.0%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%) python manage.py mutate \ --target billing.views --unit-test billing.tests
  74. 101.

    mutpy class RegexURLPattern(LocaleRegexProvider): def __init__(self, regex, callback, default_args=None, name=None): LocaleRegexProvider.__init__(self,

    regex) self.callback = callback # the view self.default_args = default_args or {} self.name = name django.urls.resolvers.RegexURLPattern
  75. 102.

    mutpy import importlib class Command(BaseCommand): def hack_django_for_mutate(self): def set_cb(self, value):

    self._cb = value def get_cb(self): module = importlib.import_module(self._cb.__module__) return module.__dict__.get(self._cb.__name__) import django.urls.resolvers as r r.RegexURLPattern.callback = property(callback, set_cb) def __init__(self, *args, **kwargs): self.hack_django_for_mutate() super().__init__(*args, **kwargs) def add_arguments(self, parser): # ...
  76. 103.

    mutpy [*] Mutation score [1.48715 s]: 100.0% - all: 22

    - killed: 22 (100.0%) - survived: 0 (0.0%) - incompetent: 0 (0.0%) - timeout: 0 (0.0%) python manage.py mutate \ --target billing.views --unit-test billing.tests
  77. 105.

    Links ✤ https://github.com/pytest-dev/pytest ✤ https://github.com/pytest-dev/pytest-flask ✤ https://github.com/pytest-dev/pytest-django ✤ https://bitbucket.org/ned/coveragepy ✤

    https://github.com/pytest-dev/pytest-cov ✤ https://bitbucket.org/khalas/mutpy ✤ https://github.com/sixty-north/cosmic-ray