Property Based Testing: Hypothesis

Hypothesis is a property based framework for Python. This talk will introduce Property Based Testing and show you how Hypothesis can help you improve your testing

Transcript

1. Hypothesis Property Based Testing
2. class HistoricalEvent(object): def __init__(self, event_id, description, event_time): """ :type id: int :type description: str :type event_time: datetime.datetime """ self.id = event_id self.description = description self.event_time = event_time def __eq__(self, other): return self.__dict__ == other.__dict__ def __repr__(self): return str(self.__dict__)
3. class EventEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, HistoricalEvent): return { "id": obj.id "description": obj.description "event_time": obj.event_time.isoformat(), } return json.JSONEncoder.default(self, obj)
4. def fromJson(dct): """ :type data: dict Data hopefully representing a Historical Event :return: HistoricalEvent """ return HistoricalEvent( dct['id'], dct['description'], dateutil.parser.parse(dct['event_time']) )
5. Key Property
6. Hypothesis • Based On Haskell’s QuickCheck • Fantastic Docs • Property Based Testing ◦ You define the properties of the input and the output ◦ It throws input matching your spec at your tests using strategies ◦ Reduces failures to simple cases
7. @given( st.integers(), st.text(), datetimes(timezones=['UTC']) ) def test_to_from_json(self, id, desc, date): original_event = new HistoricalEvent(id, desc, date) encoded_event = json.dumps(event, cls=EventEncoder) decoded_event = json.loads( encoded_event, object_hook=HistoricalEvent.fromJson ) assert decoded_event == original_event
8. Ran 1 tests in 0.734s Falsifying example: test_serialization_deserialization( self=<test_user.TestUser testMethod=test_serialization_deserialization>, user={ 'event_time': datetime.datetime(99, 1, 1, 0, 0, tzinfo=<UTC>), 'id': 0, 'description': '' } ) Successfully shrunk example 64 times
9. def fromJson(dct): """ :type data: dict Data hopefully representing a Historical Event :return: HistoricalEvent """ return HistoricalEvent( dct['id'], dct['description'], dateutil.parser.parse(dct['event_time']) )
10. In [0]: example = datetime(99, 1, 1, 0, 0) In [1]: example.isoformat() Out[1]: '0099-01-01T00:00:00' In [2]: dateutil.parser.parse('0099-01-01T00:00:00') Out[2]: datetime.datetime(1999, 1, 1, 0, 0)
11. This flaw was not contrived
14. Takeaways • Functionality tested with very little code • This was not an obvious issue ◦ But a real one! The first Century was a thing!
15. Thanks Matt Bachmann GitHub @mattbachmann Slides: https://goo.gl/zdNnTW