Magic Of Python

Slides for Magic of Python Talk by Darshan Markandaiah at PyGotham 2019.

Transcript

1. Magic of Python Darshan Markandaiah @darshanMark dmark11235@gmail.com bit.ly/magic-of-python 1
2. Agenda bit.ly/magic-of-python 2
3. What are Magic Methods? bit.ly/magic-of-python 3
4. Groups of Magic Methods • Define Custom Behavior • Enable Operator Overloading • Emulate Built in types bit.ly/magic-of-python 4
5. Code for a basic Shopping Cart class BaseCart: def __init__(self, user_id): self.user_id = user_id # Mapping of item_id -> item_count self.cart_items = dict() def add_item(self, item_id, item_count=1): self.cart_items[item_id] = self.cart_items.get(item_id, 0) + item_count bit.ly/magic-of-python 5
6. >>> cart_1 = BaseCart("user_id_1") >>> cart_1.add_item("item_1") >>> print(cart_1) <__main__.BaseCart object at 0x10a196f28> bit.ly/magic-of-python 6
7. Define Custom Behavior Define Custom Behavior 7
8. __repr__ / __str__ Define Custom Behavior 8
9. class Cart(BaseCart): def __str__(self): return f"Cart for user_id:{self.user_id} has {len(self.cart_items)} items." def __repr__(self): return f"Cart('{self.user_id}')" Define Custom Behavior 9
10. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1") >>> cart_1.add_item("item_id_2") >>> cart_1 Cart('user_id_1') >>> print(cart_1) Cart for user_id:user_id_1 has 2 items. >>> print(f"String formatting: {cart_1}") String formatting: Cart for user_id:user_id_1 has 2 items. Define Custom Behavior 10
11. __bool__ Define Custom Behavior 11
12. class Cart(BaseCart): def __bool__(self): return len(self.cart_items) > 0 Define Custom Behavior 12
13. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1") >>> if cart_1: # operations for a non empty cart object print("Cart contains at least 1 item.") else: # operations for an empty cart object print("Cart is empty.") Cart contains at least 1 item. Define Custom Behavior 13
14. Other Customization Methods Method Invocation __hash__ hash(object) __del__ del object* Define Custom Behavior; *: called when reference count reaches 0 14
15. Enable Operator Overloading Enable Operator Overloading 15
16. __gt__ Enable Operator Overloading 16
17. class Cart(BaseCart): def __gt__(self, other): return len(self.cart_items) > len(other.cart_items) Enable Operator Overloading 17
18. >>> cart_1 = Cart("user_id_1") >>> cart_2 = Cart("user_id_2") >>> cart_1.add_item("item_id_1") >>> cart_2.add_item("item_id_1") >>> print(cart_1 > cart_2) False cart_1.add_item("item_id_2") >>> print(cart_1 > cart_2) True Enable Operator Overloading 18
19. Other Comparison Methods Method Invocation __ge__ self >= other __lt__ self < other __le__ self <= other __eq__ self == other __ne__ self != other Enable Operator Overloading 19
20. @functools.total_ordering Enable Operator Overloading 20
21. from functools import total_ordering @total_ordering class Cart(BaseCart): def __gt__(self, other): return len(self.cart_items) > len(other.cart_items) def __eq__(self, other): return len(self.cart_items) == len(other.cart_items) Enable Operator Overloading 21
22. >>> cart_1 = Cart("user_id_1") >>> cart_2 = Cart("user_id_2") >>> cart_1.add_item("item_id_1") >>> cart_2.add_item("item_id_1") >>> print(cart_1 < cart_2) False >>> print(cart_1 >= cart_2) True Enable Operator Overloading 22
23. __add__ Enable Operator Overloading 23
24. class Cart(BaseCart): def __add__(self, other): new_cart = Cart(self.user_id) for item_id, item_count in self.cart_items.items(): new_cart.add_item(item_id, item_count) for item_id, item_count in other.cart_items.items(): new_cart.add_item(item_id, item_count) return new_cart Enable Operator Overloading 24
25. >>> cart_1, cart_2 = Cart("user_id_1"), Cart("user_id_2") >>> cart_1.add_item("item_id_1") >>> cart_2.add_item("item_id_1") >>> cart_2.add_item("item_id_2") >>> cart_3 = cart_1 + cart_2 >>> print(cart_3.cart_items) {'item_id_1': 2, 'item_id_2': 1} Enable Operator Overloading 25
26. Other arithmetic magic methods Method Invocation __sub__ self - other __mul__ self * other __div__ self / other __pow__ self ** other Enable Operator Overloading 26
27. Reflected numerical magic methods Method Invocation __radd__ other + self __rsub__ other - self __rmul__ other * self __rdiv__ other / self __rpow__ other ** self Enable Operator Overloading 27
28. Unary Operations Method Invocation __abs__ abs(self) __neg__ -self __pos__ +self and others found in the documentation... Enable Operator Overloading 28
29. Emulate Container Types Emulate Container Types 29
30. __len__ Emulate Container Types 30
31. class Cart(BaseCart): def __len__(self): return sum(self.cart_items.values()) Define Custom Behavior 31
32. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1", 2) >>> cart_1.add_item("item_id_2") >>> len(cart_1) 3 Define Custom Behavior 32
33. __contains__ Emulate Container Types 33
34. class Cart(BaseCart): def __contains__(self, key): return key in self.cart_items Emulate Container Types 34
35. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1") >>> print("item_id_1" in cart_1) True >>> print("item_id_2" in cart_1) False Emulate Container Types 35
36. __iter__ Emulate Container Types 36
37. Iterator Protocol • Iterables • Iterators How For Loops Work: Trey Hunner 37
38. >>> iterable = [1, 2] >>> iterator = iter(iterable) >>> print(iterator) <list_iterator at 0x108c2e048> >>> print(next(iterator)) 1 >>> print(next(iterator)) 2 >>> print(next(iterator)) StopIteration How For Loops Work: Trey Hunner 38
39. __iter__ Emulate Container Types 39
40. class Cart(BaseCart): def __iter__(self): for item_id in self.cart_items: yield item_id Emulate Container Types 40
41. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1") >>> cart_1.add_item("item_id_1") >>> cart_1.add_item("item_id_2") >>> for item in cart_1: print(item) 'item_id_1' 'item_id_2' Emulate Container Types 41
42. Other methods for Emulating Containers Method Name Similar __getitem__ self[key] __setitem__ self[key] = value __delitem__ del(self[key]) and others found in the documentation... Emulate Container Types 42
43. collections.abc bit.ly/magic-of-python 43
44. Use Case: Checking for interfaces bit.ly/magic-of-python 44
45. >>> from collections import abc >>> isinstance(list(), abc.Sequence) True >>> isinstance(dict(), abc.Mapping) True >>> isinstance(list(), abc.Hashable) False >>> isinstance(str(), abc.Hashable) True bit.ly/magic-of-python 45
46. Use Case: Offer Free Mixin Methods bit.ly/magic-of-python 46
47. from collections import abc class Cart(BaseCart, abc.Mapping): def __iter__(self): for key in self.cart_items: yield key def __len__(self): return len(self.cart_items) def __getitem__(self, key): return self.cart_items[key] bit.ly/magic-of-python 47
48. >>> cart_1 = Cart("user_id_1") >>> cart_1.add_item("item_id_1") >>> cart_1.add_item("item_id_2") >>> 'item_id_1' in cart_1 True >>> list(cart_1.keys()) ['item_id_1', 'item_id_2'] bit.ly/magic-of-python 48
49. other ABCs that offer free mixin methods bit.ly/magic-of-python 49
50. Context Managers bit.ly/magic-of-python 50
51. file_handle = open("/path/to/file") try: content = file_handle.read() finally: file_handle.close() with open("/path/to/file") as file_handle: content = file_handle.read() bit.ly/magic-of-python 51
52. Defining your own Context Managers • __enter__ • __exit__ bit.ly/magic-of-python 52
53. class FileReader: def __init__(self, filename): self.filename = filename self.file_handle = None def __enter__(self): self.file_handle = open(self.filename) return self.file_handle def __exit__(self, exc_type, exc_value, traceback): if self.file_handle: self.file_handle.close() with FileReader("/pat/to/file") as file_handle: content = file_handle.read() bit.ly/magic-of-python 53
54. @contextlib.contextmanager bit.ly/magic-of-python 54
55. Recap • Enumerated over Magic Methods • Using abc in the collections module to get free mixin functionality • Using contextmanagers to manage runtime contexts bit.ly/magic-of-python 55
56. Resources and Thanks • Rafe Kettler: A Guide to Python's Magic Methods • Dan Bader: Enriching Your Python Classes With Dunder Methods • Trey Hunner: How to make an iterator in Python • Trey Hunner: How for loops work in Python • Python Data Model Documentation • Python collections module documentation bit.ly/magic-of-python 56
57. Thank You! @darshanMark dmark11235@gmail.com bit.ly/magic-of-python 57