Describing Descriptors

Oftentimes beginner programmers go through traditional features when learning a language. For Python beginners this might involve variables, control structures like if-statements, while and for loops, dictionaries, and finally classes. However, if we read the Python documentation we find that another feature that can be used in Python is that of the descriptor protocol. Descriptors allow the programmer to override the storing and retrieving of different class instance variables such that special behaviours can be followed. For example, we might want some variable to follow some special validation. We could do this using __setattr__ on the containing class but perhaps we want to reuse the validation in another class, or we want other validations for other variables and we don’t want __setattr__ to become a huge if/elif/else block. In this talk, I will walk attendees through what a descriptor is, what use cases they can use them in, how to implement a descriptor, and common descriptors in the Python ecosystem that users may or may not have identified as descriptors (often just referred to as magic).

Transcript

1. Describing Descriptors Matthew Egan 1
2. • Matthew Egan • DiviPay • Sydneysider • mattjegan • NullMatthew • https://mattjegan.com Any opinions conveyed in this talk are my own and not my employers 2
3. Thanks • Adam Jacquier-Parr - www.github.com/aljp • Christopher Di Bella - www.cjdb.com.au • Daniel Kniaz & Russell Martin - www.divipay.com 3
4. Who is this for? • Everyone who is comfortable with classes in Python • I won’t be covering meta-classes in this talk 4
5. Why Descriptors? • Descriptors are a feature of Python that is often overlooked • Used primarily by library developers rather than application developers 5
6. PSA: Use Python 3 • This talk uses Python 3.6+ • Python 2 is now at end of life and won’t be maintained after 2020 • Long live Python 3! 6
7. What? • A problem • Some solutions • A better solution (hint: it’s descriptors) 7
8. Classes - A refresher 8
9. A Problem • How can we make sure the name is always capitalized? 9
10. The current Person class: Our dream Person class: 10
11. __setattr__ • Is called whenever we try to set an instance variable • Takes the calling object, the name of the attribute being set, and the new value 11
12. __setattr__ 12
14. __setattr__ • This does the job • Locked to this class • Gets out of hand with many properties 14
15. @property and @name.setter 15
17. @property and @name.setter • This does the job • Locked to this class • Gets out of hand with many properties 17
18. What about… Simple Class Code + Reusable = Maintainable 18
19. Furthermore what about… Simple Class Code + Reusable = Maintainable 19
20. The Descriptor Protocol • Python exposes the descriptor protocol as a way to customize the storing and retrieval of instance attributes • Self-contained • Reusable 20
21. The Descriptor Protocol • __get__ is called when accessing the attribute • __set__ is called when storing the attribute • __delete__ is called when the attribute is being deleted • __set_name__ is called once when the class object is created (Python 3.6+, PEP487) 21
22. The Descriptor Protocol 0 350,000 700,000 1,050,000 1,400,000 __get__ __set__ __delete__ __set_name__ 22
23. __get__ 23
24. __set__ 24
25. __delete__ 25
26. __set_name__ • Only called on class creation 26
27. Types of Descriptors Data Descriptor Non-data Descriptor __set__ YES NO __delete__ YES 27
28. Descriptor Precedence Data Descriptor __dict__ Non-data Descriptor 28
29. Non-data Descriptors 29 • staticmethod • classmethod • abc.abstractmethod • functools.partialmethod
30. WeakKeyDictionary 30
31. Example 31
32. Example 32
33. A Solution How do we solve our problem of capitalizing the name? 33
35. Use Cases • Replicating customisation for multiple attrs without caring about name • Django generic foreign keys • Validation • Better error messages 35
36. Django Generic Foreign Keys • Needs to store 2 values rather than 1 36
37. Validation 37
38. Error Messages Not nice Nice 38
39. Error Messages 39
40. https://bit.ly/describing-descriptors 40 mattjegan NullMatthew https://mattjegan.com