Python: Encapsulation with Descriptors

Presented at OSCON 2014 in Portland, OR.

Transcript

1. Encapsulation with Descriptors Luciano Ramalho
 luciano@ramalho.org 2014-07-24
2. The scenario @ramalhoorg @ramalhoorg • Selling organic bulk foods • An order has several items • Each item has a description, 
 weight (kg), unit price (per kg)
 and a sub-total
3. @ramalhoorg ➊
4. @ramalhoorg ➊ the first doctest ============== LineItem tests ============== ! A line item for a bulk food order has description, weight and price fields:: ! >>> from bulkfood import LineItem >>> raisins = LineItem('Golden raisins', 10, 6.95) >>> raisins.weight, raisins.description, raisins.price (10, 'Golden raisins', 6.95) ! A ``subtotal`` method gives the total price for that line item:: ! >>> raisins.subtotal() 69.5
5. ➊ the simplest thing possible @ramalhoorg @ramalhoorg class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price
6. @ramalhoorg ➊ perhaps too simple... >>> raisins = LineItem('Golden raisins', 10, 6.95) >>> raisins.weight = -20 >>> raisins.subtotal() -139.0 a negative amount is due?!
7. ➊ perhaps too simple... @ramalhoorg @ramalhoorg >>> raisins = LineItem('Golden raisins', 10, 6.95) >>> raisins.weight = -20 >>> raisins.subtotal() -139.0 a negative amount is due?! Jeff Bezos of Amazon: Birth of a Salesman
 WSJ.com - http://j.mp/VZ5not “We found that customers could
 order a negative quantity of books! And we would credit their credit card with the price...” Jeff Bezos
8. ➊ the classic solution @ramalhoorg @ramalhoorg class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.set_weight(weight) self.price = price ! def subtotal(self): return self.get_weight() * self.price ! def get_weight(self): return self.__weight ! def set_weight(self, value): if value > 0: self.__weight = value else: raise ValueError('value must be > 0') protected attribute changes in existing code
9. @ramalhoorg • Previously we could get the weight of a line item simply via item.weight, but not anymore... • This breaks code • Python offers another way... ➊ but it’s a breaking change >>> raisins.weight Traceback (most recent call last): ... AttributeError: 'LineItem' object has no attribute 'weight'
10. @ramalhoorg ➊ by the way... • Protected attributes in Python exist only for safety reasons • to avoid accidental assignment or overriding • not to prevent intentional use (or misuse)
11. ➊ by the way... @ramalhoorg @ramalhoorg • Protected attributes in Python exist only for safety reasons • to avoid accidental assignment or overriding • not to prevent intentional use (or misuse) >>> raisins._LineItem__weight 10
12. @ramalhoorg ➋
13. Assigning a negative ``weight`` is forbidden:: ! >>> raisins.weight = -20 Traceback (most recent call last): ... ValueError: value must be > 0 >>> raisins.weight 10 @ramalhoorg ➋ the second doctest looks like a violation of encapsulation but business logic is enforced weight is unchanged
14. Assigning a negative ``weight`` is forbidden:: ! >>> raisins.weight = -20 Traceback (most recent call last): ... ValueError: value must be > 0 >>> raisins.weight 10 @ramalhoorg ➋ validation with property to make this work, we implement weight as a property
15. ➋ property implementation @ramalhoorg @ramalhoorg class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price ! @property def weight(self): return self.__weight ! @weight.setter def weight(self, value): if value > 0: self.__weight = value else: raise ValueError('value must be > 0') must be new-style class properties are only supported in new-style classes
16. @ramalhoorg ➋ property implementation class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price ! @property def weight(self): return self.__weight ! @weight.setter def weight(self, value): if value > 0: self.__weight = value else: raise ValueError('value must be > 0') within __init__ the property is already in use
17. @ramalhoorg ➋ property implementation class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price ! @property def weight(self): return self.__weight ! @weight.setter def weight(self, value): if value > 0: self.__weight = value else: raise ValueError('value must be > 0') the protected __weight attribute is touched only within the property methods
18. @ramalhoorg ➋ property implementation class LineItem(object): ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price ! @property def weight(self): return self.__weight ! @weight.setter def weight(self, value): if value > 0: self.__weight = value else: raise ValueError('value must be > 0') what if we need the same logic applied to the price attribute? duplicate the setter and getter?!?
19. @ramalhoorg ➌
20. __init__ subtotal description weight {descriptor} price {descriptor} LineItem @ramalhoorg ➌ the managed attributes we will use descriptors to manage access to the weight and price attributes, enforcing the business logic
21. ➌ @ramalhoorg @ramalhoorg descriptor implementation class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ! ! class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price new-style class
22. ➌ descriptor details @ramalhoorg @ramalhoorg weight and price are attributes of the LineItem class the get/set logic is moved
 to the Quantity descriptor class, where it can be reused via composition
23. @ramalhoorg ➌ a class builds instances class instances
24. @ramalhoorg class instances ➌ a class builds instances
25. @ramalhoorg ➌ descriptor instances
    attach to managed class the LineItem class has 2 instances of Quantity bound to it
26. ➌ @ramalhoorg @ramalhoorg class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ! ! class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price descriptor managed class managed attributes
27. ➌ use of the descriptor @ramalhoorg @ramalhoorg class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price the Quantity instances are created at import time import time: when a module is loaded and
 its classes and functions are defined
28. @ramalhoorg @ramalhoorg class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price ➌ use of the descriptor each Quantity instance manages one LineItem attribute
29. ➌ use of the descriptor @ramalhoorg @ramalhoorg class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price every access to weight and price goes through their descriptors
30. ➌ the challenge @ramalhoorg @ramalhoorg class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price when each descriptor is instantiated, the LineItem class does not exist, and the managed attributes don’t exist either
31. ➌ the challenge @ramalhoorg @ramalhoorg for example, the weight attribute is only created after Quantity() is evaluated class LineItem(object): weight = Quantity() price = Quantity() ! def __init__(self, description, weight, price): self.description = description self.weight = weight self.price = price ! def subtotal(self): return self.weight * self.price
32. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') @ramalhoorg ➌ descriptor implementation a descriptor is a class that defines __get__ , __set__ or __delete__ 

33. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') @ramalhoorg ➌ descriptor implementation self is the descriptor instance, bound to weight or price
34. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ➌ descriptor implementation @ramalhoorg @ramalhoorg self is the descriptor instance, bound to weight or price instance is the LineItem instance being handled
35. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ➌ descriptor implementation @ramalhoorg @ramalhoorg self is the descriptor instance, bound to weight or price owner is LineItem, the class to which instance belongs instance is the LineItem instance being handled
36. @ramalhoorg @ramalhoorg ➌ descriptor implementation __get__ and __set__ read and write the target attribute in each LineItem instance
37. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ➌ descriptor implementation @ramalhoorg @ramalhoorg target_name is the name of the LineItem instance attribute where where we store the value of the attribute managed by this descriptor instance (self)
38. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') ➌ descriptor implementation @ramalhoorg @ramalhoorg __get__ and __set__ use the getattr and setattr built-in functions to read and write the target attribute in the managed instance
39. @ramalhoorg @ramalhoorg ➌ descriptor implementation each descriptor instance manages a specific LineItem instance attribute, and needs a specific target_name
40. class Quantity(object): __counter = 0 ! def __init__(self): prefix = '_' + self.__class__.__name__ key = self.__class__.__counter self.target_name = '%s_%s' % (prefix, key) self.__class__.__counter += 1 ! def __get__(self, instance, owner): return getattr(instance, self.target_name) ! def __set__(self, instance, value): if value > 0: setattr(instance, self.target_name, value) else: raise ValueError('value must be > 0') @ramalhoorg ➌ descriptor implementation each Quantity instance must create and use a unique target attribute name
41. ➌ the target attributes @ramalhoorg @ramalhoorg >>> raisins = LineItem('Golden raisins', 10, 6.95) >>> dir(raisins) ['_Quantity_0', '_Quantity_1', '__class__', '__delattr__' ... 'description', 'price', 'subtotal', 'weight'] >>> raisins.weight 10 >>> raisins._Quantity_0 10 >>> raisins.price 6.95 >>> raisins._Quantity_1 6.95 _Quantity_0 and _Quantity_1 are the instance attributes where the weight and price values of each LineItem instance are stored
42. __init__ __get__ __set__ target_name «descriptor» Quantity __init__ subtotal description _Quantity_0 _Quantity_1 LineItem «weight» «price» «get/set target attribute» @ramalhoorg ➌ the target attributes _Quantity_0 stores the value of weight _Quantity_1 stores the value of price
43. __init__ subtotal description weight {descriptor} price {descriptor} LineItem @ramalhoorg ➌ the managed attributes clients of the LineItem class don’t need to know how (or if) weight and price are managed and they don’t need to know that _Quantity_0 and _Quantity_1 exist!
44. @ramalhoorg ➌ room for improvement • It would be better if the target attributes were proper protected attributes • _LineItem__weight instead of _Quantity_0 • To do so, we must find out the name of the managed attribute (eg. weight) • this is not as easy at it sounds • the extra complication may not be worthwhile
45. @ramalhoorg What now? • If a descriptor needs to know the real name of the managed class attribute (perhaps to persist to a database using descriptive column names) • ...then you need to control
 the construction of the 
 managed class with a 
 class decorator or a…

46. The full tutorial http://www.oscon.com/oscon2014/public/schedule/detail/34107
47. @ramalhoorg References • Raymond Hettinger’s 
 Descriptor HowTo Guide • part of the official Python docs • Alex Martelli’s
 Python in a Nutshell, 2e. • Python 2.5 but still relevant, includes the complete attribute access algorithm
48. @ramalhoorg References • David Beazley’s: • Python Essential Reference, 
 4th edition • Python Cookbook,
 3rd edition • Slides for this talk:
 http://speakerdeck.com/ramalho