Dustin Ingram - What Is and What Can Be: An Exploration from `type` to Metaclasses

Most of us use `type` every day, but few can say they know it well. This talk explores `type` and along the way, reveals how it relates to `object`, `class` and more, eventually arriving at deeper understanding of metaclasses in Python.

https://us.pycon.org/2016/schedule/presentation/2138/

Transcript

1. What Is and What Can Be An Exploration from type to metaclasses
2. I'm Dustin http://github.com/di
4. Irked.
5. Irked. >>> j = object()
6. Irked. >>> j = object() >>> j.hi = 'there'
7. Irked. >>> j = object() >>> j.hi = 'there' Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'object' object has no attribute 'hi'
8. type
9. type >>> type(['foo', 'bar'])
10. type >>> type(['foo', 'bar']) <class 'list'>
11. type >>> type(list)
12. type >>> foo = list()
13. type >>> foo = list() >>> foo []
14. type >>> foo = list() >>> foo [] >>> type(foo) <class 'list'>
15. type >>> type(list)
16. type >>> type(list) <class 'type'>
17. type >>> type(type)
18. type >>> type(type) <class 'type'>
19. type >>> type(None)
20. type >>> type(None) <class 'NoneType'>
21. type >>> def func(): ... pass ... >>> type(func)
22. type >>> def func(): ... pass ... >>> type(func) <class 'function'>
23. type >>> import types >>> dir(types) ['BuiltinFunctionType', 'BuiltinMethodType', 'CodeType', 'CoroutineType', 'DynamicClassAttribute', 'FrameType', 'FunctionType', 'GeneratorType', 'GetSetDescriptorType', 'LambdaType', 'MappingProxyType', 'MemberDescriptorType', 'MethodType', 'ModuleType', 'SimpleNamespace', 'TracebackType', ... ]
24. type >>> import types >>> types.FunctionType <class 'function'>
25. type >>> import types >>> type(types)
26. type >>> import types >>> type(types) is types.ModuleType True >>> type(types) <class 'module'>
27. type >>> class FooClass: ... pass ... >>> type(FooClass)
28. type >>> class FooClass: ... pass ... >>> type(FooClass()) <class '__main__.FooClass'>
29. type >>> class FooClass: ... pass ... >>> type(FooClass)
30. type >>> class FooClass: # Python 2.7 ... pass ... >>> type(FooClass) <type 'classobj'>
31. type >>> class FooClass: # Python 3.5 ... pass ... >>> type(FooClass) <class 'type'>
32. type >>> class FooClass(object): # Python 2.7 ... pass ... >>> type(FooClass) <type 'type'>
33. Python is dead. Long live Python
34. type >>> type(42)
35. type >>> type(42) <class 'int'>
36. type >>> type(42) is int True
37. type >>> type(42) is int True >>> type(42)() 0
38. type >>> type(42) is int True >>> type(42)() 0 >>> int() 0
39. type >>> j = type()
40. type >>> j = type() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: type() takes 1 or 3 arguments
41. type >>> class
42. type >>> class FooClass
43. type >>> class FooClass(object):
44. type >>> class FooClass:
45. type >>> class FooClass: ... hi = 'there'
46. type >>> j = type(
47. type >>> j = type( ... 'FooClass',
48. type >>> j = type( ... 'FooClass', ... (object,),
49. type >>> j = type( ... 'FooClass', ... (object,), ... {'hi': 'there'}, ... )
50. type >>> j = type( ... 'FooClass', ... (object,), ... {'hi': 'there'}, ... ) >>> type(j) <class 'type'>
51. type >>> j = type( ... 'FooClass', ... (object,), ... {'hi': 'there'}, ... ) >>> j.hi 'there'
52. type >>> j = type( ... '', ... (object,), ... {'hi': 'there'}, ... ) >>> j.hi 'there'
53. type >>> j = type( ... '', ... (), ... {'hi': 'there'}, ... ) >>> j.hi 'there'
54. type >>> j = type( ... '', ... (), ... {}, ... ) >>> j.hi = 'there' >>> j.hi 'there'
55. (side note)
56. stub $ pip install pretend >>> from pretend import stub >>> j = stub(hi='there') >>> j.hi 'there'
57. Metaclasses
58. Metaclasses "The subject of metaclasses in Python has caused hairs to raise and even brains to explode."
59. Metaclasses "The subject of metaclasses in Python has caused hairs to raise and even brains to explode." — Guido van Rossum
60. Metaclasses classes : instances : : metaclasses : classes
61. Metaclasses >>> j = type( ... 'FooClass', ... (object,), ... {'hi': 'there'}, ... ) >>> type(j) <class 'type'>
62. Metaclasses >>> class MyMeta(type): ... pass ...
63. Metaclasses >>> class MyMeta(type): ... pass ... >>> class FooClass(metaclass=MyMeta): ... pass ...
64. Metaclasses >>> class MyMeta(type): ... def __new__(meta, name, bases, attrs): ... return super().__new__( ... meta, name, bases, attrs ... ) ... >>> class FooClass(metaclass=MyMeta): ... pass ...
65. Metaclasses >>> class MyMeta(type): ... def __new__(meta, name, bases, attrs): ... print('New {}'.format(name)) ... return super().__new__( ... meta, name, bases, attrs ... ) ... >>> class FooClass(metaclass=MyMeta): ... pass ... New FooClass
66. Metaclasses >>> class MyMeta(type): ... def __call__(cls, *args, **kwargs): ... print('Call {}'.format( ... cls.__name__ ... )) ... return super().__call__( ... *args, **kwargs ... ) ... >>> class FooClass(metaclass=MyMeta): ... pass ... >>> f = FooClass() Call FooClass   
67. Metaclasses "Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why)." — Tim Peters
68. Metadog
69. Metadog >>> class Dog(): ... def sit(self): ... print("*sitting*") ...
70. Metadog >>> class Dog(): ... def sit(self): ... print("Growl!") ... print("*sitting*") ...
71. Metadog >>> class Dog(): ... def sit(self): ... print("Growl!") ... print("*sitting*") ... def stay(self): ... print("Growl!") ... print("*staying*") ...
72. Metadog >>> class Dog(): ... def _woof(self): ... print("Woof!") ... def sit(self): ... self._woof() ... print("*sitting*") ... def stay(self): ... self._woof() ... print("*staying*") ...
73. Metadog >>> from functools import wraps >>> def woof(f): ... @wraps(f) ... def wrapper(*args, **kwargs): ... print('Woof!') ... return f(*args, **kwargs) ... return wrapper ...
74. Metadog >>> class Dog(): ... @woof ... def sit(self): ... print("*sitting*") ... @woof ... def stay(self): ... print("*staying*") ...
75. Metadog >>> class Dog(): ... @woof ... def sit(self): ... print("*sitting*") ... @woof ... def stay(self): ... print("*staying*") ... def play_dead(self): ... print("*playing_dead*") ...
76. Metadog >>> from inspect import isfunction >>>
77. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ...
78. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ... def __new__(meta, name, bases, attrs): ...
79. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ... def __new__(meta, name, bases, attrs): ... for name, attr in attrs.items(): ...
80. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ... def __new__(meta, name, bases, attrs): ... for name, attr in attrs.items(): ... if isfunction(attr): ...
81. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ... def __new__(meta, name, bases, attrs): ... for name, attr in attrs.items(): ... if isfunction(attr): ... attrs[name] = woof(attr) ...
82. Metadog >>> from inspect import isfunction >>> class MetaDog(type): ... def __new__(meta, name, bases, attrs): ... for name, attr in attrs.items(): ... if isfunction(attr): ... attrs[name] = woof(attr) ... return type.__new__( ... meta, name, bases, attrs ... ) ...
83. Metadog >>> class Dog(metaclass=MetaDog): ... def sit(self): ... print("(sitting)") ... def stay(self): ... print("(staying)") ... def play_dead(self): ... print("*playing_dead*") ...
84. Metadog >>> my_dog = Dog()
85. Metadog >>> my_dog = Dog() >>> my_dog.sit() Woof! *sitting*
86. Metadog >>> my_dog = Dog() >>> my_dog.sit() Woof! *sitting* >>> my_dog.play_dead() Woof! *playing dead*
87. Metadog
88. Metaclasses >>> class FooClass(): ... pass ...
89. Metaclasses >>> class FooClass(): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b False
90. Singleton
91. Singleton >>> class Singleton(): ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
92. Singleton >>> class Singleton(): ... _instance = None ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
93. Singleton >>> class Singleton(): ... _instance = None ... def __new__(cls, *args, **kwargs): ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
94. Singleton >>> class Singleton(): ... _instance = None ... def __new__(cls, *args, **kwargs): ... if not cls._instance: ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
95. Singleton >>> class Singleton(): ... _instance = None ... def __new__(cls, *args, **kwargs): ... if not cls._instance: ... cls._instance = object.__new__( ... cls, *args, **kwargs ... ) ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
96. Singleton >>> class Singleton(): ... _instance = None ... def __new__(cls, *args, **kwargs): ... if not cls._instance: ... cls._instance = object.__new__( ... cls, *args, **kwargs ... ) ... return cls._instance ... >>> class FooClass(Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
97. Singleton >>> class Singleton(type): ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
98. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
99. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
100. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... return super().__new__(meta, name, bases, attrs) ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
101. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... return super().__new__(meta, name, bases, attrs) ... def __call__(cls, *args, **kwargs): ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
102. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... return super().__new__(meta, name, bases, attrs) ... def __call__(cls, *args, **kwargs): ... if not cls._instance: ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
103. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... return super().__new__(meta, name, bases, attrs) ... def __call__(cls, *args, **kwargs): ... if not cls._instance: ... cls._instance = super().__call__(*args, **kwargs) ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
104. Singleton >>> class Singleton(type): ... def __new__(meta, name, bases, attrs): ... attrs['_instance'] = None ... return super().__new__(meta, name, bases, attrs) ... def __call__(cls, *args, **kwargs): ... if not cls._instance: ... cls._instance = super().__call__(*args, **kwargs) ... return cls._instance ... >>> class FooClass(metaclass=Singleton): ... pass ... >>> a, b = FooClass(), FooClass() >>> a is b True
105. Singleton
106. Still irked, probably.
107. Still irked, probably. >>> j = object() >>> j.hi = 'there' Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'object' object has no attribute 'hi'
108. Still irked, probably. >>> class FooClass(): ... pass ... >>> hasattr(FooClass(), '__dict__') True >>> hasattr(object(), '__dict__') False
109. Still irked, probably. >>> from stackoverflow import getsize¹ >>> getsize(object()) 16 >>> getsize(0) 24 >>> getsize(dict()) 280 >>> getsize(FooClass()) 344 1 http://stackoverflow.com/a/30316760/4842627
110. Conclusion
111. Conclusion >>> j = type('', (), {})
112. Conclusion >>> class MetaDog(type): ...
113. Conclusion >>> class FooClass(metaclass=Singleton): ...
114. Conclusion
115. Thanks! http://github.com/di