Introduction to MetaClasses

In this talk, we’ll explore metaclasses in Python and learn how to leverage them for having custom behavior across the codebase. We’ll explore the infamous type and how it comes into play in metaclasses. Lastly, we’ll cover how metaclasses can be useful for Abstract Base Classes which comes from the abc module in the standard library.

A class is a blueprint for an object. It describes how an instance of the class (ie an object) would behave. A metaclass is the class of a class. It defines how a class behaves. When you define a class in Python, its metaclass is called which when combined with __init__ and __new__ methods allows doing extra things like registering class with some registry or replace it entirely with something different.

The whole magic comes with type. It is a metaclass in Python, which itself is a class and it is of its own type. (yes, inception!).

Transcript

1. Introduction to MetaClasses Sanyam Khurana ErSanyamKhurana CuriousLearner PyGotham 2020 | New York City | PyGotham TV
2. def __who_am_i__(): • One of you! A part of the community • Goes by “CuriousLearner” over the web / IRC • FOSS contributor • Bug triager for CPython • Contributor to Django and Mozilla’s ecosystem of projects • Maintainer for django-phone-verify
3. MetaProgramming ErSanyamKhurana CuriousLearner
4. MetaClasses ErSanyamKhurana CuriousLearner
5. Everything is an “Object” ErSanyamKhurana CuriousLearner
6. Everything is an “Object” ErSanyamKhurana CuriousLearner int str
 float functions
 class
7. Classes are objects ErSanyamKhurana CuriousLearner
8. Classes are objects ErSanyamKhurana CuriousLearner
9. Classes are objects ErSanyamKhurana CuriousLearner
10. Classes are objects ErSanyamKhurana CuriousLearner Assign class to a variable Copy class Add attributes to a class Pass class as parameters to a function Classes are objects
11. ErSanyamKhurana CuriousLearner Django Phone Verify
12. The esoteric `type` ErSanyamKhurana CuriousLearner • You might already have seen or used type in your Python code
13. ErSanyamKhurana CuriousLearner Classes are objects
14. What’s the type of built-in classes ErSanyamKhurana CuriousLearner
15. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type
16. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type Name of the class to be created
17. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type Name of the class to be created Tuple of all the base classes that the new class should inherit from
18. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type Name of the class to be created Tuple of all the base classes that the new class should inherit from Dictionary of all the attributes of new class in key value pairs
19. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type
20. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type
21. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type
22. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type Both are equivalent
23. How to define metaclass for your class? ErSanyamKhurana CuriousLearner
24. How to define metaclass for your class? ErSanyamKhurana CuriousLearner
25. How to define custom meta class for your class? ErSanyamKhurana CuriousLearner
26. The esoteric `type` ErSanyamKhurana CuriousLearner Creating classes using type That’s the entire magic of dynamic class creation that happens when the interpreter sees the block of code indicating a class. type is the default metaclass
27. The world of Analogies ErSanyamKhurana CuriousLearner child : parent :: parent : grandparent object : class :: class : metaclass
28. What’s the type of type ErSanyamKhurana CuriousLearner
29. What’s the type of type ErSanyamKhurana CuriousLearner
30. What’s the type of type ErSanyamKhurana CuriousLearner
31. What’s the type of type ErSanyamKhurana CuriousLearner
32. What’s the type of type ErSanyamKhurana CuriousLearner Inception
33. What’s the type of type ErSanyamKhurana CuriousLearner
34. What’s a MetaClass : Revisited ErSanyamKhurana CuriousLearner It’s a class factory. Provides a way to create classes on the fly. Let’s you hook-into the class creation logic to validate things ensuring the newly derived class conforms to a certain API standard.
35. Singleton Classes ErSanyamKhurana CuriousLearner
36. Singleton Classes ErSanyamKhurana CuriousLearner
37. Singleton Classes ErSanyamKhurana CuriousLearner
38. Singleton Classes ErSanyamKhurana CuriousLearner
39. Singleton Classes ErSanyamKhurana CuriousLearner
40. Singleton Classes ErSanyamKhurana CuriousLearner
41. Singleton Classes ErSanyamKhurana CuriousLearner
42. Abstract Base Classes ErSanyamKhurana CuriousLearner
43. Abstract Base Classes ErSanyamKhurana CuriousLearner
44. Deeper look at MetaClasses • Called before class body is executed • Must return an instance of dict with local namespace for all the code in the class body. • PEP-3115 — Metaclasses in Python 3000 ErSanyamKhurana CuriousLearner __prepare__ method
45. Deeper look at MetaClasses ErSanyamKhurana CuriousLearner
46. Deeper look at MetaClasses • The __new__ method is the constructor — generates the new instance • __init__ is just a initializer — called after instance is created • __new__ is looked up statically on the instance (class) itself and not up in the hierarchy such as in the metaclass. ErSanyamKhurana CuriousLearner __new__ & __init__ method
47. ErSanyamKhurana CuriousLearner Deeper look at Instance Creation Image Source: https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/
48. ErSanyamKhurana CuriousLearner Deeper look at Class Creation Image Source: https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/
49. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122
50. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122
51. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122
52. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122 __iter__
53. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122 __iter__
54. ErSanyamKhurana CuriousLearner MetaClass: A deeper look Source: https://github.com/python/cpython/blob/3.9/Lib/enum.py#L122 __iter__ __len__
55. Multiple Inheritance in MetaClasses ErSanyamKhurana CuriousLearner
56. Multiple Inheritance in Metaclasses ErSanyamKhurana CuriousLearner
57. Multiple Inheritance in Metaclasses Only works when you have single root MetaClass from which all base classes inherit from. ErSanyamKhurana CuriousLearner
58. Multiple Inheritance in Metaclasses Only works when you have single root MetaClass from which all base classes inherit from. ErSanyamKhurana CuriousLearner
59. When to use MetaClasses? Before you uses metaclass, ask yourself: - Can I accomplish what I want with a decorator? If yes, then choose decorator. - Am I trying to provide an API for programmers to use (using my library/framework), specifically for derived classes? Would MetaClass reduce the amount of code an application programmer has to write? If yes, then use metaclasses. ErSanyamKhurana CuriousLearner
60. Use cases of MetaClasses • Validation framework for attributes of classes • Hook into the class creation process to control either the inheritance to or fro from the custom class. • Emulating behaviour in other programming languages, such as making a final class as seen in Java. • Hook into the class creation process to maintain order of class definitions (not instance creations) • Controlling meta stuff about the class like enforcing all the private methods of the class to have docstrings. ErSanyamKhurana CuriousLearner
61. References • https://stackoverflow.com/questions/100003/what-are-metaclasses-in- python • https://stackoverflow.com/questions/392160/what-are-some-concrete- use-cases-for-metaclasses • https://eli.thegreenplace.net/2011/08/14/python-metaclasses-by- example/ • https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/ • https://www.python.org/dev/peps/pep-3115/ • https://docs.python.org/3/reference/datamodel.html#customizing-class- creation ErSanyamKhurana CuriousLearner
62. Subscribe for updates: SanyamKhurana.com/python.html ErSanyamKhurana CuriousLearner
63. Many Thanks! Slides available at: https://speakerdeck.com/CuriousLearner/ introduction-to-metaclasses Reach out to me at: sanyam@sanyamkhurana.com ErSanyamKhurana CuriousLearner