Hynek Schlawack @hynek@mastodon.social • @hynek Subclassing, Composition, Python, & You.

No content

No content

No content

class Base: pass class Sub(Base): pass Subclassing

class Base: pass class Sub(Base): pass Subclassing

@dataclass class Base: x: int class Sub(Base): y = 0 def compute(self): self.y += self.x * 2

@dataclass class Base: x: int def cool_new_feature(self): self.y = "LOL" class Sub(Base): y = 0 def compute(self): self.y += self.x * 2

Bidirectional Relationship

Composition

@dataclass class A: b: B y: int = 0 Composition

@dataclass class A: b: B y: int = 0 Composition

@dataclass class A: b: B y: int = 0 def compute(self): self.y += self.b.x * 2 Composition

@dataclass class A: b: B y: int = 0 def compute(self): self.y += self.b.x * 2 Composition

Why do we subclass?

Code Sharing

No content

No content

No content

Goose Chase Program Flow™

class A: def method(self): ... class B: def method(self): ... class C: def method(self): ... class You(A, B, C): def foo(self): self.method()

class A: def method(self): self.bar() class B(A): def bar(self): ... class C(B): def bar(self): ... class You(C): def foo(self): self.method()

class A: def method(self): self.bar() class B(A): def bar(self): ... class C(B): def bar(self): ... class You(C): def foo(self): self.method()

No content

— Architecture Patterns with Python, page 131

No content

Repository

Tracking Repository

No content

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set()

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set() @abc.abstractmethod def _add(self, product): raise NotImplementedError @abc.abstractmethod def _get(self, sku): raise NotImplementedError

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set() def add_product(self, product): self._add(product) self.seen.add(product) @abc.abstractmethod def _add(self, product): raise NotImplementedError @abc.abstractmethod def _get(self, sku): raise NotImplementedError

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set() def add_product(self, product): self._add(product) self.seen.add(product) def get_by_sku(self, sku): if product := self._get(sku): self.seen.add(product) return product @abc.abstractmethod def _add(self, product): raise NotImplementedError @abc.abstractmethod def _get(self, sku): raise NotImplementedError

No content

class FakeTrackingRepository(AbstractTrackingRepository): def __init__(self, products): super().__init__() self._products = set(products)

class FakeTrackingRepository(AbstractTrackingRepository): def __init__(self, products): super().__init__() self._products = set(products) def _add(self, product): self._products.add(product) def _get(self, sku): return next(( p for p in self._products if p.sku == sku), None)

class FakeTrackingRepository(AbstractTrackingRepository): def __init__(self, products): super().__init__() self._products = set(products) def _add(self, product): self._products.add(product) def _get(self, sku): return next(( p for p in self._products if p.sku == sku), None)

No content

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set() def add_product(self, product): self._add(product) self.seen.add(product) def get_by_sku(self, sku): if product := self._get(sku): self.seen.add(product) return product @abc.abstractmethod def _add(self, product): raise NotImplementedError @abc.abstractmethod def _get(self, sku): raise NotImplementedError

class AbstractTrackingRepository(abc.ABC): def __init__(self): self.seen = set() def add_product(self, product): self._add(product) self.seen.add(product) def get_by_sku(self, sku): if product := self._get(sku): self.seen.add(product) return product @abc.abstractmethod def _add(self, product): raise NotImplementedError @abc.abstractmethod def _get(self, sku): raise NotImplementedError

class AbstractRepository(abc.ABC): @abc.abstractmethod def add(self, product: Product) -> None: ... @abc.abstractmethod def get(self, sku: str) -> Product | None: ...

class AbstractRepository(abc.ABC): @abc.abstractmethod def add(self, product: Product) -> None: ... @abc.abstractmethod def get(self, sku: str) -> Product | None: ... Nominal subtyping

class AbstractRepository(abc.ABC): @abc.abstractmethod def add(self, product: Product) -> None: ... @abc.abstractmethod def get(self, sku: str) -> Product | None: ... Nominal subtyping (metaclass=abc.ABCMeta):

@typing.runtime_checkable class Repository(typing.Protocol): def add(self, product: Product) -> None: ... def get(self, sku: str) -> Product | None: ...

@typing.runtime_checkable class Repository(typing.Protocol): def add(self, product: Product) -> None: ... def get(self, sku: str) -> Product | None: ...

@typing.runtime_checkable class Repository(typing.Protocol): def add(self, product: Product) -> None: ... def get(self, sku: str) -> Product | None: ... Structural subtyping

No content

class FakeRepository: def __init__(self, products): self._products = set(products)

class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product)

class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product)

class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product) def get(self, sku: str) -> Product | None: return next(( p for p in self._products if p.sku == sku), None)

class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product) def get(self, sku: str) -> Product | None: return next(( p for p in self._products if p.sku == sku), None)

class FakeRepository(AbstractRepository): def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product) def get(self, sku: str) -> Product | None: return next(( p for p in self._products if p.sku == sku), None)

class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product) def get(self, sku: str) -> Product | None: return next(( p for p in self._products if p.sku == sku), None) AbstractRepository.register(FakeRepository)

@AbstractRepository.register class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product: Product ) -> None: self._products.add(product) def get(self, sku: str) -> Product | None: return next(( p for p in self._products if p.sku == sku), None)

No content

@dataclass class TrackingRepository: _repo: Repository seen: set[Product] = field(default_factory=set)

@dataclass class TrackingRepository: _repo: Repository seen: set[Product] = field(default_factory=set) def add_product(self, product: Product) -> None: self._repo.add(product) self.seen.add(product)

@dataclass class TrackingRepository: _repo: Repository seen: set[Product] = field(default_factory=set) def add_product(self, product: Product) -> None: self._repo.add(product) self.seen.add(product) def get_by_sku(self, sku: str) -> Product | None: if product := self._repo.get(sku): self.seen.add(product) return product

@dataclass class TrackingRepository: _repo: Repository seen: set[Product] = field(default_factory=set) def add_product(self, product): self._repo.add(product) self.seen.add(product) def get_by_sku(self, sku): if product := self._repo.get(sku): self.seen.add(product) return product class FakeRepository: def __init__(self, products): self._products = set(products) def add(self, product): self._products.add(product) def get(self, sku): return next( ( p for p in self._products if p.sku == sku ), None )

Strategy Pattern

Testing

Subclassing requires knowledge + self-discipline

Composition mechanically enforces discipline Subclassing requires knowledge + self-discipline

Muddling of Namespaces

Namespaces are one honking great idea – let’s do more of those! — Uncle Timmy, Zen of Python, PEP 20

Namespace Pollution

No content

def decorator(cls): cls.a = 42 return cls

def decorator(cls): cls.a = 42 return cls @decorator class C: pass

def decorator(cls): cls.a = 42 return cls @decorator class C: pass assert hasattr(C, "a")

def decorator(cls): cls.a = 42 return cls @decorator class C: pass assert hasattr(C, "a") == C = decorator(_C)

No content

class Question(django.db.models.Model): question_text = CharField(max_length=200) pub_date = DateTimeField('date published')

class Question(django.db.models.Model): question_text = CharField(max_length=200) pub_date = DateTimeField('date published') @dataclass class Question: question_text: str pub_date: datetime @attrs.define

class Question(django.db.models.Model): question_text = CharField(max_length=200) pub_date = DateTimeField('date published') @dataclass class Question: question_text: str pub_date: datetime

['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__match_args__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'pub_date', 'question_text'] dir(Question("Hi!", today())) – dataclass Edition

['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__match_args__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'pub_date', 'question_text'] dir(Question("Hi!", today())) – dataclass Edition 32

['DoesNotExist', 'MultipleObjectsReturned', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_check_column_name_clashes', '_check_constraints', '_check_default_pk', '_check_field_name_clashes', '_check_fields', '_check_id_field', '_check_index_together', '_check_indexes', '_check_local_fields', '_check_long_column_names', '_check_m2m_through_same_relationship', '_check_managers', '_check_model', '_check_model_name_db_lookup_clashes', '_check_ordering', '_check_property_name_related_field_accessor_clashes', '_check_single_primary_key', '_check_swappable', '_check_unique_together', '_do_insert', '_do_update', '_get_FIELD_display', '_get_expr_references', '_get_field_value_map', '_get_next_or_previous_by_FIELD', '_get_next_or_previous_in_order', '_get_pk_val', '_get_unique_checks', '_meta', '_perform_date_checks', '_perform_unique_checks', '_prepare_related_fields_for_save', '_save_parents', '_save_table', '_set_pk_val', '_state', 'check', 'choice_set', 'clean', 'clean_fields', 'date_error_message', 'delete', 'from_db', 'full_clean', 'get_constraints', 'get_deferred_fields', 'get_next_by_pub_date', 'get_previous_by_pub_date', 'id', 'objects', 'pk', 'prepare_database_save', 'pub_date', 'question_text', 'refresh_from_db', 'save', 'save_base', 'serializable_value', 'unique_error_message', 'validate_constraints', 'validate_unique'] dir(Question("Hi!", today())) – ORM Edition

['DoesNotExist', 'MultipleObjectsReturned', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_check_column_name_clashes', '_check_constraints', '_check_default_pk', '_check_field_name_clashes', '_check_fields', '_check_id_field', '_check_index_together', '_check_indexes', '_check_local_fields', '_check_long_column_names', '_check_m2m_through_same_relationship', '_check_managers', '_check_model', '_check_model_name_db_lookup_clashes', '_check_ordering', '_check_property_name_related_field_accessor_clashes', '_check_single_primary_key', '_check_swappable', '_check_unique_together', '_do_insert', '_do_update', '_get_FIELD_display', '_get_expr_references', '_get_field_value_map', '_get_next_or_previous_by_FIELD', '_get_next_or_previous_in_order', '_get_pk_val', '_get_unique_checks', '_meta', '_perform_date_checks', '_perform_unique_checks', '_prepare_related_fields_for_save', '_save_parents', '_save_table', '_set_pk_val', '_state', 'check', 'choice_set', 'clean', 'clean_fields', 'date_error_message', 'delete', 'from_db', 'full_clean', 'get_constraints', 'get_deferred_fields', 'get_next_by_pub_date', 'get_previous_by_pub_date', 'id', 'objects', 'pk', 'prepare_database_save', 'pub_date', 'question_text', 'refresh_from_db', 'save', 'save_base', 'serializable_value', 'unique_error_message', 'validate_constraints', 'validate_unique'] dir(Question("Hi!", today())) – ORM Edition 91

class QuestionsRepository: conn: sqlalchemy.engine.Connection def get(self, id: int) -> Question: row := self.conn.execute( select( tables.questions.c.pub_date ).where( tables.questions.c.id == id ) ).one() return Question(id=id, pub_date=row.pub_date)

class QuestionsRepository: conn: sqlalchemy.engine.Connection def get(self, id: int) -> Question: row := self.conn.execute( select( tables.questions.c.pub_date ).where( tables.questions.c.id == id ) ).one() return Question(id=id, pub_date=row.pub_date)

class QuestionsRepository: conn: sqlalchemy.engine.Connection def get(self, id: int) -> Question: row := self.conn.execute( select( tables.questions.c.pub_date ).where( tables.questions.c.id == id ) ).one() return Question(id=id, pub_date=row.pub_date)

['DoesNotExist', 'MultipleObjectsReturned', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_check_column_name_clashes', '_check_constraints', '_check_default_pk', '_check_field_name_clashes', '_check_fields', '_check_id_field', '_check_index_together', '_check_indexes', '_check_local_fields', '_check_long_column_names', '_check_m2m_through_same_relationship', '_check_managers', '_check_model', '_check_model_name_db_lookup_clashes', '_check_ordering', '_check_property_name_related_field_accessor_clashes', '_check_single_primary_key', '_check_swappable', '_check_unique_together', '_do_insert', '_do_update', '_get_FIELD_display', '_get_expr_references', '_get_field_value_map', '_get_next_or_previous_by_FIELD', '_get_next_or_previous_in_order', '_get_pk_val', '_get_unique_checks', '_meta', '_perform_date_checks', '_perform_unique_checks', '_prepare_related_fields_for_save', '_save_parents', '_save_table', '_set_pk_val', '_state', 'check', 'choice_set', 'clean', 'clean_fields', 'date_error_message', 'delete', 'from_db', 'full_clean', 'get_constraints', 'get_deferred_fields', 'get_next_by_pub_date', 'get_previous_by_pub_date', 'id', 'objects', 'pk', 'prepare_database_save', 'pub_date', 'question_text', 'refresh_from_db', 'save', 'save_base', 'serializable_value', 'unique_error_message', 'validate_constraints', 'validate_unique']

No content

Readability

Readability Control

Readability Convenience Control

Readability Convenience Control Performance

No content

Seth Michael Larson Working on urllib3 full-time for one week

Programs are meant to be read by humans and only incidentally for computers to execute. — Abelson & Sussman, SICP Seth Michael Larson Working on urllib3 full-time for one week

No content

No content

Functions

? class A: b: B def other_method(): ... def method(self): self.other_method() self.b.even_other_method() class B: a: A def even_other_method(): ... def method(self): self.a.other_method() self.even_other_method() or

class A: b: B def other_method(): ... def method(self): self.other_method() self.b.even_other_method() class B: a: A def even_other_method(): ... def method(self): self.a.other_method() self.even_other_method() or def function(a, b): a.other_method() b.even_other_method()

class C: a: A b: B def method(self): self.a.other_method() self.b.even_other_method()

Why anyway!?

Metaclasses

Exceptions

class MyError(Exception): pass Exceptions

class MyError(Exception): pass class MyError(ValueError, KeyError): pass Exceptions

class MyError(Exception): pass class MyError(ValueError, KeyError): pass Exceptions

Specialization

E-Mail Addresses

class Mailbox: id: UUID addr: str pwd: str E-Mail Addresses

class Mailbox: id: UUID addr: str pwd: str class Forwarder: id: UUID addr: str targets: list[str] E-Mail Addresses

class Mailbox: id: UUID addr: str pwd: str class Forwarder: id: UUID addr: str targets: list[str] E-Mail Addresses EmailAddr = Forwarder | Mailbox

class Mailbox: id: UUID addr: str pwd: str class Forwarder: id: UUID addr: str targets: list[str] E-Mail Addresses EmailAddr = Forwarder | Mailbox

Duplication is far cheaper than a wrong abstraction. — Sandi Metz

Create one class, make fields optional

class AddrType(enum.Enum): MAILBOX = "mailbox" FORWARDER = "forwarder" class EmailAddr: type: AddrType id: UUID addr: str Create one class, make fields optional

class AddrType(enum.Enum): MAILBOX = "mailbox" FORWARDER = "forwarder" class EmailAddr: type: AddrType id: UUID addr: str # Only useful if type == AddrType.MAILBOX pwd: str | None # Only useful if type == AddrType.FORWARDER target: list[str] | None Create one class, make fields optional

class AddrType(enum.Enum): MAILBOX = "mailbox" FORWARDER = "forwarder" class EmailAddr: type: AddrType id: UUID addr: str # Only useful if type == AddrType.MAILBOX pwd: str | None # Only useful if type == AddrType.FORWARDER target: list[str] | None Create one class, make fields optional

Composition

class EmailAddr: id: UUID addr: str Composition

class EmailAddr: id: UUID addr: str class Mailbox: email: EmailAddr pwd: str Composition

class EmailAddr: id: UUID addr: str class Mailbox: email: EmailAddr pwd: str class Forwarder: email: EmailAddr targets: list[str] Composition

class EmailAddr: id: UUID addr: str class Mailbox: email: EmailAddr pwd: str class Forwarder: email: EmailAddr targets: list[str] Composition

class EmailAddr: id: UUID addr: str class Mailbox: email: EmailAddr pwd: str class Forwarder: email: EmailAddr targets: list[str] Composition

Create a Common Base Class, Then Specialize

class EmailAddr: id: UUID addr: str Create a Common Base Class, Then Specialize

class EmailAddr: id: UUID addr: str class Mailbox(EmailAddr): pwd: str class Forwarder(EmailAddr): targets: list[str] Create a Common Base Class, Then Specialize

class EmailAddr: id: UUID addr: str class Mailbox(EmailAddr): pwd: str class Forwarder(EmailAddr): targets: list[str] Create a Common Base Class, Then Specialize

No content

type EmailAddr struct { addr string } type Mailbox struct { EmailAddr pwd string }

type EmailAddr struct { addr string } type Mailbox struct { EmailAddr pwd string } func main() { mbox := Mailbox{EmailAddr{"addr@example.com"}, "a hash"} fmt.Println(mbox.addr) }

type EmailAddr struct { addr string } type Mailbox struct { EmailAddr pwd string } func main() { mbox := Mailbox{EmailAddr{"addr@example.com"}, "a hash"} fmt.Println(mbox.addr) }

No content

No content

No content

No content

I never allow myself to have an opinion on anything that I don’t know the other side’s argument better than they do. — Charlie Munger

No content

OX.CX/SC

OX.CX/SC @HYNEK (@MASTODON.SOCIAL)

OX.CX/SC @HYNEK (@MASTODON.SOCIAL) VRMD.DE