Types What they are, why to care, and how to use them in Python Oliver Ford 10 June, 2018 Arm - Mbed Cloud 

Table of contents 1. Background 2. Motivation 3. Python annotations 1 

Background 

What is a type anyway • Abstraction for a bunch of possible values • “integer” • Tool for reasoning about program correctness • Follows syntactic structure of terms: • if λx.M is a function; • and x : A (read: ‘x has type A’), M : B; • then (λx.M) : (A → B) Background 2 

Motivation 

Isn’t this against everything Python stands for? Zen of Python: • Explicit is better than implicit • Readability counts • In the face of ambiguity, refuse the temptation to guess Motivation 3 

But if it smells like a duck I don’t care what type it is • Oh, but you do! • ‘Duck typing’ is an abstraction for a bunch of possible values • Typing is not at all incompatible Motivation 4 

Some (real!) code • How do I call this? • def save_event(event): • The docstring: ":param event: The event to save." doesn’t help much! • What does this return? • def deployed_devices(self): • Probably some sort of collection of Devices that are ’deployed’, right? • Nope - the integer count of those • Or this? • def get_firmware_manifest(...): • ":return: manifest contents or None (if none)." • OK, but what is the contents? bytes? dict of... ‘stuff’? - reads the implementation - oh, it’s str • What’s in here? • before_listen: dict Motivation 5 

Python annotations 

Basic syntax • identifier: type = value • identifier: CustomisableType[type] = value • def function(identifier: type) -> type: ... • def function(identifier: type = value) -> type: ... Examples • password: str = 'hunter2' • def increment(num: int, by: int = 1) -> int: ... Note These are syntactic examples - 'hunter2' can be inferred to be str, we don’t need to annotate it. Python annotations 6 

Sets & Tuples & Lists • identifier: Set[type] = value • identifier: Tuple[type, type] = value • identifier: Tuple[type, ...] = value • identifier: List[type] = value Examples • pythons: Set[str] = {'cleese', 'idle', 'palin'} • talk: Tuple[str, str] = ('types', 'pylondinium') • initials: Tuple[str, ...] = ('o', 'j', 'f') • talks: List[Tuple[str, str]] = [('types', 'pylondinium')] Python annotations 7 

Dictionaries • Two paramaters now: key type; value type • identifier: Dict[type, type] = value Examples • age_people: Dict[int, List[str]] = {123: ['fred']} • is_quite_old: Dict[str, bool] = {'fred': True} • def people_ages( age_people: Dict[int, str] = age_people, ) -> Dict[str, int]: return { person: age for age, people in age_people.items() for person in people } • def print_keys(d: Dict): ... Python annotations 8 

Sequences & Iterables Observation We don’t always care about the precise container type - especially in functions: def contains_nuts(wrapper) -> bool: return 'nuts' in wrapper • Iterable[type] when we just want to iter through, like above; • Sequence[type] when we want some order, no sets! Note Be flexible in arguments; precise in return types: def sur_initials( peoples_initials: Iterable[Sequence[str]], ) -> Set[str]: return {initials[-1] for initials in peoples_initials} Python annotations 9 

None & Optionals • Not all functions return values • Remember def print_keys(d: Dict)? • We can use def identifier() -> None • Optional[type] indicates a value maybe None Examples • def print_keys(d: Dict) -> None • def __init__(self) -> None • def print_keys(d: Dict, colour: Optional[str]) Python annotations 10 

Unions • Optional’s great if a value might be None, but what if it might be something else? • identifier: Union[type, type] = value Examples int_or_str: Union[int, str] = ( 1 if random() > 0.01 else 'stupid, evil example' ) Python annotations 11 

Variadic arguments • *args: type =⇒ args: Tuple[type, ...] • **kwargs: type =⇒ kwargs: Dict[str, type] Examples • def cat(*strs: str) -> str: return ''.join(args) • def set_feature_flags(**features: bool) Python annotations 12 

Callables • Two parameters again: argument types; return type • identifier: Callable[[type], type] = value • identifier: Callable = value Examples • f: Callable = lambda: None • def map_int(f: Callable[[int], int], Iterable[Int]) Python annotations 13 

Generators & Iterators • Generator parameters: yield type, send type, return type • AsyncGenerator parameters: yield type, send type • Iterator parameters: yield type • Most often we have only Iterators, i.e. Generator[yield_type, None, None]s Examples • ints: Iterator[int] = (i for i in range(100)) • chunks: Iterator = requests.get( url, stream=True, ).iter_content() Python annotations 14 

Classes class Foo(object): pass foo1: Foo = Foo() # because issubclass(Foo, object) foo2: object = Foo() cls1: Type[Foo] = Foo().__class__ # = Foo cls2: Type[object] = Foo().__class__ # = Foo # because isinstance(Foo, type) inherited from object cls3: type = Foo().__class__ # = Foo Python annotations 15 

Metaclasses class BarMeta(type): pass class Bar(object, metaclass=BarMeta): pass bar1: Bar = Bar() bar2: object = Bar() cls1: Type[Bar] = Bar().__class__ # = Bar cls2: Type[object] = Bar().__class__ # = Bar # because isinstance(Bar, BarMeta) cls3: BarMeta = Bar().__class__ # = Bar # because issubclass(BarMeta, type) cls4: type = Bar().__class__ # = Bar Python annotations 16 

Better dictionaries & tuples • Having seen classes as types, we might think we can surely do better than Dict and Tuple... • NamedTuple • TypedDict # everything else presented is `from typing` from mypy_extensions import TypedDict class PersonAge(NamedTuple): name: str age: int class Message(TypedDict): body: Dict # or better, its own TypedDict! host: str message_id: int routing_key: Sequence[str] Python annotations 17 

Custom types & aliases • Earlier: peoples_initials: Iterable[Sequence[str]] • ... it gets hard to read • Alias: Initials = Sequence[str], then Iterable[Initials] • New types give more safety: DeviceId = NewType('DeviceId', str) • ... now we can pass a DeviceId('1337deadbeef') rather than ambiguous str Note Not every type can be used in NewType - it must be subclassable. Python annotations 18 

Type variables • Type variables stand in for a yet undetermined particular type at run-time - just as regular variables do for values • identifier = TypeVar(identifier_str) • identifier = TypeVar(identifier_str, type, type) • identifier = TypeVar(identifier_str, bound=type) Examples • T_any = TypeVar('T_any') • T_int_or_str = TypeVar('T_int_or_str', int, str) • T_int_like = TypeVar('T_int_like', bound=int) Note A type variable that’s one of n types is not the same as a Union of those types! ‘Why’ next... Python annotations 19 

Generics • We say that the sum operation is generic over any type that it might reasonably be summing - viz. it doesn’t much care what the types are; it’ll call __add__ and return the same type • It’s not just that it admits a bunch of Union[int, float, str, etc] - it’s that whichever it’s given is also the type it returns! T_summable = TypeVar('T_summable', int, float, str, etc) def sum(*args: T_summable) -> Optional[T_summable]: if not args: return None return reduce(lambda acc, n: acc + n, args, 0) Python annotations 20 

Supports • Sometimes we’re willing to admit anything with a certain ‘magic method’ • We have some built in SupportsMethod types to help Examples • def mode_score(*scores: SupportsInt) -> int: ... • def transmit(data: SupportsBytes) -> None: ... Python annotations 21 

Protocols (in 3.7) • PEP 544 (3.7 track) introduces structural subtyping (aka ‘static duck typing’) through Protocols, analogous to Rust traits or Java interfaces • An ABC specifies a Protocol, which we then use as a type for values which implement it • It may be thought of as a more flexible Supports # Adapted from PEP 544 class Template(Protocol): name: str value: int = 0 class Concrete: # does not inherit - not a typo! def __init__(self, name: str, value: int) -> None: self.name = name self.value = value var: Template = Concrete('value', 42) Python annotations 22 

Miscellaneous • Forward references: def __lt__(self, other: 'ClassName') -> bool: ... • cast(type, thing) if you need to • e.g. mypy can fail to understand isinstance checks, so you can help it out with a cast afterward • Any - use it sparingly • It is considered to have ‘all’ attributes. • It (almost always) defeats the point. • TYPE_CHECKING is True iff we’re, well, type-checking - it’s False at run-time • Sometimes an import for a type might cause a run-time-only problem; • If it’s not worth fixing, we can opt to import only when type-checking. Python annotations 23 

Lots more I haven’t covered... • Covariance vs. contravariance vs. invariance • ContextManager • Coroutine • Awaitable • Generic • re (regular expressions) • ... • See https://docs.python.org/3/library/typing.html Python annotations 24 

If this excites you... • For the Python: PEPs 48{2,3,4}; 544 • For the theory: Pierce’s Types and Programming Languages • For the win: Rust • :troll-face: Python annotations 25 

Thank you!   @OJFord #PyLondinium18 Python annotations 25