Types: what they are, why to care, and how to use them in Python

Transcript

1. Types What they are, why to care, and how to use them in Python Oliver Ford 10 June, 2018 Arm - Mbed Cloud
2. Table of contents 1. Background 2. Motivation 3. Python annotations 1
3. Background
4. What is a type anyway • Abstraction for a bunch of possible values • “integer” • Tool for reasoning about program correctness • Follows syntactic structure of terms: • if λx.M is a function; • and x : A (read: ‘x has type A’), M : B; • then (λx.M) : (A → B) Background 2
5. Motivation
6. Isn’t this against everything Python stands for? Zen of Python: • Explicit is better than implicit • Readability counts • In the face of ambiguity, refuse the temptation to guess Motivation 3
7. But if it smells like a duck I don’t care what type it is • Oh, but you do! • ‘Duck typing’ is an abstraction for a bunch of possible values • Typing is not at all incompatible Motivation 4
8. Some (real!) code • How do I call this? • def save_event(event): • The docstring: ":param event: The event to save." doesn’t help much! • What does this return? • def deployed_devices(self): • Probably some sort of collection of Devices that are ’deployed’, right? • Nope - the integer count of those • Or this? • def get_firmware_manifest(...): • ":return: manifest contents or None (if none)." • OK, but what is the contents? bytes? dict of... ‘stuff’? - reads the implementation - oh, it’s str • What’s in here? • before_listen: dict Motivation 5
9. Python annotations
10. Basic syntax • identifier: type = value • identifier: CustomisableType[type] = value • def function(identifier: type) -> type: ... • def function(identifier: type = value) -> type: ... Examples • password: str = 'hunter2' • def increment(num: int, by: int = 1) -> int: ... Note These are syntactic examples - 'hunter2' can be inferred to be str, we don’t need to annotate it. Python annotations 6
11. Sets & Tuples & Lists • identifier: Set[type] = value • identifier: Tuple[type, type] = value • identifier: Tuple[type, ...] = value • identifier: List[type] = value Examples • pythons: Set[str] = {'cleese', 'idle', 'palin'} • talk: Tuple[str, str] = ('types', 'pylondinium') • initials: Tuple[str, ...] = ('o', 'j', 'f') • talks: List[Tuple[str, str]] = [('types', 'pylondinium')] Python annotations 7
12. Dictionaries • Two paramaters now: key type; value type • identifier: Dict[type, type] = value Examples • age_people: Dict[int, List[str]] = {123: ['fred']} • is_quite_old: Dict[str, bool] = {'fred': True} • def people_ages( age_people: Dict[int, str] = age_people, ) -> Dict[str, int]: return { person: age for age, people in age_people.items() for person in people } • def print_keys(d: Dict): ... Python annotations 8
13. Sequences & Iterables Observation We don’t always care about the precise container type - especially in functions: def contains_nuts(wrapper) -> bool: return 'nuts' in wrapper • Iterable[type] when we just want to iter through, like above; • Sequence[type] when we want some order, no sets! Note Be flexible in arguments; precise in return types: def sur_initials( peoples_initials: Iterable[Sequence[str]], ) -> Set[str]: return {initials[-1] for initials in peoples_initials} Python annotations 9
14. None & Optionals • Not all functions return values • Remember def print_keys(d: Dict)? • We can use def identifier() -> None • Optional[type] indicates a value maybe None Examples • def print_keys(d: Dict) -> None • def __init__(self) -> None • def print_keys(d: Dict, colour: Optional[str]) Python annotations 10
15. Unions • Optional’s great if a value might be None, but what if it might be something else? • identifier: Union[type, type] = value Examples int_or_str: Union[int, str] = ( 1 if random() > 0.01 else 'stupid, evil example' ) Python annotations 11
16. Variadic arguments • *args: type =⇒ args: Tuple[type, ...] • **kwargs: type =⇒ kwargs: Dict[str, type] Examples • def cat(*strs: str) -> str: return ''.join(args) • def set_feature_flags(**features: bool) Python annotations 12
17. Callables • Two parameters again: argument types; return type • identifier: Callable[[type], type] = value • identifier: Callable = value Examples • f: Callable = lambda: None • def map_int(f: Callable[[int], int], Iterable[Int]) Python annotations 13
18. Generators & Iterators • Generator parameters: yield type, send type, return type • AsyncGenerator parameters: yield type, send type • Iterator parameters: yield type • Most often we have only Iterators, i.e. Generator[yield_type, None, None]s Examples • ints: Iterator[int] = (i for i in range(100)) • chunks: Iterator = requests.get( url, stream=True, ).iter_content() Python annotations 14
19. Classes class Foo(object): pass foo1: Foo = Foo() # because issubclass(Foo, object) foo2: object = Foo() cls1: Type[Foo] = Foo().__class__ # = Foo cls2: Type[object] = Foo().__class__ # = Foo # because isinstance(Foo, type) inherited from object cls3: type = Foo().__class__ # = Foo Python annotations 15
20. Metaclasses class BarMeta(type): pass class Bar(object, metaclass=BarMeta): pass bar1: Bar = Bar() bar2: object = Bar() cls1: Type[Bar] = Bar().__class__ # = Bar cls2: Type[object] = Bar().__class__ # = Bar # because isinstance(Bar, BarMeta) cls3: BarMeta = Bar().__class__ # = Bar # because issubclass(BarMeta, type) cls4: type = Bar().__class__ # = Bar Python annotations 16
21. Better dictionaries & tuples • Having seen classes as types, we might think we can surely do better than Dict and Tuple... • NamedTuple • TypedDict # everything else presented is `from typing` from mypy_extensions import TypedDict class PersonAge(NamedTuple): name: str age: int class Message(TypedDict): body: Dict # or better, its own TypedDict! host: str message_id: int routing_key: Sequence[str] Python annotations 17
22. Custom types & aliases • Earlier: peoples_initials: Iterable[Sequence[str]] • ... it gets hard to read • Alias: Initials = Sequence[str], then Iterable[Initials] • New types give more safety: DeviceId = NewType('DeviceId', str) • ... now we can pass a DeviceId('1337deadbeef') rather than ambiguous str Note Not every type can be used in NewType - it must be subclassable. Python annotations 18
23. Type variables • Type variables stand in for a yet undetermined particular type at run-time - just as regular variables do for values • identifier = TypeVar(identifier_str) • identifier = TypeVar(identifier_str, type, type) • identifier = TypeVar(identifier_str, bound=type) Examples • T_any = TypeVar('T_any') • T_int_or_str = TypeVar('T_int_or_str', int, str) • T_int_like = TypeVar('T_int_like', bound=int) Note A type variable that’s one of n types is not the same as a Union of those types! ‘Why’ next... Python annotations 19
24. Generics • We say that the sum operation is generic over any type that it might reasonably be summing - viz. it doesn’t much care what the types are; it’ll call __add__ and return the same type • It’s not just that it admits a bunch of Union[int, float, str, etc] - it’s that whichever it’s given is also the type it returns! T_summable = TypeVar('T_summable', int, float, str, etc) def sum(*args: T_summable) -> Optional[T_summable]: if not args: return None return reduce(lambda acc, n: acc + n, args, 0) Python annotations 20
25. Supports • Sometimes we’re willing to admit anything with a certain ‘magic method’ • We have some built in SupportsMethod types to help Examples • def mode_score(*scores: SupportsInt) -> int: ... • def transmit(data: SupportsBytes) -> None: ... Python annotations 21
26. Protocols (in 3.7) • PEP 544 (3.7 track) introduces structural subtyping (aka ‘static duck typing’) through Protocols, analogous to Rust traits or Java interfaces • An ABC specifies a Protocol, which we then use as a type for values which implement it • It may be thought of as a more flexible Supports # Adapted from PEP 544 class Template(Protocol): name: str value: int = 0 class Concrete: # does not inherit - not a typo! def __init__(self, name: str, value: int) -> None: self.name = name self.value = value var: Template = Concrete('value', 42) Python annotations 22
27. Miscellaneous • Forward references: def __lt__(self, other: 'ClassName') -> bool: ... • cast(type, thing) if you need to • e.g. mypy can fail to understand isinstance checks, so you can help it out with a cast afterward • Any - use it sparingly • It is considered to have ‘all’ attributes. • It (almost always) defeats the point. • TYPE_CHECKING is True iff we’re, well, type-checking - it’s False at run-time • Sometimes an import for a type might cause a run-time-only problem; • If it’s not worth fixing, we can opt to import only when type-checking. Python annotations 23
28. Lots more I haven’t covered... • Covariance vs. contravariance vs. invariance • ContextManager • Coroutine • Awaitable • Generic • re (regular expressions) • ... • See https://docs.python.org/3/library/typing.html Python annotations 24
29. If this excites you... • For the Python: PEPs 48{2,3,4}; 544 • For the theory: Pierce’s Types and Programming Languages • For the win: Rust • :troll-face: Python annotations 25
30. Thank you!   @OJFord #PyLondinium18 Python annotations 25