Slide 1

Slide 1 text

Getting Started with Statically Typed Programming in Python 3.10 Peacock (Yoichi Takai), at PyCon US 2022 (2022/05/01)

Slide 2

Slide 2 text

Prolog Self-Introduction, Table of Contents

Slide 3

Slide 3 text

Slides: slides.peacock0803sz.com/pyconus2022/index.html Expected: Google Chrome Feel free to Take pictures or Screenshots and Tweet Hashtag: #pyconus2022 3 / 57 ` ` ^ Slides ^

Slide 4

Slide 4 text

Self-Introduction Name: Peacock / Yoichi Takai Social media IDs: peacock0803sz Twitter, GitHub, Facebook… Please call me Peacock 4 / 57 ` `

Slide 5

Slide 5 text

Hobbies & Favourites Playing the Clarinet Listening Music (Most is classical) Skiing , Gadgets… 21-years old, This is my first trip abroad 5 / 57

Slide 6

Slide 6 text

My Works and Communities Company: CMScom (Full-time since 2020/06 ~) Web application developer, Flask / Pyramid / Plone etc… Experience learning Haskell and TypeScript Now, reading "Types and Programming Languages" As known as "TaPL" Member of PloneJP (Plone User’s Group in Japan) 6 / 57

Slide 7

Slide 7 text

My Activities Related to PyCon JP Staff of PyCon JP (since 2020 - ), 2022: Vice-Chair Operating Member of PyCon JP Association Director of PyCon JP TV It’s the YouTube live about PyCons and local events held once a month 7 / 57

Slide 8

Slide 8 text

Today’s Topics 1. Why do I talk about typing? 2. First step of typing, How to write basically That is what I most want to say 3. Generics, User-Defined types (Best practice included) 4. Updates overview recently & Backward compatibility for 3.9 or before 5. Overview of new features on 3.10 8 / 57

Slide 9

Slide 9 text

Why do I talk about Typing? It’s been 7 years since typing appeared In Python 3.5, at 2015 Many big PEPs were adopted and updated over the years Even now, I think many people don’t know where to start Because there is little coherent information 9 / 57

Slide 10

Slide 10 text

What I will NOT talk about Developing library with typing Configures and options of mypy How to use them in CI, ex: GitHub actions, Circle CI and etc… History of type hinting Implementation of typing, mypy Abstract Syntax Tree (AST) 10 / 57

Slide 11

Slide 11 text

11 / 57 First step of Typing Basic Typing

Slide 12

Slide 12 text

What makes you happy? It knows the type when you reference it in the editor It gets angry when I try to give it the wrong one The completion will work when accessing the return value of a function using a dot. 12 / 57

Slide 13

Slide 13 text

Without the Typing We don’t know the error… 13 / 57

Slide 14

Slide 14 text

With the Typing and, the editor can tell the argument is wrong Editor tells a wrong argument 14 / 57

Slide 15

Slide 15 text

In a code review, without Typing… Boss < What type does this function return? You < Humm… str or False or None …? Boss < THAT’S TOO MANY TYPES! You < :-( 4 else: return post_id # this is str 15 / 57 1 def need_new_post(): 2 if ...: retrun None 3 elif ...: retrun False

Slide 16

Slide 16 text

With Typing… Boss < It looks like this function may return 3 types… Isn’t that too much? You < I see. That could be a bad design. Let me fix it. Boss < Sure, please. 1 def need_new_post() -> None | False | str: 16 / 57 2 if ...: retrun None 3 elif ...: retrun False 4 else: return post_id # this is str

Slide 17

Slide 17 text

Let’s start with function definitions Nothing too difficult! Just write type After the arguments, write colon and the type Before the colon at the end of the function definition, write an arrow and the type 17 / 57

Slide 18

Slide 18 text

Using built-in types bool , bytes , float , int , str You don’t need to do anything to use them. None : Used for functions that return nothing. 18 / 57 ` ` ` ` ` ` ` ` ` ` ` `

Slide 19

Slide 19 text

Escaping from complex type puzzles Any Can hold instances of any type. It’s better not to use it Import and use from typing when necessary. 1 from typing import Any 2 very_dangerous_last_resort: Any 19 / 57 ` ` ` `

Slide 20

Slide 20 text

(Since 3.9) Generics in standard Collections Before 3.9, from typing import Dict, List … Now, it’s depreciated Use dict , frozenset , list , set , tuple Collections can be written with [] for the type inside. ref: Official Documentation 20 / 57 ` ` ` ` ` ` ` ` ` ` ` ` ` `

Slide 21

Slide 21 text

Module Availabilities in standard library… collections : (ex: deque, defaultdict) collections.abc : Iterable , Callable , or other Protocol-Related re : Regular Expressions contextlib : Context-Related items 21 / 57 ` ` ` ` ` ` ` ` ` ` ` `

Slide 22

Slide 22 text

Using types from collections.abc differently There are many types in collections.abc . It’s better to use a collection with a few methods to increase portability. Ex. Iterable is better than list . It’s a good idea to look at the methods used in your functions. The figure on the next page shows the relationship… 22 / 57 ` ` ` ` ` ` ` `

Slide 23

Slide 23 text

Great method inheritance tree 23 / 57

Slide 24

Slide 24 text

The differences between tuple and other Sequences tuple is fixed up to the length of information Specify the type for the number of elements Or you can mix types, such as tuple[int, str, float] . 24 / 57 ` ` ` `

Slide 25

Slide 25 text

A sequence, such as a list , has the same constraint for all elements Can be used regardless of the length of the Sequence by setting only one. 25 / 57 ` `

Slide 26

Slide 26 text

26 / 57 Advanced: Generic Types

Slide 27

Slide 27 text

Union (multiple types) union can be represented by | since 3.10 1 def square(number: int | float) -> int | float: 7 # Compatible with typing.Union 8 int | str == typing.Union[int, str] 27 / 57 ` ` ` ` 2 return number ** 2 1 # Nested unions are flattened 2 (int | str) | float == int | str | float 3 # Redundant types are removed 4 int | str | int == int | str 5 # Orders are ignored 6 int | str == str | int

Slide 28

Slide 28 text

Optional type Shorthand, Optional[T] is equivalent to Union with None. Behaves just like Union: T | None If you use it in a function return value or something, it will propagate, so be careful when you use it. 3 age: Optional[int] 4 age = 17 5 age = None # This is also valid 28 / 57 ` ` ` ` 1 from typing import Optional 2

Slide 29

Slide 29 text

Avoid using Optional as much as possible 6 return None When you use the function, you might write another guard and return None . 29 / 57 1 def get_content() -> str | None: 2 r = request.get("https://example.com") 3 if r.status_code != 200: 4 # Guard section (early return) 5 logging.warning("Response: %d", r.status_code) 7 return r.text ` `

Slide 30

Slide 30 text

In this case… It would be cleaner to raise a raise RuntimeError . The cost of raising exceptions in Python is (relatively) low Little performance impact, unlike Java. The lack of null-safe methods in Python is also a factor, Unlike JavaScript. But, if there were such methods, they would be abused. 30 / 57 ` `

Slide 31

Slide 31 text

Callable (callable objects) 13 return func(*args, **kw) 31 / 57 1 from collections.abc import Callable # since 3.9 2 from functools import wraps 3 4 def validate(func: Callable 5 ) -> Callable[..., Callable]: 6 @wraps(func) 7 def wrapper(*args, **kw) -> Callable: 8 try: 9 j = request.json 10 if j is None: raise BadRequest 11 except BadRequest: 12 return abort(400) 14 return wrapper

Slide 32

Slide 32 text

User-defined Generic types Example: a generic mapping type 13 except KeyError: return default 32 / 57 1 from typing import TypeVar, Generic 2 KT, VT = TypeVar("KT"), TypeVar("VT") 3 4 class Mapping(Generic[KT, VT]): 5 def __getitem__(self, key: KT) -> VT: pass 6 7 X, Y = TypeVar("X"), TypeVar("Y") 8 9 def lookup_name( # Using Mapping 10 mapping: Mapping[X, Y], key: X, default: Y 11 ) -> Y: 12 try: return mapping[key]

Slide 33

Slide 33 text

33 / 57 How to use new features in older versions

Slide 34

Slide 34 text

What is the __future__ module? Using typing new feature in the older versions, write from __future__ import annotations In addition to typing, it was also used to call 3.x features in 2.x. ex) print_func , unicode_literals etc … ref: __future__, future statement 34 / 57 ` ` ` ` ` ` ` `

Slide 35

Slide 35 text

35 / 57 Recent Update

Slide 36

Slide 36 text

Recent Python updates Ver. Status Release EoS Schedule Main new feature 3.11 Alpha 7 2022-10 2027-10 PEP 619 Performance tuning 3.10 Bugfix 2021-10 2026-10 PEP 619 Pattern Matching 3.9 Bugfix 2020-10 2025-10 PEP 596 Union Operators 3.8 Security 2019-10 2024-10 PEP 569 = in f-string 3.7 Security 2018-06 2023-06 PEP 537 dataclasses 2.7 EoL 2010-07 2020-01 PEP 373 Improving Decimals https://www.python.org/downloads/ 36 / 57 ` `

Slide 37

Slide 37 text

New Features on Type Hints in 3.10 PEP 604: New Type Union Operator PEP 612: Parameter Specification Variables PEP 613: TypeAlias PEP 647: User-Defined Type Guards See Also: https://docs.python.org/3/whatsnew/3.10.html#new- features-related-to-type-hints 37 / 57

Slide 38

Slide 38 text

PEP 604: New Type Union Operator | The union above type can be used as an operator. You can also use it when asking isinstance() . 1 int | str == typing.Union[int, str] 2 # Compatible with typing.Union Optional[T] can also be written as T | None using this. 38 / 57 ` ` ` ` ` ` ` `

Slide 39

Slide 39 text

PEP 612: Parameter Specification Variables Motivation Needed a way to represent a function that has the same arguments as the specified function 1 from typing import Callable, TypeVar 2 R = TypeVar("R") 3 4 def add_logging(f: Callable[..., R] 5 ) -> Callable[..., R]: 6 def inner(*args: object, **kw: object) -> R: 7 log_to_database() 8 return f(*args, **kwargs) 9 return inner 39 / 57

Slide 40

Slide 40 text

1 @add_logging 2 def takes_int_str(x: int, y: str) -> int: 3 return x + 7 4 5 await takes_int_str(1, "A") 6 await takes_int_str("B", 2) # fails at runtime Approach Adding an argument type called ParameterSpecification solves the problem. It can be used with Callable to behave like a generic callable object. 40 / 57 ` ` ` `

Slide 41

Slide 41 text

Before 11 12 @add_logging 13 def foo(x: int, y: str) -> int: 14 return x + 7 41 / 57 1 from typing import Callable, TypeVar 2 Ps, R = TypeVar("Ps"), TypeVar("R") 3 4 def add_logging(f: Callable[Ps, R] 5 ) -> Callable[Ps, R]: 6 # args: tuple...?, kwargs: dict...? 7 def inner(*args, **kwargs) -> R: 8 log_to_database(*args, **kwargs) 9 return f(*args, **kwargs) 10 return inner

Slide 42

Slide 42 text

After 10 11 @add_logging 12 def foo(x: int, y: str) -> int: return x + 7 42 / 57 1 from typing import Callable, ParameterSpecification, T 2 Ps, R = ParameterSpecification("Ps"), TypeVar("R") 3 4 def add_logging(f: Callable[Ps, R] 5 ) -> Callable[Ps, R]: 6 def inner(*args: Ps.args, **kwargs: Ps.kwargs) -> R: 7 log_to_database(*args, **kwargs) 8 return f(*args, **kwargs) 9 return inner

Slide 43

Slide 43 text

PEP 613: TypeAlias Motivation We consider global variables without type hints to be type aliases. This tends to cause problems with forwarding references, scoping, etc. So, we’re going to make it possible to explicitly define type aliases. 43 / 57

Slide 44

Slide 44 text

Approach Add a new typing.TypeAlias You can write like this T: TypeAlias = int You can write like: T: TypeAlias = "int" Using ForwardReference 44 / 57 ` ` ` ` ` `

Slide 45

Slide 45 text

Before & After 7 x: TypeAlias = int # type alias 8 x: TypeAlias = “MyClass” # type alias 45 / 57 1 x = 1 # untyped global expression 2 x: int = 1 # typed global expression 3 4 x = int # untyped global expression 5 x: Type[int] = int # typed global expression 6

Slide 46

Slide 46 text

PEP 647: User-Defined Type Guards Motivation Type checker tools use a technique called type narrowing to determine the type of information. 5 else: pass # Type of val is narrowed to None 46 / 57 1 def func(val: Optional[str]): 2 # "is None" type guard 3 if val is not None: 4 pass # Type of val is narrowed to str

Slide 47

Slide 47 text

However, that will not work as intended if the user function is used. 6 if is_str_list(val): print(" ".join(val)) TypeGuard allows you to define user-defined type guards via the new typing. 47 / 57 1 def is_str_list(val: List[object]) -> bool: 2 """Determines whether all objects in the list are st 3 return all(isinstance(x, str) for x in val) 4 5 def func1(val: List[object]): 7 # => Error: invalid type ` `

Slide 48

Slide 48 text

1 from typing import TypeGuard 2 3 def is_str_list(val: List[object] 4 ) -> TypeGuard[List[str]]: 5 return all(isinstance(x, str) for x in val) And, type narrowing works like next: 48 / 57

Slide 49

Slide 49 text

10 else: reveal_type(val) # OneOrTwoStrs 49 / 57 1 def is_two_element_tuple( 2 val: Tuple[str, ...] 3 ) -> TypeGuard[Tuple[str, str]]: 4 return len(val) == 2 5 6 OneOrTwoStrs = Union[Tuple[str], Tuple[str, str]] 7 def func(val: OneOrTwoStrs): 8 if is_two_element_tuple(val): 9 reveal_type(val) # Tuple[str, str]

Slide 50

Slide 50 text

New Feature on Type Hints in 3.11 Python 3.11 is focused on performance tuning, but there are also new features in Typing… PEP 673: Self Type, It’s a way to annotate methods that return an instance of their class 50 / 57

Slide 51

Slide 51 text

Motivation Calling set_scale on a subclass of Shape , the type checker still infers the return type to be Shape . 7 Circle().set_scale(0.5).set_radius(2.7) 8 # => error: Shape has no attribute "set_radius" 51 / 57 ` ` ` ` ` ` 1 class Circle(Shape): 2 def set_radius(self, r: float) -> Circle: 3 self.radius = r 4 return self 5 6 Circle().set_scale(0.5) # *Shape*, not Circle

Slide 52

Slide 52 text

Workaround, but it’s unintuitive… 15 Circle().set_scale(0.5).set_radius(2.7) # => Circle 52 / 57 1 from typing import TypeVar 2 TShape = TypeVar("TShape", bound="Shape") 3 4 class Shape: 5 def set_scale(self: TShape, scale: float 6 ) -> TShape: 7 self.scale = scale 8 return self 9 10 class Circle(Shape): 11 def set_radius(self, radius: float) -> Circle: 12 self.radius = radius 13 return self 14

Slide 53

Slide 53 text

typing.Self resolves that unintuitively 12 Circle().set_scale(0.5) # => It would be *Circle*! 53 / 57 ` ` 1 from typing import Self 2 class Shape: 3 def set_scale(self, scale: float) -> Self: 4 self.scale = scale 5 return self 6 7 class Circle(Shape): 8 def set_radius(self, radius: float) -> Self: 9 self.radius = radius 10 return self 11

Slide 54

Slide 54 text

Summary 1. Benefits, starting with def , Built-in types 2. Standard collections (since 3.9) 3. Collections, Union, Optional, Callable, User-defined Generics 4. Updates Overview, __future__ for compatibility 54 / 57 ` ` ` `

Slide 55

Slide 55 text

5. New features on Python 3.10 and 3.11 1. 3.10: New Type Union Operator, Parameter Specific Variables, TypeAlias, User-Defined Type Guards 2. 3.11: Self Type 55 / 57

Slide 56

Slide 56 text

Pages I used for reference (Thanks) There are links that I referenced. Thank you! https://docs.python.org/3/library/typing.html https://docs.python.org/3.10/whatsnew/3.10.html https://docs.python.org/3.11/whatsnew/3.11.html https://future-architect.github.io/articles/20201223 (ja) https://qiita.com/tk0miya/items/931da13af292060356b9 (ja) https://qiita.com/tk0miya/items/1b093c2eee7668fffb62 (ja) https://qiita.com/tk0miya/items/a27191f5d627601930ed (ja) 56 / 57

Slide 57

Slide 57 text

Thank you See you again next time in Japan! Powered by Slidev, using theme-briks