Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with Statically Typed Programming in Python 3.10 / PyConUS2022

Getting Started with Statically Typed Programming in Python 3.10 / PyConUS2022

Peacock

May 01, 2022
Tweet

More Decks by Peacock

Other Decks in Programming

Transcript

  1. Getting Started
    with Statically Typed Programming
    in Python 3.10
    Peacock (Yoichi Takai),
    at PyCon US 2022 (2022/05/01)

    View full-size slide

  2. Prolog
    Self-Introduction,
    Table of Contents

    View full-size slide

  3. Slides: slides.peacock0803sz.com/pyconus2022/index.html
    Expected: Google Chrome
    Feel free to Take pictures
    or Screenshots and Tweet
    Hashtag: #pyconus2022
    3 / 57
    ` `
    ^ Slides ^

    View full-size slide

  4. Self-Introduction
    Name: Peacock / Yoichi Takai
    Social media IDs: peacock0803sz
    Twitter, GitHub, Facebook…
    Please call me Peacock
    4 / 57
    ` `

    View full-size slide

  5. Hobbies & Favourites
    Playing the Clarinet
    Listening Music (Most is classical)
    Skiing , Gadgets…
    21-years old, This is my first trip abroad
    5 / 57

    View full-size slide

  6. My Works and Communities
    Company: CMScom (Full-time since 2020/06 ~)
    Web application developer, Flask / Pyramid /
    Plone etc…
    Experience learning Haskell and TypeScript
    Now, reading "Types and Programming Languages"
    As known as "TaPL"
    Member of PloneJP (Plone User’s Group in Japan)
    6 / 57

    View full-size slide

  7. My Activities Related to PyCon JP
    Staff of PyCon JP (since 2020 - ), 2022: Vice-Chair
    Operating Member of PyCon JP Association
    Director of PyCon JP TV
    It’s the YouTube live about PyCons and local
    events held once a month
    7 / 57

    View full-size slide

  8. Today’s Topics
    1. Why do I talk about typing?
    2. First step of typing, How to write basically
    That is what I most want to say
    3. Generics, User-Defined types (Best practice included)
    4. Updates overview recently & Backward compatibility
    for 3.9 or before
    5. Overview of new features on 3.10
    8 / 57

    View full-size slide

  9. Why do I talk about Typing?
    It’s been 7 years since typing appeared
    In Python 3.5, at 2015
    Many big PEPs were adopted and updated over the
    years
    Even now, I think many people don’t know where to
    start
    Because there is little coherent information
    9 / 57

    View full-size slide

  10. What I will NOT talk about
    Developing library with typing
    Configures and options of mypy
    How to use them in CI, ex: GitHub actions, Circle CI
    and etc…
    History of type hinting
    Implementation of typing, mypy
    Abstract Syntax Tree (AST)
    10 / 57

    View full-size slide

  11. 11 / 57
    First step of Typing
    Basic Typing

    View full-size slide

  12. What makes you happy?
    It knows the type when you reference it in the editor
    It gets angry when I try to give it the wrong one
    The completion will work when accessing the return
    value of a function using a dot.
    12 / 57

    View full-size slide

  13. Without the Typing
    We don’t know the error…
    13 / 57

    View full-size slide

  14. With the Typing
    and, the editor can tell the argument is wrong
    Editor tells a wrong argument
    14 / 57

    View full-size slide

  15. In a code review, without Typing…
    Boss < What type does this function return?
    You < Humm… str or False or None …?
    Boss < THAT’S TOO MANY TYPES!
    You < :-(
    4 else: return post_id # this is str
    15 / 57
    1 def need_new_post():
    2 if ...: retrun None
    3 elif ...: retrun False

    View full-size slide

  16. With Typing…
    Boss < It looks like this function may return 3 types…
    Isn’t that too much?
    You < I see. That could be a bad design. Let me fix it.
    Boss < Sure, please.
    1 def need_new_post() -> None | False | str:
    16 / 57
    2 if ...: retrun None
    3 elif ...: retrun False
    4 else: return post_id # this is str

    View full-size slide

  17. Let’s start with function definitions
    Nothing too difficult! Just write type
    After the arguments, write colon and the type
    Before the colon at the end of the function definition,
    write an arrow and the type
    17 / 57

    View full-size slide

  18. Using built-in types
    bool , bytes , float , int , str
    You don’t need to do anything to use them.
    None : Used for functions that return nothing.
    18 / 57
    ` ` ` ` ` ` ` ` ` `
    ` `

    View full-size slide

  19. Escaping from complex type puzzles
    Any Can hold instances of any type.
    It’s better not to use it
    Import and use from typing when necessary.
    1 from typing import Any
    2 very_dangerous_last_resort: Any
    19 / 57
    ` `
    ` `

    View full-size slide

  20. (Since 3.9) Generics in standard Collections
    Before 3.9, from typing import Dict, List …
    Now, it’s depreciated
    Use dict , frozenset , list , set , tuple
    Collections can be written with [] for the type
    inside.
    ref: Official Documentation
    20 / 57
    ` `
    ` ` ` ` ` ` ` ` ` `
    ` `

    View full-size slide

  21. Module Availabilities in standard library…
    collections : (ex: deque, defaultdict)
    collections.abc : Iterable , Callable , or
    other Protocol-Related
    re : Regular Expressions
    contextlib : Context-Related items
    21 / 57
    ` `
    ` ` ` ` ` `
    ` `
    ` `

    View full-size slide

  22. Using types from collections.abc
    differently
    There are many types in collections.abc .
    It’s better to use a collection with a few methods to
    increase portability.
    Ex. Iterable is better than list .
    It’s a good idea to look at the methods used in your
    functions.
    The figure on the next page shows the relationship…
    22 / 57
    ` `
    ` `
    ` ` ` `

    View full-size slide

  23. Great method inheritance tree
    23 / 57

    View full-size slide

  24. The differences between tuple and other
    Sequences
    tuple is fixed up to the length of information
    Specify the type for the number of elements
    Or you can mix types, such as tuple[int,
    str, float] .
    24 / 57
    ` `
    `
    `

    View full-size slide

  25. A sequence, such as a list , has the same
    constraint for all elements
    Can be used regardless of the length of the
    Sequence by setting only one.
    25 / 57
    ` `

    View full-size slide

  26. 26 / 57
    Advanced:
    Generic Types

    View full-size slide

  27. Union (multiple types)
    union can be represented by | since 3.10
    1 def square(number: int | float) -> int | float:
    7 # Compatible with typing.Union
    8 int | str == typing.Union[int, str]
    27 / 57
    ` ` ` `
    2 return number ** 2
    1 # Nested unions are flattened
    2 (int | str) | float == int | str | float
    3 # Redundant types are removed
    4 int | str | int == int | str
    5 # Orders are ignored
    6 int | str == str | int

    View full-size slide

  28. Optional type
    Shorthand, Optional[T] is equivalent to Union
    with None.
    Behaves just like Union: T | None
    If you use it in a function return value or something,
    it will propagate, so be careful when you use it.
    3 age: Optional[int]
    4 age = 17
    5 age = None # This is also valid
    28 / 57
    ` `
    ` `
    1 from typing import Optional
    2

    View full-size slide

  29. Avoid using Optional as much as possible
    6 return None
    When you use the function, you might write another
    guard and return None .
    29 / 57
    1 def get_content() -> str | None:
    2 r = request.get("https://example.com")
    3 if r.status_code != 200:
    4 # Guard section (early return)
    5 logging.warning("Response: %d", r.status_code)
    7 return r.text
    ` `

    View full-size slide

  30. In this case…
    It would be cleaner to raise a raise RuntimeError .
    The cost of raising exceptions in Python is
    (relatively) low
    Little performance impact, unlike Java.
    The lack of null-safe methods in Python is also a
    factor, Unlike JavaScript.
    But, if there were such methods, they would be
    abused.
    30 / 57
    ` `

    View full-size slide

  31. Callable (callable objects)
    13 return func(*args, **kw)
    31 / 57
    1 from collections.abc import Callable # since 3.9
    2 from functools import wraps
    3
    4 def validate(func: Callable
    5 ) -> Callable[..., Callable]:
    6 @wraps(func)
    7 def wrapper(*args, **kw) -> Callable:
    8 try:
    9 j = request.json
    10 if j is None: raise BadRequest
    11 except BadRequest:
    12 return abort(400)
    14 return wrapper

    View full-size slide

  32. User-defined Generic types
    Example: a generic mapping type
    13 except KeyError: return default
    32 / 57
    1 from typing import TypeVar, Generic
    2 KT, VT = TypeVar("KT"), TypeVar("VT")
    3
    4 class Mapping(Generic[KT, VT]):
    5 def __getitem__(self, key: KT) -> VT: pass
    6
    7 X, Y = TypeVar("X"), TypeVar("Y")
    8
    9 def lookup_name( # Using Mapping
    10 mapping: Mapping[X, Y], key: X, default: Y
    11 ) -> Y:
    12 try: return mapping[key]

    View full-size slide

  33. 33 / 57
    How to use new features
    in older versions

    View full-size slide

  34. What is the __future__ module?
    Using typing new feature in the older versions, write
    from __future__ import annotations
    In addition to typing, it was also used to call 3.x
    features in 2.x.
    ex) print_func , unicode_literals etc …
    ref: __future__, future statement
    34 / 57
    ` `
    ` `
    ` ` ` `

    View full-size slide

  35. 35 / 57
    Recent Update

    View full-size slide

  36. Recent Python updates
    Ver. Status Release EoS Schedule Main new feature
    3.11 Alpha 7 2022-10 2027-10 PEP 619 Performance tuning
    3.10 Bugfix 2021-10 2026-10 PEP 619 Pattern Matching
    3.9 Bugfix 2020-10 2025-10 PEP 596 Union Operators
    3.8 Security 2019-10 2024-10 PEP 569 = in f-string
    3.7 Security 2018-06 2023-06 PEP 537 dataclasses
    2.7 EoL 2010-07 2020-01 PEP 373 Improving Decimals
    https://www.python.org/downloads/
    36 / 57
    ` `

    View full-size slide

  37. New Features on Type Hints in 3.10
    PEP 604: New Type Union Operator
    PEP 612: Parameter Specification Variables
    PEP 613: TypeAlias
    PEP 647: User-Defined Type Guards
    See Also:
    https://docs.python.org/3/whatsnew/3.10.html#new-
    features-related-to-type-hints
    37 / 57

    View full-size slide

  38. PEP 604: New Type Union Operator |
    The union above type can be used as an operator.
    You can also use it when asking isinstance() .
    1 int | str == typing.Union[int, str]
    2 # Compatible with typing.Union
    Optional[T] can also be written as T | None
    using this.
    38 / 57
    ` `
    ` `
    ` ` ` `

    View full-size slide

  39. PEP 612: Parameter Specification Variables
    Motivation
    Needed a way to represent a function that has the
    same arguments as the specified function
    1 from typing import Callable, TypeVar
    2 R = TypeVar("R")
    3
    4 def add_logging(f: Callable[..., R]
    5 ) -> Callable[..., R]:
    6 def inner(*args: object, **kw: object) -> R:
    7 log_to_database()
    8 return f(*args, **kwargs)
    9 return inner
    39 / 57

    View full-size slide

  40. 1 @add_logging
    2 def takes_int_str(x: int, y: str) -> int:
    3 return x + 7
    4
    5 await takes_int_str(1, "A")
    6 await takes_int_str("B", 2) # fails at runtime
    Approach
    Adding an argument type called
    ParameterSpecification solves the problem.
    It can be used with Callable to behave like a
    generic callable object.
    40 / 57
    ` `
    ` `

    View full-size slide

  41. Before
    11
    12 @add_logging
    13 def foo(x: int, y: str) -> int:
    14 return x + 7
    41 / 57
    1 from typing import Callable, TypeVar
    2 Ps, R = TypeVar("Ps"), TypeVar("R")
    3
    4 def add_logging(f: Callable[Ps, R]
    5 ) -> Callable[Ps, R]:
    6 # args: tuple...?, kwargs: dict...?
    7 def inner(*args, **kwargs) -> R:
    8 log_to_database(*args, **kwargs)
    9 return f(*args, **kwargs)
    10 return inner

    View full-size slide

  42. After
    10
    11 @add_logging
    12 def foo(x: int, y: str) -> int: return x + 7
    42 / 57
    1 from typing import Callable, ParameterSpecification, T
    2 Ps, R = ParameterSpecification("Ps"), TypeVar("R")
    3
    4 def add_logging(f: Callable[Ps, R]
    5 ) -> Callable[Ps, R]:
    6 def inner(*args: Ps.args, **kwargs: Ps.kwargs) -> R:
    7 log_to_database(*args, **kwargs)
    8 return f(*args, **kwargs)
    9 return inner

    View full-size slide

  43. PEP 613: TypeAlias
    Motivation
    We consider global variables without type hints to
    be type aliases.
    This tends to cause problems with forwarding
    references, scoping, etc.
    So, we’re going to make it possible to explicitly
    define type aliases.
    43 / 57

    View full-size slide

  44. Approach
    Add a new typing.TypeAlias
    You can write like this T: TypeAlias = int
    You can write like: T: TypeAlias = "int"
    Using ForwardReference
    44 / 57
    ` `
    ` `
    ` `

    View full-size slide

  45. Before & After
    7 x: TypeAlias = int # type alias
    8 x: TypeAlias = “MyClass” # type alias
    45 / 57
    1 x = 1 # untyped global expression
    2 x: int = 1 # typed global expression
    3
    4 x = int # untyped global expression
    5 x: Type[int] = int # typed global expression
    6

    View full-size slide

  46. PEP 647: User-Defined Type Guards
    Motivation
    Type checker tools use a technique called type
    narrowing to determine the type of information.
    5 else: pass # Type of val is narrowed to None
    46 / 57
    1 def func(val: Optional[str]):
    2 # "is None" type guard
    3 if val is not None:
    4 pass # Type of val is narrowed to str

    View full-size slide

  47. However, that will not work as intended if the user
    function is used.
    6 if is_str_list(val): print(" ".join(val))
    TypeGuard allows you to define user-defined type
    guards via the new typing.
    47 / 57
    1 def is_str_list(val: List[object]) -> bool:
    2 """Determines whether all objects in the list are st
    3 return all(isinstance(x, str) for x in val)
    4
    5 def func1(val: List[object]):
    7 # => Error: invalid type
    ` `

    View full-size slide

  48. 1 from typing import TypeGuard
    2
    3 def is_str_list(val: List[object]
    4 ) -> TypeGuard[List[str]]:
    5 return all(isinstance(x, str) for x in val)
    And, type narrowing works like next:
    48 / 57

    View full-size slide

  49. 10 else: reveal_type(val) # OneOrTwoStrs
    49 / 57
    1 def is_two_element_tuple(
    2 val: Tuple[str, ...]
    3 ) -> TypeGuard[Tuple[str, str]]:
    4 return len(val) == 2
    5
    6 OneOrTwoStrs = Union[Tuple[str], Tuple[str, str]]
    7 def func(val: OneOrTwoStrs):
    8 if is_two_element_tuple(val):
    9 reveal_type(val) # Tuple[str, str]

    View full-size slide

  50. New Feature on Type Hints in 3.11
    Python 3.11 is focused on performance tuning,
    but there are also new features in Typing…
    PEP 673: Self Type, It’s a way to annotate methods that
    return an instance of their class
    50 / 57

    View full-size slide

  51. Motivation
    Calling set_scale on a subclass of Shape , the type
    checker still infers the return type to be Shape .
    7 Circle().set_scale(0.5).set_radius(2.7)
    8 # => error: Shape has no attribute "set_radius"
    51 / 57
    ` ` ` `
    ` `
    1 class Circle(Shape):
    2 def set_radius(self, r: float) -> Circle:
    3 self.radius = r
    4 return self
    5
    6 Circle().set_scale(0.5) # *Shape*, not Circle

    View full-size slide

  52. Workaround, but it’s unintuitive…
    15 Circle().set_scale(0.5).set_radius(2.7) # => Circle
    52 / 57
    1 from typing import TypeVar
    2 TShape = TypeVar("TShape", bound="Shape")
    3
    4 class Shape:
    5 def set_scale(self: TShape, scale: float
    6 ) -> TShape:
    7 self.scale = scale
    8 return self
    9
    10 class Circle(Shape):
    11 def set_radius(self, radius: float) -> Circle:
    12 self.radius = radius
    13 return self
    14

    View full-size slide

  53. typing.Self resolves that unintuitively
    12 Circle().set_scale(0.5) # => It would be *Circle*!
    53 / 57
    ` `
    1 from typing import Self
    2 class Shape:
    3 def set_scale(self, scale: float) -> Self:
    4 self.scale = scale
    5 return self
    6
    7 class Circle(Shape):
    8 def set_radius(self, radius: float) -> Self:
    9 self.radius = radius
    10 return self
    11

    View full-size slide

  54. Summary
    1. Benefits, starting with def , Built-in types
    2. Standard collections (since 3.9)
    3. Collections, Union, Optional, Callable, User-defined
    Generics
    4. Updates Overview, __future__ for compatibility
    54 / 57
    ` `
    ` `

    View full-size slide

  55. 5. New features on Python 3.10 and 3.11
    1. 3.10: New Type Union Operator, Parameter
    Specific Variables, TypeAlias, User-Defined Type
    Guards
    2. 3.11: Self Type
    55 / 57

    View full-size slide

  56. Pages I used for reference (Thanks)
    There are links that I referenced. Thank you!
    https://docs.python.org/3/library/typing.html
    https://docs.python.org/3.10/whatsnew/3.10.html
    https://docs.python.org/3.11/whatsnew/3.11.html
    https://future-architect.github.io/articles/20201223 (ja)
    https://qiita.com/tk0miya/items/931da13af292060356b9 (ja)
    https://qiita.com/tk0miya/items/1b093c2eee7668fffb62 (ja)
    https://qiita.com/tk0miya/items/a27191f5d627601930ed (ja)
    56 / 57

    View full-size slide

  57. Thank you
    See you again next time in Japan!
    Powered by Slidev, using theme-briks

    View full-size slide