Python Static Typing with MyPy

First presented at Python-Madrid meetup

Transcript

1. Python static typing with MyPy @Kartones Diego Muñoz
2. 01. Intro 02. Type Hints 03. MyPy 04. Advanced Topics Agenda Agenda
3. Intro
4. Weak Typing vs Strong Typing Static Typing vs Dynamic Typing Intro
5. Strong typing helps scale up codebases Intro
6. Dynamic typing speeds up development cycles Intro
7. But requires type-checking logic, decorators, validators... Intro
8. Type hinting + type checking bring that “safety net” back Intro
9. Type Hints
10. Python 3.5 - Function/Method annotations - Method argument annotations Type Hints Python 3.6 - Variable annotations Python 2.7 - All of them * def my_method(an_integer: int = 0) -> int: the_result: int = an_integer + 1 return the_result
11. 3 Modes - Comment annotations - Type hints - Stub files Type Hints
12. Comment Annotations - For Python 2 Type Hints from typing import Union def my_method(an_integer, a_float): # type: (int, float) -> Union[None, float] if a_float == 0: return None the_result = an_integer / a_float # type: float return the_result
13. Type Hints - For Python 3 Type Hints from typing import Union def my_method(an_integer: int, a_float: float) -> Union[None, float]: if a_float == 0: return None the_result: float = an_integer / a_float return the_result
14. Stub files - .pyi files - Code you cannot annotate directly (e.g. external library) - stubgen <module> Type Hints from typing import Any, List from django.views.generic import View class MyLibView(object): http_method_names: List def __init__(self, **kwargs: Any) -> None: ... def as_view(self, cls: View, **initkwargs: Any) -> View: ... def sum_values(self, first: int, second: int) -> int: ...
15. - Primitives Type Hints from typing import Union def my_method(an_integer: int, a_float: float) -> Union[None, float]: if a_float == 0: return None the_result: float = an_integer / a_float return the_result
16. - Unknown type Type Hints from typing import Any class MyDjangoView(View): def post(self, request: WSGIRequest, **kwargs: Any) -> JsonResponse: pass
17. - Lists and Tuples Type Hints from typing import List, Tuple def numeric_tuple_to_list(source_tuple: Tuple) -> List: return [item for item in source_tuple] def numeric_tuple_to_list(source_tuple: Tuple[int]) -> List[int]: return [item for item in source_tuple]
18. - Dictionaries Type Hints from typing import Dict def my_method(input_data: Dict) -> None: for key, value in input_data.items(): print(f"{key}:{value}") def my_method_2(input_data: Dict[str, int]) -> None: for key, value in input_data.items(): print(f"{key}:{value}")
19. - Multiple types Type Hints from typing import Union def my_method(an_integer: int, a_float: float) -> Union[None, float]: if a_float == 0: return None the_result: float = an_integer / a_float return the_result
20. - “Nullable type” (type or None) Type Hints from typing import Optional def my_method(an_integer: int, a_float: float) -> Optional[float]: if a_float == 0: return None the_result: float = an_integer / a_float return the_result
21. Annotations not enforced by Python Left for type checkers to use (gradual typing) Type Hints
22. MyPy
23. Static type checker tool MyPy
24. How to run - From command line MyPy mypy myproj myproj/models.py:46: error: Function is missing a type annotation for one or more arguments myproj/models.py:54: error: Unsupported operand types for >= ("int" and "str") myproj/tests/models_test.py:253: error: Argument 2 to "my_method" of "MyClass" has incompatible type "int"; expected "str"
25. How to run - As a “linter” test MyPy import subprocess import sys def test_mypy_compliance() -> None: mypy_binary = subprocess.check_output("which mypy", shell=True, stderr=sys.stderr) .decode("ascii") .replace("\n", "") result = subprocess.call("{} apu".format(mypy_binary), shell=True, stdout=sys.stdout, stderr=sys.stderr) assert result == 0
26. How to run - As an IDE plugin MyPy
27. Advanced Topics
28. Python 3 - Inform that a variable is coerced to a certain type Advanced Topics from typing import cast # let’s imagine my_input comes from an external module print(5 + cast(int, my_input))
29. Python 3 - Supports Inheritance Advanced Topics class Vehicle: pass class Car(Vehicle): pass def start_engine(vehicle: Vehicle) -> None: print(vehicle.__class__) mycar = Car() start_engine(mycar)
30. Python 3 - Types can also be specified Advanced Topics from typing import Type def validate_type(a_vehicle_class: Type) -> bool: return a_vehicle_class in [Car, Truck]
31. Python 3 - Iterators Advanced Topics from typing import Iterator def find_items() -> Iterator[MyClass]: # ... yield item
32. Python 3 - Function/Method handlers Advanced Topics from typing import Callable def my_method1(func: Callable, value: int) -> int: return func(value) def my_method2(func: Callable[[int], int], value: int) -> int: return func(value)
33. Python 3 - Type aliases Advanced Topics from typing import List Vector = List[float]
34. Python 3 - Forward references are problematic - Solution: - Python < 3.7: Quotes - Python >= 3.7: PEP 563: Postponed Evaluation of Annotations Advanced Topics class ClassA: def a_method(self, target: "ClassB") -> "ClassB": target.b_method() return target class ClassB: def b_method(self) -> None: pass
35. Python 3 - Import cycles are problematic - Solution: Type checking flag + Quotes Advanced Topics from typing import TYPE_CHECKING if TYPE_CHECKING: from b import ClassB def some_method(target: "ClassB") -> None: target.b_method() from a import some_method class ClassB: def b_method(self) ->None: pass def a_method(self) -> None: some_method(self)
36. Dropbox’s PyAnnotate: Automatically annotate your code Advanced Topics
37. Needs changes to your tests → use Pytest-annotate Advanced Topics
38. Python 2-style comment annotations :( Advanced Topics
39. Instagram’s MonkeyType: Automatically annotate your code Advanced Topics
40. Custom Python profiler → no need to change code Advanced Topics
41. Python 3-style type hints :) Advanced Topics
42. Type hints: https://www.python.org/dev/peps/pep-0483/ https://www.python.org/dev/peps/pep-0484/ MyPy: http://mypy-lang.org/ PyAnnotate: https://github.com/dropbox/pyannotate https://github.com/kensho-technologies/pytest-annotate MonkeyType: https://github.com/Instagram/MonkeyType Advanced Topics
43. THE END Thanks! @Kartones Slides: https://speakerdeck.com/ticketeaeng