Optional static typing in Python - Mypy

Transcript

1. Mypy : Optional static typing for Python Wasim Thabraze
2. Outline • Motivation • Type Annotations • Usage of Mypy • Project Typeshed • How to type hint existing codebase
3. Static type checking • A language is statically-typed if the type of a variable is known at compile time instead of at runtime. • Examples include C, C++, Java etc. • The big benefit of static type checking is that it allows many type errors to be caught early in the development cycle.
4. Dynamic type checking • Dynamic type checking is the process of verifying the type safety of a program at runtime • Examples include Python, JavaScript, Ruby etc • Dynamic typing is more flexible and results in more compact programs since it doesn’t types to be spelled out
5. Motivation Can you guess the argument types? def process(client, type, data, rate=1): pass
6. What can we do? Attempt #1 Use docstrings?
7. Possible drawbacks of docstrings • Really verbose • Tough to parse • No early error checking • Unsync of docstrings and code
8. What can we do? Attempt #2 Use Pytest?
9. What can we do? Attempt #3 Use Type annotations?
10. Type annotations Type hinting is literally what the words mean, you annotate or hint the object(s) you’re using.
11. Type annotations • Formulated by PEP-484 and PEP-3107 • Since Python 3.5 type-annotations have been officially added to Python
12. Type annotations in Python 3 def process(client: ClientClass, type:str, data: List[str], rate: int=1) -> None:
13. Type annotations in Python 2 Since the type annotation syntax was introduced in Python 3, for code that needs to be Python 2.7 compatible, type annotations are given in comments.
14. Type annotations in Python 2 def process(client, type, data, rate=1): # type: (ClientClass, str, List[str], int)-> None """Docstring comes after type comment.""" ...
15. More complex example from typing import List, Dict, Union, Any, Optional def func(data: Any) -> List[Dict[int, Union[int, str]]] def send_email(address: Union[str, List[str]], sender: str, cc: Optional[List[str]], subject='', body: Optional[List[str]] = None ) -> bool:
16. Understanding complex types Any : Use Any if you don’t know the type Union : Use Union when something could be one of a few types Optional : Use Optional for values that could be None Iterable : Use Iterable for generic iterables
17. How to evaluate the code? Mypy to the rescue!
18. Installing mypy • Mypy requires Python 3.3 and later • To install Mypy simply run $ pip3 install mypy • And you’re good to go
19. Invoking Mypy mypy [--py2] filename.py
20. Example def greeting(name: str) -> str: return ‘Hello ’+ name greeting(5)
21. Example def greeting(name: str) -> str: return ‘Hello ’+ name greeting(5) $ mypy filename.py error: Argument 1 to "greeting" has incompatible type "int"; expected "str"
22. Using Mypy via stub files • Instead of type annotating code we can create stub files (.pyi files) to declare the types of variables and methods • Stub files do not contain run time logic and the method body is replaced with ellipses. x: int def func1(code: str) -> int: ... def func2(a: int, b: int = ...) -> int: …
23. Type inference i = 5 # Inferred type is int i = ‘123’ # error: Incompatible types in assignment def func(l: List[object],k: List[int]) -> None: l = k # error: Incompatible types in assignment
24. Explicit types • We can override the inferred type of a variable by using a variable type annotation. from typing import Union i: Union[int, str] = 1
25. Things to note • Type annotations doesn’t cause any overhead when running a program • The type annotations are treated as comments • Import the types you use in your type hints using the typing module • Docstrings should appear after the type comment
26. When to do static checking? • mypy (linter on steroids) • Unit tests and Integration tests • Flake8 or Pylint
27. Project Typeshed • Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages. • Each Python module is represented by a .pyi stub. This is a normal Python file except that all the methods are empty. • This data can be used for static analysis, type checking or type inference.
28. Type checking projects • First start by ignoring imported modules $ mypy --slient-imports • Incorporate stub files(.pyi files) incrementally • As you incorporate stub files $ mypy --almost-silent
29. Type checking projects continued ... • Mypy doesn’t type check ‘C’ extension modules • In such case we can silence the type checker on a particular line as import frobnicate # type: ignore frobnicate.start()
30. Advantages • Improved understanding or refactoring of code • Helps catching errors before runtime • Easy to incorporate into CI • Got editor support via plugins (Pycharm, Emacs) • Active community. Not perfect but improving quickly
31. Notable drawbacks • Adding stub files duplicates your code base and now every function has two definitions • It’s not possible to annotate contents inside functions • There is no check that the implementation matches signature of the stub file • We cannot type check the code in the stub file
32. Closing notes • Mypy is an active project and you may encounter issues at some point • Not all (top at least) Python packages have stub files associated with them • Also try tools like Pyre, Ptype and MonkeyType • Static typing sounds like time consuming but worth it in long term
33. Q & A