Igor Davydenko
November 02, 2016
360

Oh my py. Type hints in Python

Slides for Dnepr.py#5 talk about type hints / type annotations in Python 3 & mypy. How to use them, when and what the common problems

Igor Davydenko

November 02, 2016

Transcript

1. Oh my py
Oh my py
!
!
Igor Davydenko
2016, Dnepr.py#5

2. Type hints
Type hints
in Python
in Python
And how to use them
And how to use them
Igor Davydenko
2016, Dnepr.py#5

3. Brief History aka
Brief History aka PEP-484
PEP-484
Designed on top of PEP-3107 (Function Annotations)
Created on September 2014
Became part of standard library in Python 3.5
Endorsed by Guido van Rossum

4. Basics
Basics
def hello(text: str) -> str:
return 'Hello, {0}'.format(text)
def sum(a: int, b: int) -> int:
return a + b

5. Basics
Basics
from decimal import Decimal, ROUND_UP
PRECISION = Decimal('.01')
def quantize_payment(hours: float, rate: Decimal) -> Decimal:
return (Decimal(hours) * rate).quantize(PRECISION, ROUND_UP)

6. str, bytes, Text, AnyStr
str, bytes, Text, AnyStr
from typing import AnyStr, Text
def response(body: bytes=None, text: str=None) -> bytes:
...
def response(content: AnyStr=None) -> bytes:
...
def render_template(path: Text) -> str:
...

7. list, List, Tuple, Set, Sequence
list, List, Tuple, Set, Sequence
from typing import List, Sequence, Set, Tuple
def avg(data: list) -> float:
return sum(data, 0.) / len(data)
def avg(data: List[float]) -> float:
return sum(data, 0.) / len(data)
def split_name(value: str) -> Tuple[str, str]:
return tuple(value.split(' ', 1))
def avg_tuple(data: Tuple[float, ...]) -> float:
return sum(data, 0.) / len(data)
def unique(data: List[str]) -> Set[str]:
return set(data)
def avg(data: Sequence[float]) -> float:
return sum(data, 0.) / len(data)

8. dict, Dict, Mapping
dict, Dict, Mapping
from typing import Dict, Mapping
def read_data(slug: str) -> dict:
...
def read_data(slug: str) -> Dict[str, str]:
...
def process_data(data: Mapping[str, str]) -> None:
for key, value in data.items():
...

9. Any, Union, Optional
Any, Union, Optional
from typing import Any, Optional, Sequence, Union
def read_data(slug: str) -> Dict[str, Union[int, str]]:
...
def read_data(slug: str) -> Dict[str, Any]:
...
def avg(data: Sequence[float]) -> Optional[float]:
if len(data):
return sum(data, 0.) / len(data)
return None

10. Callable
Callable
from random import randint
from typing import Callable
def random_int_factory(min: int=0, max: int=100) -> Callable[[], int]:
def random_int():
return randint(min, max)
return random_int

11. Aliases
Aliases
from aiohttp import web
View = Callable[[web.Request], web.Response]
def middleware() -> Callable[[web.Application, View], Awaitable[View]]:
async def factory(app: web.Application, handler: View) -> View:
async def middleware(request: web.Request) -> web.Response:
...

12. Classes
Classes
class Schema(object):
name = None # type: str
def __init__(self, name: str=None) -> None:
self.name = name or self.name
def clone(self) -> 'Schema':
return Schema(self.name)
def load(self) -> bool:
with open(SCHEMA_PATH / '{0}.json'.format(self.name)) as handler:
def validate(self, data: Dict[str, Any]) -> Dict[str, Any]:
return fastjsonschema.compile(json_schema).validate(data)

13. Types
Types
from typing import NewType
UserID = NewType('UserID', int)
def fetch_user(user_id: UserID) -> Any:
...
fetch_user(UserID(1)) # OK
fetch_user(1) # Type check will fail

14. Inline Type Hints
Inline Type Hints
WEEKS = defaultdict(lambda: {
False: list(range(8)),
True: list(range(16)),
}) # type: Dict[str, Dict[bool, List[int]]]

15. Stubs
Stubs
package/module.py
package/module.py
def avg(data):
if not len(data):
return None
return sum(data, 0.) / len(data)
def avg_unique(data):
return avg(unique(data))
def unique(data):
return set(data)

16. Stubs
Stubs
package/module.pyi
package/module.pyi
def avg(data: Sequence[float]) -> Optional[float]: ...
def avg_unique(data: Sequence[float]) -> Optional[float]: ...
def unique(data: Sequence[float]) -> Set[float]: ...

17. Typeshed
Typeshed
python/typeshed
Provide stubs for standard library
And some widely-used shared libraries
Have stubs for Python 2 & Python 3
Curated by PSF

18. Python 2 Type Hints
Python 2 Type Hints
def hello(name): # type: (name: str) -> str
return 'Hello, {0}'.format(name)
def multi_line_annotations(address, # type: Union[str, List[str]]
sender, # type: str
subject, # type: str
body # type: str
):
# type: (...) -> bool
...

19. mypy
mypy

20. Introduction
Introduction
Static type checker for Python
Static type checker for Python
Works with Python 3 & Python 2 code
Still experimental
Still experimental
Developed at Dropbox
Again. Endorsed by Guido

21. Usage
Usage
\$ pip install mypy-lang typed-ast
\$ mypy ...
\$ mypy --fast-parser ...

22. Conﬁguration. Step 1
Conﬁguration. Step 1
mypy.ini
mypy.ini
[mypy]
fast_parser = True
check_untyped_defs = True
warn_redundant_casts = True

23. Conﬁguration. Step 2
Conﬁguration. Step 2
mypy.ini
mypy.ini
silent_imports = True
[mypy]
fast_parser = True
check_untyped_defs = True
warn_redundant_casts = True

24. Conﬁguration. Step 3
Conﬁguration. Step 3
mypy.ini
mypy.ini
disallow_untyped_defs = True
[mypy]
fast_parser = True
silent_imports = True
check_untyped_defs = True
warn_redundant_casts = True

25. Output
Output
env/bin/mypy project/
project/signals.py: note: In function "cache_api_urls":
project/signals.py:25: error: Argument 2 to "api_url" has incompatible type "str"; expected "int"

26. Problems
Problems

27. No Hype
No Hype
mypy is not widely used
No viable beneﬁts for users
Hard to migrate large codebase to type hints

28. Long Lines / Ugly Code
Long Lines / Ugly Code
Before
Before
async def retrieve_tweets(pool, count=50):
...
A er
A er
async def retrieve_tweets(
pool: Pool,
count: int=50
) -> Sequence[Mapping[str, Any]]:
...

29. Stubs
Stubs
Hard to maintain
Hard to maintain
Easy to get into situation, when implementation != stub
Completely same problems as with tests & documentation

30. Incomplete Stubs
Incomplete Stubs
import asyncio
def main() -> int:
loop = asyncio.get_event_loop()
asyncio.ensure_future(some_async_def),
asyncio.ensure_future(some_other_async_def),
]
loop.close()
return 0

31. Incomplete Stubs
Incomplete Stubs
\$ mypy -c '...'
: note: In function "main":
:6: error: "module" has no attribute "ensure_future"
:7: error: "module" has no attribute "ensure_future"
:9: error: "module" has no attribute "gather"
:6: error: Name 'some_async_def' is not defined
:7: error: Name 'some_other_async_def' is not defined

32. Incomplete Stubs
Incomplete Stubs
from lxml import etree
def fetch_data(url: str) -> etree._Element:
...
def use_data():
data = fetch_data(URL)
for item in data.iterfind(...): # Will fail with "has no attribute" error
...

33. # noqa: F401
# noqa: F401
from typing import Dict, List, Set
def process_data(data: Dict[str, str]) -> List[int]:
uniques = set() # type: Set[int]
for value in data.values():
return list(uniques)
module.py:1:1: F401 'typing.Set' imported but unused

34. Solution to
Solution to # noqa: F401
# noqa: F401
PEP-526 implements syntax for variable annotations.
uniques: Set[int] = set()
number: int # Works even without assignment
class Schema(object):
name: str
data: Dict[str, str] = {}
Included in Python 3.6
Included in Python 3.6

35. Circular Imports
Circular Imports
project/models.py
project/models.py
from .utils import some_func
class Model(object):
def shortcut_for_some_func(self) -> int:
return some_func(self)
project/utils.py
project/utils.py
import typing
if typing.TYPE_CHECKING:
from .models import Model
def some_func(model: 'Model') -> int:
...

36. # type: ignore
# type: ignore
Sooner or later, but you'll need to use # type: ingore
asyncio.gather(*tasks, loop=loop) # type: ignore
for item in data.iterfind('...'): # type: ignore
mypy is still experimental and you'll need to use # type: ignore a er yelling WTF !

38. pytype
pytype
Type checker from Google
Needs Python 2.7 to run
Able to type check Python 3 code though

39. enforce
enforce
RussBaz/enforce
Runtime type checking
Designed to use at tests or for data validation
import enforce
@enforce.runtime_validation
def hello(name: str) -> str:
return 'Hello, {0}!'.format(name)
hello('world')
hello(1) # Will fail with RuntimeTypeError

40. mypy-django
mypy-django
machinails/mypy-django
Type hints bindings for Django
Experimental
Not full coverage of Django internals

41. Conclusion
Conclusion
"
" #
# #
# \$
\$ %
% &
& '
'

42. Questions?
Questions?
Twi er:
Twi er: @playpausenstop
@playpausenstop
GitHub:
GitHub: @playpauseandstop
@playpauseandstop