Oh my py. Type hints in Python

Slides for Dnepr.py#5 talk about type hints / type annotations in Python 3 & mypy. How to use them, when and what the common problems

Transcript

1. Oh my py Oh my py ! ! Igor Davydenko 2016, Dnepr.py#5
2. Type hints Type hints in Python in Python And how to use them And how to use them Igor Davydenko 2016, Dnepr.py#5
3. Brief History aka Brief History aka PEP-484 PEP-484 Designed on top of PEP-3107 (Function Annotations) Created on September 2014 Became part of standard library in Python 3.5 Endorsed by Guido van Rossum
4. Basics Basics def hello(text: str) -> str: return 'Hello, {0}'.format(text) def sum(a: int, b: int) -> int: return a + b
5. Basics Basics from decimal import Decimal, ROUND_UP PRECISION = Decimal('.01') def quantize_payment(hours: float, rate: Decimal) -> Decimal: return (Decimal(hours) * rate).quantize(PRECISION, ROUND_UP)
6. str, bytes, Text, AnyStr str, bytes, Text, AnyStr from typing import AnyStr, Text def response(body: bytes=None, text: str=None) -> bytes: ... def response(content: AnyStr=None) -> bytes: ... def render_template(path: Text) -> str: ...
7. list, List, Tuple, Set, Sequence list, List, Tuple, Set, Sequence from typing import List, Sequence, Set, Tuple def avg(data: list) -> float: return sum(data, 0.) / len(data) def avg(data: List[float]) -> float: return sum(data, 0.) / len(data) def split_name(value: str) -> Tuple[str, str]: return tuple(value.split(' ', 1)) def avg_tuple(data: Tuple[float, ...]) -> float: return sum(data, 0.) / len(data) def unique(data: List[str]) -> Set[str]: return set(data) def avg(data: Sequence[float]) -> float: return sum(data, 0.) / len(data)
8. dict, Dict, Mapping dict, Dict, Mapping from typing import Dict, Mapping def read_data(slug: str) -> dict: ... def read_data(slug: str) -> Dict[str, str]: ... def process_data(data: Mapping[str, str]) -> None: for key, value in data.items(): ...
9. Any, Union, Optional Any, Union, Optional from typing import Any, Optional, Sequence, Union def read_data(slug: str) -> Dict[str, Union[int, str]]: ... def read_data(slug: str) -> Dict[str, Any]: ... def avg(data: Sequence[float]) -> Optional[float]: if len(data): return sum(data, 0.) / len(data) return None
10. Callable Callable from random import randint from typing import Callable def random_int_factory(min: int=0, max: int=100) -> Callable[[], int]: def random_int(): return randint(min, max) return random_int
11. Aliases Aliases from aiohttp import web View = Callable[[web.Request], web.Response] def middleware() -> Callable[[web.Application, View], Awaitable[View]]: async def factory(app: web.Application, handler: View) -> View: async def middleware(request: web.Request) -> web.Response: ...
12. Classes Classes class Schema(object): name = None # type: str def __init__(self, name: str=None) -> None: self.name = name or self.name def clone(self) -> 'Schema': return Schema(self.name) def load(self) -> bool: with open(SCHEMA_PATH / '{0}.json'.format(self.name)) as handler: return json.loads(handler.read()) def validate(self, data: Dict[str, Any]) -> Dict[str, Any]: json_schema = self.load() return fastjsonschema.compile(json_schema).validate(data)
13. Types Types from typing import NewType UserID = NewType('UserID', int) def fetch_user(user_id: UserID) -> Any: ... fetch_user(UserID(1)) # OK fetch_user(1) # Type check will fail
14. Inline Type Hints Inline Type Hints WEEKS = defaultdict(lambda: { False: list(range(8)), True: list(range(16)), }) # type: Dict[str, Dict[bool, List[int]]]
15. Stubs Stubs package/module.py package/module.py def avg(data): if not len(data): return None return sum(data, 0.) / len(data) def avg_unique(data): return avg(unique(data)) def unique(data): return set(data)
16. Stubs Stubs package/module.pyi package/module.pyi def avg(data: Sequence[float]) -> Optional[float]: ... def avg_unique(data: Sequence[float]) -> Optional[float]: ... def unique(data: Sequence[float]) -> Set[float]: ...
17. Typeshed Typeshed python/typeshed Provide stubs for standard library And some widely-used shared libraries Have stubs for Python 2 & Python 3 Curated by PSF
18. Python 2 Type Hints Python 2 Type Hints def hello(name): # type: (name: str) -> str return 'Hello, {0}'.format(name) def multi_line_annotations(address, # type: Union[str, List[str]] sender, # type: str subject, # type: str body # type: str ): # type: (...) -> bool ... More info at mypy docs.
19. mypy mypy
20. Introduction Introduction Static type checker for Python Static type checker for Python Works with Python 3 & Python 2 code Still experimental Still experimental Developed at Dropbox Again. Endorsed by Guido
21. Usage Usage $ pip install mypy-lang typed-ast $ mypy ... $ mypy --fast-parser ...
22. Configuration. Step 1 Configuration. Step 1 mypy.ini mypy.ini [mypy] fast_parser = True check_untyped_defs = True warn_redundant_casts = True
23. Configuration. Step 2 Configuration. Step 2 mypy.ini mypy.ini silent_imports = True [mypy] fast_parser = True check_untyped_defs = True warn_redundant_casts = True
24. Configuration. Step 3 Configuration. Step 3 mypy.ini mypy.ini disallow_untyped_defs = True [mypy] fast_parser = True silent_imports = True check_untyped_defs = True warn_redundant_casts = True
25. Output Output env/bin/mypy project/ project/signals.py: note: In function "cache_api_urls": project/signals.py:25: error: Argument 2 to "api_url" has incompatible type "str"; expected "int"
26. Problems Problems
27. No Hype No Hype mypy is not widely used No viable benefits for users Hard to migrate large codebase to type hints
28. Long Lines / Ugly Code Long Lines / Ugly Code Before Before async def retrieve_tweets(pool, count=50): ... A er A er async def retrieve_tweets( pool: Pool, count: int=50 ) -> Sequence[Mapping[str, Any]]: ...
29. Stubs Stubs Hard to maintain Hard to maintain Easy to get into situation, when implementation != stub Completely same problems as with tests & documentation
30. Incomplete Stubs Incomplete Stubs import asyncio def main() -> int: loop = asyncio.get_event_loop() tasks = [ asyncio.ensure_future(some_async_def), asyncio.ensure_future(some_other_async_def), ] loop.run_until_complete(asyncio.gather(*tasks, loop=loop)) loop.close() return 0
31. Incomplete Stubs Incomplete Stubs $ mypy -c '...' <string>: note: In function "main": <string>:6: error: "module" has no attribute "ensure_future" <string>:7: error: "module" has no attribute "ensure_future" <string>:9: error: "module" has no attribute "gather" <string>:6: error: Name 'some_async_def' is not defined <string>:7: error: Name 'some_other_async_def' is not defined
32. Incomplete Stubs Incomplete Stubs from lxml import etree def fetch_data(url: str) -> etree._Element: ... def use_data(): data = fetch_data(URL) for item in data.iterfind(...): # Will fail with "has no attribute" error ...
33. # noqa: F401 # noqa: F401 from typing import Dict, List, Set def process_data(data: Dict[str, str]) -> List[int]: uniques = set() # type: Set[int] for value in data.values(): uniques.add(int(value)) return list(uniques) module.py:1:1: F401 'typing.Set' imported but unused
34. Solution to Solution to # noqa: F401 # noqa: F401 PEP-526 implements syntax for variable annotations. uniques: Set[int] = set() number: int # Works even without assignment class Schema(object): name: str data: Dict[str, str] = {} Included in Python 3.6 Included in Python 3.6
35. Circular Imports Circular Imports project/models.py project/models.py from .utils import some_func class Model(object): def shortcut_for_some_func(self) -> int: return some_func(self) project/utils.py project/utils.py import typing if typing.TYPE_CHECKING: from .models import Model def some_func(model: 'Model') -> int: ...
36. # type: ignore # type: ignore Sooner or later, but you'll need to use # type: ingore asyncio.gather(*tasks, loop=loop) # type: ignore for item in data.iterfind('...'): # type: ignore mypy is still experimental and you'll need to use # type: ignore a er yelling WTF !
37. Additional Notes Additional Notes
38. pytype pytype google/pytype Type checker from Google Needs Python 2.7 to run Able to type check Python 3 code though
39. enforce enforce RussBaz/enforce Runtime type checking Designed to use at tests or for data validation import enforce @enforce.runtime_validation def hello(name: str) -> str: return 'Hello, {0}!'.format(name) hello('world') hello(1) # Will fail with RuntimeTypeError
40. mypy-django mypy-django machinails/mypy-django Type hints bindings for Django Experimental Not full coverage of Django internals
41. Conclusion Conclusion " " # # # # $ $ % % & & ' '
42. Questions? Questions? Twi er: Twi er: @playpausenstop @playpausenstop GitHub: GitHub: @playpauseandstop @playpauseandstop Made in Ukraine