Oh my py Oh my py ! ! Igor Davydenko 2016, Dnepr.py#5

Type hints Type hints in Python in Python And how to use them And how to use them Igor Davydenko 2016, Dnepr.py#5

Brief History aka Brief History aka PEP-484 PEP-484 Designed on top of PEP-3107 (Function Annotations) Created on September 2014 Became part of standard library in Python 3.5 Endorsed by Guido van Rossum

Basics Basics def hello(text: str) -> str: return 'Hello, {0}'.format(text) def sum(a: int, b: int) -> int: return a + b

Basics Basics from decimal import Decimal, ROUND_UP PRECISION = Decimal('.01') def quantize_payment(hours: float, rate: Decimal) -> Decimal: return (Decimal(hours) * rate).quantize(PRECISION, ROUND_UP)

str, bytes, Text, AnyStr str, bytes, Text, AnyStr from typing import AnyStr, Text def response(body: bytes=None, text: str=None) -> bytes: ... def response(content: AnyStr=None) -> bytes: ... def render_template(path: Text) -> str: ...

list, List, Tuple, Set, Sequence list, List, Tuple, Set, Sequence from typing import List, Sequence, Set, Tuple def avg(data: list) -> float: return sum(data, 0.) / len(data) def avg(data: List[float]) -> float: return sum(data, 0.) / len(data) def split_name(value: str) -> Tuple[str, str]: return tuple(value.split(' ', 1)) def avg_tuple(data: Tuple[float, ...]) -> float: return sum(data, 0.) / len(data) def unique(data: List[str]) -> Set[str]: return set(data) def avg(data: Sequence[float]) -> float: return sum(data, 0.) / len(data)

dict, Dict, Mapping dict, Dict, Mapping from typing import Dict, Mapping def read_data(slug: str) -> dict: ... def read_data(slug: str) -> Dict[str, str]: ... def process_data(data: Mapping[str, str]) -> None: for key, value in data.items(): ...

Any, Union, Optional Any, Union, Optional from typing import Any, Optional, Sequence, Union def read_data(slug: str) -> Dict[str, Union[int, str]]: ... def read_data(slug: str) -> Dict[str, Any]: ... def avg(data: Sequence[float]) -> Optional[float]: if len(data): return sum(data, 0.) / len(data) return None

Callable Callable from random import randint from typing import Callable def random_int_factory(min: int=0, max: int=100) -> Callable[[], int]: def random_int(): return randint(min, max) return random_int

Aliases Aliases from aiohttp import web View = Callable[[web.Request], web.Response] def middleware() -> Callable[[web.Application, View], Awaitable[View]]: async def factory(app: web.Application, handler: View) -> View: async def middleware(request: web.Request) -> web.Response: ...

Classes Classes class Schema(object): name = None # type: str def __init__(self, name: str=None) -> None: self.name = name or self.name def clone(self) -> 'Schema': return Schema(self.name) def load(self) -> bool: with open(SCHEMA_PATH / '{0}.json'.format(self.name)) as handler: return json.loads(handler.read()) def validate(self, data: Dict[str, Any]) -> Dict[str, Any]: json_schema = self.load() return fastjsonschema.compile(json_schema).validate(data)

Types Types from typing import NewType UserID = NewType('UserID', int) def fetch_user(user_id: UserID) -> Any: ... fetch_user(UserID(1)) # OK fetch_user(1) # Type check will fail

Inline Type Hints Inline Type Hints WEEKS = defaultdict(lambda: { False: list(range(8)), True: list(range(16)), }) # type: Dict[str, Dict[bool, List[int]]]

Stubs Stubs package/module.py package/module.py def avg(data): if not len(data): return None return sum(data, 0.) / len(data) def avg_unique(data): return avg(unique(data)) def unique(data): return set(data)

Stubs Stubs package/module.pyi package/module.pyi def avg(data: Sequence[float]) -> Optional[float]: ... def avg_unique(data: Sequence[float]) -> Optional[float]: ... def unique(data: Sequence[float]) -> Set[float]: ...

Typeshed Typeshed python/typeshed Provide stubs for standard library And some widely-used shared libraries Have stubs for Python 2 & Python 3 Curated by PSF

Python 2 Type Hints Python 2 Type Hints def hello(name): # type: (name: str) -> str return 'Hello, {0}'.format(name) def multi_line_annotations(address, # type: Union[str, List[str]] sender, # type: str subject, # type: str body # type: str ): # type: (...) -> bool ... More info at mypy docs.

mypy mypy

Introduction Introduction Static type checker for Python Static type checker for Python Works with Python 3 & Python 2 code Still experimental Still experimental Developed at Dropbox Again. Endorsed by Guido

Usage Usage $ pip install mypy-lang typed-ast $ mypy ... $ mypy --fast-parser ...

Configuration. Step 1 Configuration. Step 1 mypy.ini mypy.ini [mypy] fast_parser = True check_untyped_defs = True warn_redundant_casts = True

Configuration. Step 2 Configuration. Step 2 mypy.ini mypy.ini silent_imports = True [mypy] fast_parser = True check_untyped_defs = True warn_redundant_casts = True

Configuration. Step 3 Configuration. Step 3 mypy.ini mypy.ini disallow_untyped_defs = True [mypy] fast_parser = True silent_imports = True check_untyped_defs = True warn_redundant_casts = True

Output Output env/bin/mypy project/ project/signals.py: note: In function "cache_api_urls": project/signals.py:25: error: Argument 2 to "api_url" has incompatible type "str"; expected "int"

Problems Problems

No Hype No Hype mypy is not widely used No viable benefits for users Hard to migrate large codebase to type hints

Long Lines / Ugly Code Long Lines / Ugly Code Before Before async def retrieve_tweets(pool, count=50): ... A er A er async def retrieve_tweets( pool: Pool, count: int=50 ) -> Sequence[Mapping[str, Any]]: ...

Stubs Stubs Hard to maintain Hard to maintain Easy to get into situation, when implementation != stub Completely same problems as with tests & documentation

Incomplete Stubs Incomplete Stubs import asyncio def main() -> int: loop = asyncio.get_event_loop() tasks = [ asyncio.ensure_future(some_async_def), asyncio.ensure_future(some_other_async_def), ] loop.run_until_complete(asyncio.gather(*tasks, loop=loop)) loop.close() return 0

Incomplete Stubs Incomplete Stubs $ mypy -c '...' : note: In function "main": :6: error: "module" has no attribute "ensure_future" :7: error: "module" has no attribute "ensure_future" :9: error: "module" has no attribute "gather" :6: error: Name 'some_async_def' is not defined :7: error: Name 'some_other_async_def' is not defined

Incomplete Stubs Incomplete Stubs from lxml import etree def fetch_data(url: str) -> etree._Element: ... def use_data(): data = fetch_data(URL) for item in data.iterfind(...): # Will fail with "has no attribute" error ...

# noqa: F401 # noqa: F401 from typing import Dict, List, Set def process_data(data: Dict[str, str]) -> List[int]: uniques = set() # type: Set[int] for value in data.values(): uniques.add(int(value)) return list(uniques) module.py:1:1: F401 'typing.Set' imported but unused

Solution to Solution to # noqa: F401 # noqa: F401 PEP-526 implements syntax for variable annotations. uniques: Set[int] = set() number: int # Works even without assignment class Schema(object): name: str data: Dict[str, str] = {} Included in Python 3.6 Included in Python 3.6

Circular Imports Circular Imports project/models.py project/models.py from .utils import some_func class Model(object): def shortcut_for_some_func(self) -> int: return some_func(self) project/utils.py project/utils.py import typing if typing.TYPE_CHECKING: from .models import Model def some_func(model: 'Model') -> int: ...

# type: ignore # type: ignore Sooner or later, but you'll need to use # type: ingore asyncio.gather(*tasks, loop=loop) # type: ignore for item in data.iterfind('...'): # type: ignore mypy is still experimental and you'll need to use # type: ignore a er yelling WTF !

Additional Notes Additional Notes

pytype pytype google/pytype Type checker from Google Needs Python 2.7 to run Able to type check Python 3 code though

enforce enforce RussBaz/enforce Runtime type checking Designed to use at tests or for data validation import enforce @enforce.runtime_validation def hello(name: str) -> str: return 'Hello, {0}!'.format(name) hello('world') hello(1) # Will fail with RuntimeTypeError

mypy-django mypy-django machinails/mypy-django Type hints bindings for Django Experimental Not full coverage of Django internals

Conclusion Conclusion " " # # # # $ $ % % & & ' '

Questions? Questions? Twi er: Twi er: @playpausenstop @playpausenstop GitHub: GitHub: @playpauseandstop @playpauseandstop Made in Ukraine