Share
Embed
Oh my py. Type hints in Python
Slides for Dnepr.py#5 talk about type hints / type annotations in Python 3 & mypy. How to use them, when and what the common problems
Igor Davydenko
November 02, 2016
Transcript
42 slides
-
Oh my py Oh my py ! ! Igor Davydenko
2016, Dnepr.py#5 -
Type hints Type hints in Python in Python And how
to use them And how to use them Igor Davydenko 2016, Dnepr.py#5 -
Brief History aka Brief History aka PEP-484 PEP-484 Designed on
top of PEP-3107 (Function Annotations) Created on September 2014 Became part of standard library in Python 3.5 Endorsed by Guido van Rossum -
Basics Basics def hello(text: str) -> str: return 'Hello, {0}'.format(text)
def sum(a: int, b: int) -> int: return a + b -
Basics Basics from decimal import Decimal, ROUND_UP PRECISION = Decimal('.01')
def quantize_payment(hours: float, rate: Decimal) -> Decimal: return (Decimal(hours) * rate).quantize(PRECISION, ROUND_UP) -
str, bytes, Text, AnyStr str, bytes, Text, AnyStr from typing
import AnyStr, Text def response(body: bytes=None, text: str=None) -> bytes: ... def response(content: AnyStr=None) -> bytes: ... def render_template(path: Text) -> str: ... -
list, List, Tuple, Set, Sequence list, List, Tuple, Set, Sequence
from typing import List, Sequence, Set, Tuple def avg(data: list) -> float: return sum(data, 0.) / len(data) def avg(data: List[float]) -> float: return sum(data, 0.) / len(data) def split_name(value: str) -> Tuple[str, str]: return tuple(value.split(' ', 1)) def avg_tuple(data: Tuple[float, ...]) -> float: return sum(data, 0.) / len(data) def unique(data: List[str]) -> Set[str]: return set(data) def avg(data: Sequence[float]) -> float: return sum(data, 0.) / len(data) -
dict, Dict, Mapping dict, Dict, Mapping from typing import Dict,
Mapping def read_data(slug: str) -> dict: ... def read_data(slug: str) -> Dict[str, str]: ... def process_data(data: Mapping[str, str]) -> None: for key, value in data.items(): ... -
Any, Union, Optional Any, Union, Optional from typing import Any,
Optional, Sequence, Union def read_data(slug: str) -> Dict[str, Union[int, str]]: ... def read_data(slug: str) -> Dict[str, Any]: ... def avg(data: Sequence[float]) -> Optional[float]: if len(data): return sum(data, 0.) / len(data) return None -
Callable Callable from random import randint from typing import Callable
def random_int_factory(min: int=0, max: int=100) -> Callable[[], int]: def random_int(): return randint(min, max) return random_int -
Aliases Aliases from aiohttp import web View = Callable[[web.Request], web.Response]
def middleware() -> Callable[[web.Application, View], Awaitable[View]]: async def factory(app: web.Application, handler: View) -> View: async def middleware(request: web.Request) -> web.Response: ... -
Classes Classes class Schema(object): name = None # type: str
def __init__(self, name: str=None) -> None: self.name = name or self.name def clone(self) -> 'Schema': return Schema(self.name) def load(self) -> bool: with open(SCHEMA_PATH / '{0}.json'.format(self.name)) as handler: return json.loads(handler.read()) def validate(self, data: Dict[str, Any]) -> Dict[str, Any]: json_schema = self.load() return fastjsonschema.compile(json_schema).validate(data) -
Types Types from typing import NewType UserID = NewType('UserID', int)
def fetch_user(user_id: UserID) -> Any: ... fetch_user(UserID(1)) # OK fetch_user(1) # Type check will fail -
Inline Type Hints Inline Type Hints WEEKS = defaultdict(lambda: {
False: list(range(8)), True: list(range(16)), }) # type: Dict[str, Dict[bool, List[int]]] -
Stubs Stubs package/module.py package/module.py def avg(data): if not len(data): return
None return sum(data, 0.) / len(data) def avg_unique(data): return avg(unique(data)) def unique(data): return set(data) -
Stubs Stubs package/module.pyi package/module.pyi def avg(data: Sequence[float]) -> Optional[float]: ...
def avg_unique(data: Sequence[float]) -> Optional[float]: ... def unique(data: Sequence[float]) -> Set[float]: ... -
Typeshed Typeshed python/typeshed Provide stubs for standard library And some
widely-used shared libraries Have stubs for Python 2 & Python 3 Curated by PSF -
Python 2 Type Hints Python 2 Type Hints def hello(name):
# type: (name: str) -> str return 'Hello, {0}'.format(name) def multi_line_annotations(address, # type: Union[str, List[str]] sender, # type: str subject, # type: str body # type: str ): # type: (...) -> bool ... More info at mypy docs. -
Introduction Introduction Static type checker for Python Static type checker
for Python Works with Python 3 & Python 2 code Still experimental Still experimental Developed at Dropbox Again. Endorsed by Guido -
Usage Usage $ pip install mypy-lang typed-ast $ mypy ...
$ mypy --fast-parser ... -
Configuration. Step 1 Configuration. Step 1 mypy.ini mypy.ini [mypy] fast_parser
= True check_untyped_defs = True warn_redundant_casts = True -
Configuration. Step 2 Configuration. Step 2 mypy.ini mypy.ini silent_imports =
True [mypy] fast_parser = True check_untyped_defs = True warn_redundant_casts = True -
Configuration. Step 3 Configuration. Step 3 mypy.ini mypy.ini disallow_untyped_defs =
True [mypy] fast_parser = True silent_imports = True check_untyped_defs = True warn_redundant_casts = True -
Output Output env/bin/mypy project/ project/signals.py: note: In function "cache_api_urls": project/signals.py:25:
error: Argument 2 to "api_url" has incompatible type "str"; expected "int" -
No Hype No Hype mypy is not widely used No
viable benefits for users Hard to migrate large codebase to type hints -
Long Lines / Ugly Code Long Lines / Ugly Code
Before Before async def retrieve_tweets(pool, count=50): ... A er A er async def retrieve_tweets( pool: Pool, count: int=50 ) -> Sequence[Mapping[str, Any]]: ... -
Stubs Stubs Hard to maintain Hard to maintain Easy to
get into situation, when implementation != stub Completely same problems as with tests & documentation -
Incomplete Stubs Incomplete Stubs import asyncio def main() -> int:
loop = asyncio.get_event_loop() tasks = [ asyncio.ensure_future(some_async_def), asyncio.ensure_future(some_other_async_def), ] loop.run_until_complete(asyncio.gather(*tasks, loop=loop)) loop.close() return 0 -
Incomplete Stubs Incomplete Stubs $ mypy -c '...' <string>: note:
In function "main": <string>:6: error: "module" has no attribute "ensure_future" <string>:7: error: "module" has no attribute "ensure_future" <string>:9: error: "module" has no attribute "gather" <string>:6: error: Name 'some_async_def' is not defined <string>:7: error: Name 'some_other_async_def' is not defined -
Incomplete Stubs Incomplete Stubs from lxml import etree def fetch_data(url:
str) -> etree._Element: ... def use_data(): data = fetch_data(URL) for item in data.iterfind(...): # Will fail with "has no attribute" error ... -
# noqa: F401 # noqa: F401 from typing import Dict,
List, Set def process_data(data: Dict[str, str]) -> List[int]: uniques = set() # type: Set[int] for value in data.values(): uniques.add(int(value)) return list(uniques) module.py:1:1: F401 'typing.Set' imported but unused -
Solution to Solution to # noqa: F401 # noqa: F401
PEP-526 implements syntax for variable annotations. uniques: Set[int] = set() number: int # Works even without assignment class Schema(object): name: str data: Dict[str, str] = {} Included in Python 3.6 Included in Python 3.6 -
Circular Imports Circular Imports project/models.py project/models.py from .utils import some_func
class Model(object): def shortcut_for_some_func(self) -> int: return some_func(self) project/utils.py project/utils.py import typing if typing.TYPE_CHECKING: from .models import Model def some_func(model: 'Model') -> int: ... -
# type: ignore # type: ignore Sooner or later, but
you'll need to use # type: ingore asyncio.gather(*tasks, loop=loop) # type: ignore for item in data.iterfind('...'): # type: ignore mypy is still experimental and you'll need to use # type: ignore a er yelling WTF ! -
pytype pytype google/pytype Type checker from Google Needs Python 2.7
to run Able to type check Python 3 code though -
enforce enforce RussBaz/enforce Runtime type checking Designed to use at
tests or for data validation import enforce @enforce.runtime_validation def hello(name: str) -> str: return 'Hello, {0}!'.format(name) hello('world') hello(1) # Will fail with RuntimeTypeError -
mypy-django mypy-django machinails/mypy-django Type hints bindings for Django Experimental Not
full coverage of Django internals -
Conclusion Conclusion " " # # # # $ $
% % & & ' ' -
Questions? Questions? Twi er: Twi er: @playpausenstop @playpausenstop GitHub: GitHub:
@playpauseandstop @playpauseandstop Made in Ukraine