Effective Modern Python 2018

Transcript

1. Effective Modern Python 2018 Ryo Takahashi 1 / 19
2. Contents Type Annotation Data Classes Packaging and Dependency Management Effective Modern Python 2018 2 / 19
3. Type Annotation python python 3.5 3.5 3 / 19
4. Benefits of type annotations Static code analysis prior to runtime PyCharm (IntelliJ) mypy Self­documenting Code itself is a document Code completion Effective Modern Python 2018 4 / 19
5. Chainer RNNLM Example train, val, test = chainer.datasets.get_ptb_words() Type of train, val, test? What kind of operations can be done on these variables? Effective Modern Python 2018 5 / 19
6. Effective Modern Python 2018 6 / 19
7. Go to Declaration def get_ptb_words(): """Gets the Penn Tree Bank dataset as long word sequences. [...] Returns: tuple of numpy.ndarray: Int32 vectors of word IDs. """ train = _retrieve_ptb_words('train.npz', _train_url) valid = _retrieve_ptb_words('valid.npz', _valid_url) test = _retrieve_ptb_words('test.npz', _test_url) return train, valid, test docstring IDEs can't parse complicated types Effective Modern Python 2018 7 / 19
8. Add Type Annotations from typing import Dict def load_vocab(vocabfile: str) -> Dict[int, str]: [...] # typed! vocab = load_vocab(vocabfile) Effective Modern Python 2018 8 / 19
9. Add Type Annotations If you can't modify function declaration... # function of other libraries def load_vocab(vocabfile): [...] from typing import Dict # typed! vocab: Dict[int, str] = load_vocab(vocabfile) Effective Modern Python 2018 9 / 19
10. Effective Modern Python 2018 10 / 19
11. Data Classes python python 3.7 3.7 11 / 19
12. Easier Way to Write Classes from dataclasses import dataclass from typing import Dict, Optional @dataclass class Vocabulary: padding_token: str oov_token: str max_vocab_size: int pretrained_files: Optional[Dict[str, str]] = None Similar to Scala's Case Classes Generate __init__, __eq__, and __repr__ automatically Effective Modern Python 2018 12 / 19
13. Conventional Class class Vocabulary: def __init__( self, padding_token: str, oov_token: str, max_vocab_size: int, pretrained_files: Optional[Dict[str, str]] = None, ): self.padding_token = padding_token self.oov_token = oov_token self.max_vocab_size = max_vocab_size self.pretrained_files = pretrained_files def __eq__(self, other): if isinstance(self, other.__class__): return self.__dict__ == other.__dict__ return False Effective Modern Python 2018 13 / 19
14. Packaging and Dependency Management 14 / 19
15. How to make your program pip installable? Other languages have great tools Scala: sbt JavaScript: npm, yarn Rust: cargo Go: dep Python: ... Effective Modern Python 2018 15 / 19
16. Ancient Way Dump all installed packages and force to download them all: $ pip freeze > requirements.txt $ pip install -r requirements.txt Need to manage virtual environment at your own responsibility Don't you have unnecessary dependency? How is version compatibility of Python and libraries? Metadata management? Effective Modern Python 2018 16 / 19
17. Medieval Way Use standard Setuptools Messy documentation Need to write messy setup.py Effective Modern Python 2018 17 / 19
18. Modern Way Use Pipenv, Hatch, Flit, or Poetry Work as another pip Clean documentation I personally recommend Poetry $ poetry init $ poetry add chainer # equivalent to `pip install chainer` $ poetry build # make wheels $ poetry publish # upload to PyPI # pip installable after this command Effective Modern Python 2018 18 / 19
19. Further Reading Type Annotation The other (great) benefit of Python type annotations Data Classes The Ultimate Guide to Data Classes in Python 3.7 – Real Python Packaging and Dependency Management Python's New Package Landscape How to Publish an Open­Source Python Package to PyPI – Real Python Effective Modern Python 2018 19 / 19