Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effective Modern Python 2018

4883f191bcdbd6a260fa02e4c4fce28f?s=47 Ryo Takahashi
December 06, 2018

Effective Modern Python 2018

4883f191bcdbd6a260fa02e4c4fce28f?s=128

Ryo Takahashi

December 06, 2018
Tweet

Transcript

  1. Effective Modern Python 2018 Ryo Takahashi 1 / 19

  2. Contents Type Annotation Data Classes Packaging and Dependency Management Effective

    Modern Python 2018 2 / 19
  3. Type Annotation python python 3.5 3.5 3 / 19

  4. Benefits of type annotations Static code analysis prior to runtime

    PyCharm (IntelliJ) mypy Self­documenting Code itself is a document Code completion Effective Modern Python 2018 4 / 19
  5. Chainer RNNLM Example train, val, test = chainer.datasets.get_ptb_words() Type of

    train, val, test? What kind of operations can be done on these variables? Effective Modern Python 2018 5 / 19
  6. Effective Modern Python 2018 6 / 19

  7. Go to Declaration def get_ptb_words(): """Gets the Penn Tree Bank

    dataset as long word sequences. [...] Returns: tuple of numpy.ndarray: Int32 vectors of word IDs. """ train = _retrieve_ptb_words('train.npz', _train_url) valid = _retrieve_ptb_words('valid.npz', _valid_url) test = _retrieve_ptb_words('test.npz', _test_url) return train, valid, test docstring IDEs can't parse complicated types Effective Modern Python 2018 7 / 19
  8. Add Type Annotations from typing import Dict def load_vocab(vocabfile: str)

    -> Dict[int, str]: [...] # typed! vocab = load_vocab(vocabfile) Effective Modern Python 2018 8 / 19
  9. Add Type Annotations If you can't modify function declaration... #

    function of other libraries def load_vocab(vocabfile): [...] from typing import Dict # typed! vocab: Dict[int, str] = load_vocab(vocabfile) Effective Modern Python 2018 9 / 19
  10. Effective Modern Python 2018 10 / 19

  11. Data Classes python python 3.7 3.7 11 / 19

  12. Easier Way to Write Classes from dataclasses import dataclass from

    typing import Dict, Optional @dataclass class Vocabulary: padding_token: str oov_token: str max_vocab_size: int pretrained_files: Optional[Dict[str, str]] = None Similar to Scala's Case Classes Generate __init__, __eq__, and __repr__ automatically Effective Modern Python 2018 12 / 19
  13. Conventional Class class Vocabulary: def __init__( self, padding_token: str, oov_token:

    str, max_vocab_size: int, pretrained_files: Optional[Dict[str, str]] = None, ): self.padding_token = padding_token self.oov_token = oov_token self.max_vocab_size = max_vocab_size self.pretrained_files = pretrained_files def __eq__(self, other): if isinstance(self, other.__class__): return self.__dict__ == other.__dict__ return False Effective Modern Python 2018 13 / 19
  14. Packaging and Dependency Management 14 / 19

  15. How to make your program pip installable? Other languages have

    great tools Scala: sbt JavaScript: npm, yarn Rust: cargo Go: dep Python: ... Effective Modern Python 2018 15 / 19
  16. Ancient Way Dump all installed packages and force to download

    them all: $ pip freeze > requirements.txt $ pip install -r requirements.txt Need to manage virtual environment at your own responsibility Don't you have unnecessary dependency? How is version compatibility of Python and libraries? Metadata management? Effective Modern Python 2018 16 / 19
  17. Medieval Way Use standard Setuptools Messy documentation Need to write

    messy setup.py Effective Modern Python 2018 17 / 19
  18. Modern Way Use Pipenv, Hatch, Flit, or Poetry Work as

    another pip Clean documentation I personally recommend Poetry $ poetry init $ poetry add chainer # equivalent to `pip install chainer` $ poetry build # make wheels $ poetry publish # upload to PyPI # pip installable after this command Effective Modern Python 2018 18 / 19
  19. Further Reading Type Annotation The other (great) benefit of Python

    type annotations Data Classes The Ultimate Guide to Data Classes in Python 3.7 – Real Python Packaging and Dependency Management Python's New Package Landscape How to Publish an Open­Source Python Package to PyPI – Real Python Effective Modern Python 2018 19 / 19