Pythonic

# Pythonic

Slides : https://speakerdeck.com/dreampuf/pythonic

## Agenda

- Python idiom
- syntax, libraries
- Python tool
- virtualenv, pip, pyreverse, SimpleHTTPServer, Ipython
- Python resource
- PEP, pycoder's weekly, python book

## Python idiom

- What the heck does “pythonic” mean?

To be Pythonic is to use the Python constructs and datastructures with clean, readable idioms.
- 'There should be one-- and preferably only one --obvious way to do it' - from 'The Zen of Python' by Tim Peters

### Syntax

### Library

## Python tool

### VirtualEnv

### Pip

- Why Native Packages at All ?
- It doesn’t scale.
- It’s hard to integrate.
- You have to install build tools on target servers.
it’s also a security feature: Many attacks require a working C compiler.
- It can leave your app in an inconsistent state.

## Reference

- [What the heck does "pythonic" mean?](http://halitalptekin.tumblr.com/post/30028271874/pythonic-syntax)
- [Python Objects, types, classes, instances - a glossary](http://eli.thegreenplace.net/2012/03/30/python-objects-types-classes-and-instances-a-glossary/)
- [Python Application Deployment with Native Packages](http://hynek.me/articles/python-app-deployment-with-native-packages/)
- [Writing Idiomatic Python](http://www.jeffknupp.com/blog/2012/10/04/writing-idiomatic-python/)
- [Presentation: pip and virtualenv](http://mathematism.com/2009/07/30/presentation-pip-and-virtualenv/)
- [Welcome to The Hitchhiker’s Guide to Packaging](http://guide.python-distribute.org)
- [Index of Python Enhancement Proposals (PEPs)](http://www.python.org/dev/peps/)
- [PyCoder's Weekly](http://pycoders.com)
- [Google Python Style Guide](http://google-styleguide.googlecode.com/svn/trunk/pyguide.html)
- [Python Guide](http://python-guide.org)
- [PythonBook](http://pythonbooks.revolunet.com)
- [Python Tutor](http://www.pythontutor.com)

Transcript

1. Pythonic Dreampuf Nov. 2012
2. Agenda • Python idiom • Python tools • Python Resources
3. Python idiom There should be one, and preferably only one, obvious way to do it - from 'The Zen of Python' by Tim Peters
4. Python idiom To be Pythonic is to use the Python constructs and datastructures with clean, readable idioms. What the heck does “pythonic” mean?
5. Python idiom >>> import this
6. Python idiom Syntax & Builtin
7. Python idiom temp = foo foo = bar bar = temp (foo, bar) = (bar, foo)
8. Python idiom log_severity = None if 'severity' in configuration: log_severity = configuration['severity'] else: log_severity = log.Info log_severity = \ configuration.get('severity', log.Info)
9. Python idiom result_list = ['True', 'False', 'File not found'] result_string = '' for result in result_list: result_string += result result_list = ['True', 'False', 'File not found'] result_string = ''.join(result_list)
10. Python idiom file_handle = open(path_to_file, 'r') for line in file_handle.readlines(): if some_function_that_throws_exceptions(line): # do something file_handle.close() with open(path_to_file, 'r') as file_handle: for line in file_handle: if some_function_that_throws_exceptions(line): # do something # No need to explicitly call 'close'.
11. Python idiom if name == 'Tom' or name == 'Dick' or name == 'Harry': is_generic_name = True if name in ('Tom', 'Dick', 'Harry'): is_generic_name = True
12. Python idiom def Profile_view(request): if not check_is_logined(request): throw HTTP401 ... def Setting_view(request): if not check_is_logined(request): throw HTTP401 ... @check_is_logined_otherwise_throw def Profile_view(request): ... @check_is_logined_otherwise_throw def Setting_view(request): ...
13. Python idiom def Profile_view(request): if not check_is_logined(request): throw HTTP401 ... def Setting_view(request): if not check_is_logined(request): throw HTTP401 ... @check_is_logined_otherwise_throw def Profile_view(request): ... @check_is_logined_otherwise_throw def Setting_view(request): ... Setting_view = check_is_logined_otherwise_throw(Setting_view)
14. Python idiom some_other_list = range(1, 100) my_weird_list_of_numbers = list() for element in some_other_list: if is_prime(element): my_weird_list_of_numbers.append(element+5) some_other_list = range(1, 100) my_weird_list_of_numbers = \ [element + 5 for element in some_other_list if is_prime(element)]
15. Python idiom index = 0 for element in my_container: print (index, element) index+=1 for index, element in enumerate(my_container): print (index, element)
16. Python idiom def list_gen(arg): ls = [] for i in do_some_thing(arg): ls.append(i) return ls def list_gen(arg): for i in do_some_thing(arg): yield i
17. Python idiom ls = [1,2,3,4,5,6] for i in range(len(ls)): ls[i] += 3 map(lambda x: x+3, ls)
18. Python idiom acc = 10 for i in (1,2,3): acc = acc ** i reduce(lambda x,y:x**y, [1,2,3], 10)
19. Python idiom ls = [1,2,3,4,5,6] new_ls = [] for i in range(len(ls)): if ls[i] % 2 == 0: new_ls.append(ls[i]) filter(lambda x: x % 2 == 0, ls)
20. Python idiom filter(lambda x: x % 2 == 0, ls) map(lambda x: x+3, ls) [i+3 for i in ls] [i for i in ls if i % 2 == 0]
21. Python idiom itertools
22. Itertools Iterator Arguments Results Example count() start, [step] start, start+step, start+2*step, ... count(10) --> 10 11 12 13 14 ... cycle() p p0, p1, ... plast, p0, p1, ... cycle('ABCD') --> A B C D A B C D ... repeat() elem [,n] elem, elem, elem, ... endlessly or up to n times repeat(10, 3) --> 10 10 10 chain() p, q, ... p0, p1, ... plast, q0, q1, ... chain('ABC', 'DEF') --> A B C D E F compress() data, selectors (d[0] if s[0]), (d[1] if s[1]), ... compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F dropwhile() pred, seq seq[n], seq[n+1], starting when pred fails dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1 product('ABCD', repeat=2) p, q, ... [repeat=1] AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD
23. Itertools.groupby [('"AS-IS".', ['"AS-IS".']), ('"Defect"', ['"Defect"']), ('"Pro-', ['"Pro-']), ('"Project', ['"Project']), ('"Right', ['"Right']), ('"Small', ['"Small', '"Small']), ('"small', ['"small']), ('#1787]', ['#1787]']), ("&c.'", ["&c.'"]), ("''Tis", ["''Tis"]), ("'A", ["'A", "'A", "'A", "'A", "'A"]), ... ) keywords = groupby(sorted(hamlet.split())) [('the', 970), ('and', 708), ('of', 666), ('to', 632), ('I', 521), ('a', 466), ('my', 444), ('in', 391), ('you', 383), ('Ham.', 358), ('is', 318), ('his', 284), ('it', 274), ('not', 260), ... ]
24. Python idiom collections
25. collections deque list-like container with fast appends and pops on either end >= 2.4 Counter dict subclass for counting hashable objects >= 2.7 defaultdict dict subclass that calls a factory function to supply missing values >= 2.5 OrderedDict dict subclass that remembers the order entries were added >= 2.7 namedtuple() factory function for creating tuple subclasses with named fields >= 2.6
26. collections.deque $ python -m timeit \ -s 'from collections import deque; ls = deque();' \ 'ls.appendleft(1);' 10000000 loops, best of 3: 0.126 usec per loop $ python -m timeit -s 'ls = [];' 'ls.insert(1, 0);' 100000 loops, best of 3: 41.9 usec per loop
27. collections.Counter >>> # Tally occurrences of words in a list >>> cnt = Counter() >>> for word in ['red', 'blue', 'red', 'green', 'blue', 'blue']: ... cnt[word] += 1 >>> cnt Counter({'blue': 3, 'red': 2, 'green': 1}) >>> # Find the ten most common words in Hamlet >>> import re >>> words = re.findall('\w+', open('hamlet.txt').read().lower()) >>> Counter(words).most_common(10) [('the', 1143), ('and', 966), ('to', 762), ('of', 669), ('i', 631), ('you', 554), ('a', 546), ('my', 514), ('hamlet', 471), ('in', 451)]
28. collections.defaultdict >>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)] >>> d = defaultdict(list) >>> for k, v in s: ... d[k].append(v) ... >>> d.items() [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
29. Python idiom unittest
31. Unittest $ python test.py ... ---------------------------------------------------------------------- Ran 3 tests in 0.000s OK $ python test.py TestSequenceFunctions $ python test.py TestSequenceFunctions.test_choice
32. Python idiom functools
33. functools.wraps >>> from functools import wraps >>> def my_decorator(f): ... @wraps(f) ... def wrapper(*args, **kwds): ... print 'Calling decorated function' ... return f(*args, **kwds) ... return wrapper ... >>> @my_decorator ... def example(): ... """Docstring""" ... print 'Called example function' ... >>> example() Calling decorated function Called example function >>> example.__name__ 'example' >>> example.__doc__ 'Docstring'
34. functools.partial >>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo.__doc__ = 'Convert base 2 string to an int.' >>> basetwo('10010') 18 >>> int("1001") 1001 >>> int("1001", base=2) 9
35. Python idiom contextlib
36. contextlib.contextmanager from contextlib import contextmanager @contextmanager def tag(name): print "<%s>" % name yield print "</%s>" % name >>> with tag("h1"): ... print "foo" ... <h1> foo </h1>
37. contextlib.closing from contextlib import closing import urllib with closing(urllib.urlopen('http://www.python.org')) as page: for line in page: print line from contextlib import contextmanager @contextmanager def closing(thing): try: yield thing finally: thing.close()
38. Python tools
39. Python tool virtualenv
40. Virtualenv virtualenv is a tool to create isolated Python environments.
41. Virtualenv usage $ python >>> import sys, pprint >>> pprint.pprint(sys.path) ['', '/usr/lib/python2.6', '/home/dreampuf/.local/lib/python2.6/site-packages', '/usr/lib/python2.6/dist-packages', …] $ source test-env/bin/activate (test-env)$ python >>> import sys, pprint >>> pprint.pprint(sys.path) ['', '/home/dreampuf/test-env/lib/python2.6', '/home/dreampuf/test-env/lib/python2.6/site-packages', '/usr/lib/python2.6', '/usr/lib/python2.6/dist-packages', …] (test-env)$ deactive $
42. Python tool pip
43. Pip pip is a tool for installing and managing Python packages, such as those found in the Python Package Index. It's a replacement for easy_install.
44. Pip usage $ pip install simplejson [... progress report ...] Successfully installed simplejson $ pip install --upgrade simplejson [... progress report ...] Successfully installed simplejson $ pip uninstall simplejson Uninstalling simplejson: /home/me/env/lib/python2.7/site-packages/simplejson /home/me/env/lib/python2.7/site-packages/ simplejson-2.2.1-py2.7.egg-info Proceed (y/n)? y Successfully uninstalled simplejson Install Upgrade Removing
45. Pip usage (.py)$ pip freeze GitPython==0.3.2.RC1 PyYAML==3.10 async==0.6.1 gitdb==0.5.4 ipdb==0.7 ipython==0.13.1 nose==1.2.1 pycallgraph==0.5.1 pymongo==2.3 pythoscope==0.4.3 readline==6.2.4.1 ...
46. Virtualenv + Pip $ git clone code_stat/ code_stat_clone Cloning into 'code_stat_clone'... done. $ cd code_stat_clone/ $ ls README.md config.py mongo_stat.py requirements.txt test.py $ virtualenv .py New python executable in .py/bin/python Installing setuptools............done. Installing pip...s............done. $ source .py/bin/activate (.py)$ pip install -r requirements.txt Downloading/unpacking GitPython==0.3.2.RC1 (from -r requirements.txt (line 1)) Downloading/unpacking PyYAML==3.10 (from -r requirements.txt (line 2)) Downloading/unpacking async==0.6.1 (from -r requirements.txt (line 3)) .... (.py)$
47. Python tool pyreverse
48. Pyreverse Pyreverse is a set of utilities to reverse enginering Python code. It uses a representation of a Python project in a class hierarchy which can be used to extract any information (such as generating UML diagrams or unit tests, as pyargo and py2tests)
49. Pyreverse usage $pyreverse -o png -ASmy \ -f PUB_ONLY -p cherrypy \ cherrypy/
50. Pyreverse usage
51. Pyreverse usage
52. Python tool SimpleHTTPServer
53. SimpleHTTPServer usage $ python -m SimpleHTTPServer Serving HTTP on 0.0.0.0 port 8000 ...
54. SimpleHTTPServer usage
55. SimpleHTTPServer usage NEVER USE IN PRODUCTION
56. Python tool IPython
57. IPython IPython provides a rich toolkit to help you make the most out of using Python
58. IPython • Tab completion • Exploring your objects • Magic functions • Running and Editing • Debugging • History • System shell commands
59. IPython $ ipython In [1]: dict.<TAB> dict.clear dict.get dict.iteritems dict.keys dict.popitem dict.values dict.viewvalues dict.copy dict.has_key dict.iterkeys dict.mro dict.setdefault dict.viewitems dict.fromkeys dict.items dict.itervalues dict.pop dict.update dict.viewkeys In [1]: %timeit range(100) 1000000 loops, best of 3: 978 ns per loop In [2]: %timeit xrange(100) 1000000 loops, best of 3: 263 ns per loop
60. Python tool pdb
61. pdb (.py)$ python -m pdb mongo_stat.py > mongo_stat.py(4)<module>() -> __author__ = "jackdeng@freewheel.tv, byu@freewheel.tv, xhuang@freewheel.tv" (Pdb) n > mongo_stat.py(10)<module>() -> import re (Pdb) > mongo_stat.py(11)<module>() -> import os (Pdb) l 6 """ 7 Code Stat 8 """ 9 10 import re 11 -> import os 12 import sys 13 import time 14 import shlex (Pdb)
62. pdb ipdb = pdb + ipython
63. pudb
64. Python tool pyflakes
65. Pyflakes
66. Python Resources
67. Python Enhancement Proposals (PEPs) num title owner 1 PEP Purpose and Guidelines Warsaw, Hylton, Goodger, Coghlan 4 Deprecation of Standard Modules von Löwis 5 Guidelines for Language Evolution Prescod 6 Bug Fix Releases Aahz, Baxter 7 Style Guide for C Code GvR 8 Style Guide for Python Code GvR, Warsaw 9 Sample Plaintext PEP Template Warsaw 10 Voting Guidelines Warsaw
68. Pydoc $ pydoc enumerate Help on class enumerate in module __builtin__: class enumerate(object) | enumerate(iterable[, start]) -> iterator for index, value of iterable | | Return an enumerate object. iterable must be another object that supports | iteration. The enumerate object yields pairs containing a count (from | start, which defaults to zero) and a value yielded by the iterable argument. | enumerate is useful for obtaining an indexed list: | (0, seq[0]), (1, seq[1]), (2, seq[2]), ...
69. Python Book • A byte of Python (“Python 简明教程”) • Python Module of the Week • Expert Python Programming • Python Cookbook • http://pythonbooks.revolunet.com
70. pythonbooks.revolunet.com
71. pythonbooks.revolunet.com
72. Pycoder’s Weekly
73. python-guide.org This opinionated guide exists to provide both novice and expert Python developers a best-practice handbook to the installation, configuration, and usage of Python on a daily basis. -- Kenneth Reitz(Heroku dev)
74. Python tutor
75. Google Python Style
76. Python Resources huangxin.py 有点⼉儿欢乐
77. Q&A
78. Q&A
79. Reference(1) • - [What the heck does "pythonic" mean?](http://halitalptekin.tumblr.com/post/ 30028271874/pythonic-syntax) • - [Python Objects, types, classes, instances - a glossary](http://eli.thegreenplace.net/ 2012/03/30/python-objects-types-classes-and-instances-a-glossary/) • - [Python Application Deployment with Native Packages](http://hynek.me/articles/ python-app-deployment-with-native-packages/) • - [Writing Idiomatic Python](http://www.jeffknupp.com/blog/2012/10/04/writing- idiomatic-python/) • - [Presentation: pip and virtualenv](http://mathematism.com/2009/07/30/ presentation-pip-and-virtualenv/) • - [Welcome to The Hitchhiker’s Guide to Packaging](http://guide.python- distribute.org) • - [Index of Python Enhancement Proposals (PEPs)](http://www.python.org/dev/ peps/)
80. Reference(2) • - [PyCoder's Weekly](http://pycoders.com) • - [Google Python Style Guide](http://google-styleguide.googlecode.com/svn/trunk/ pyguide.html) • - [Python Guide](http://python-guide.org) • - [PythonBook](http://pythonbooks.revolunet.com) • - [Python Tutor](http://www.pythontutor.com)