QCon2010-douban-py4web

洪教授的经验分享

Transcript

1. Python ႿWeb 2.0ຩᅟ֥ႋႨ ޠ఼ୡ QCon Beijing 2010 http://www.flickr.com/photos/arnolouise/2986467632/
2. About Me • Pythonӱ྽ჴ • 2002୍ष൓ࢤԨPython • 2004୍ष൓ປಆ൐Ⴈ Python۽ቔ • http://www.douban.com/ people/hongqn/ • hongqn@douban.com • http://twitter.com/hongqn
3. Python • Python is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python and see almost immediate gains in productivity and lower maintenance costs. (via http://python.org/)
4. Languages in ׸ϵ ః෰(Pyrex/R/Erlang/Go/Shell) 1% C++ 3% Javascript 12% C 27% Python 58%
5. Why Python?
6. ࡥֆၞ࿐
7. ࡥֆၞ࿐ • Hello World: 1ٳᇒ
8. ࡥֆၞ࿐ • Hello World: 1ٳᇒ • ཬ۽ऎ࢖Ч: 1༯໶
9. ࡥֆၞ࿐ • Hello World: 1ٳᇒ • ཬ۽ऎ࢖Ч: 1༯໶ • ൌႨӱ྽: 1ᇛ
10. ࡥֆၞ࿐ • Hello World: 1ٳᇒ • ཬ۽ऎ࢖Ч: 1༯໶ • ൌႨӱ྽: 1ᇛ • ቓ۱׸ϵ: 3۱ᄅ
11. षؿ࿡ࢮ
12. षؿ࿡ࢮ ๤࠹۲ᇕე࿽֥ս઒ྛඔ: 13ྛ
13. षؿ࿡ࢮ import os from collections import defaultdict d = defaultdict(int) for dirpath, dirnames, filenames in os.walk('.'): for filename in filenames: path = os.path.join(dirpath, filename) ext = os.path.splitext(filename)[1] d[ext] += len(list(open(path))) for ext, n_lines in d.items(): print ext, n_lines ๤࠹۲ᇕე࿽֥ս઒ྛඔ: 13ྛ
14. ၞႿླྀቔ • ఼ᇅ෪ࣉЌᆣս઒ࢲܒౢ༉ၞ؀ • Pythonicх૧఼ਛ֥۱ದڄ۬
15. ҆ඇٚь • ഈཌ೘҆౷ 1. svn ci 2. svn up 3. restart
16. ൡႨ૫ܼ • WebႋႨ • ৖ཌ࠹ෘ • ᄎົ࢖Ч • ඔऌٳ༅
17. ሧჷپڶ • Battery Included: ѓሙ९ଽᇂ200+ଆॶ • PyPI: 9613 packages currently • ຩ઎/ඔऌ९/ም૫/Ⴓ༤/॓࿐࠹ෘ/νಆ/໓ Чԩ৘/... • easily extensible
18. ۷ᇗေ֥൞đ঺ᅵ္ ๷ࡩPython
19. ۷ᇗေ֥൞đ঺ᅵ္ ๷ࡩPython
20. ۷ᇗေ֥൞đ঺ᅵ္ ๷ࡩPython Just kidding :-p
21. ൕ২
22. Web Server
23. Web Server • python -m SimpleHTTPServer
24. Web Server • python -m SimpleHTTPServer
25. web.py import web urls = ( '/(.*)', 'hello' ) app = web.application(urls, globals()) class hello: def GET(self, name): if not name: name = 'World' return 'Hello, ' + name + '!' if __name__ == "__main__": app.run() http://webpy.org/
26. Flask import flask import Flask app = Flask(__name__) @app.route("/<name>") def hello(name): if not name: name = 'World' return 'Hello, ' + name + '!' if __name__ == "__main__": app.run() http://flask.pocoo.org/
27. WSGI http://www.python.org/dev/peps/pep-0333/
28. Why so many Python web frameworks? • Because you can write your own framework in 3 hours and a total of 60 lines of Python code. • http://bitworking.org/news/ Why_so_many_Python_web_frameworks
29. doctest def cube(x): """ >>> cube(10) 1000 """ return x * x def _test(): import doctest doctest.testmod() if __name__ == "__main__": _test()
30. nose http://somethingaboutorange.com/mrl/projects/nose/ from cube import cube def test_cube(): result = cube(10) assert result == 1000
31. numpy >>> from numpy import * >>> A = arange(4).reshape(2, 2) >>> A array([[0, 1], [2, 3]]) >>> dot(A, A.T) array([[ 1, 3], [ 3, 13]]) http://numpy.scipy.org/
32. ipython $ ipython -pylab In [1]: X = frange(0, 10, 0.1) In [2]: Y = [sin(x) for x in X] In [3]: plot(X, Y) http://numpy.scipy.org/
33. ipython $ ipython -pylab In [1]: X = frange(0, 10, 0.1) In [2]: Y = [sin(x) for x in X] In [3]: plot(X, Y) http://numpy.scipy.org/
34. virtualenv $ python go-pylons.py --no-site-packages mydevenv $ cd mydevenv $ source bin/activate (mydevenv)$ paster create -t new9 helloworld http://virtualenv.openplans.org/ Էࡹ၂۱ۄ֥࣪aۯ৖֥pythonߌ࣢
35. Pyrex/Cython cdef extern from "math.h" double sin(double) cdef double f(double x): return sin(x*x)
36. ᅽ࿐ Pythonic
37. >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. يၲğঠႧݽ http://bit.ly/pyzencn Ⴊૅ഻ႿԘੑ ૼਔ഻Ⴟ߳೥ ࡥࢱ഻Ⴟگᄖ گᄖ഻Ⴟ਱੹ ы௜഻Ⴟళส ࡗۯ഻Ⴟࣅ՘ ॖ؀ྟޓᇗေ ࠧьࡌࢹห২֥ൌႨྟᆭ଀đ္҂ॖ ິМᆃུܿᄵ ҂ေЇಸ෮Ⴕհ༂đԢ٤୆ಒקླေ ᆃဢቓ
38. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! ֒թᄝ؟ᇕॖିđ҂ေӇ൫ಀ҉ҩ ط൞࣐ਈᅳ၂ᇕđቋݺ൞ື၂၂ᇕ ૼཁ֥ࢳथٚσ ෙಖᆃѩ҂ಸၞđၹູ୆҂൞ Python ᆭڳ ቓ္ྸݺݖ҂ቓđ֌҂ࡌන෬ࣼ׮ ൭ߎ҂ೂ҂ቓ ೂݔ୆໭مཟದ૭ඍ୆֥ٚσđପ ड़ק҂൞၂۱ݺٚσĠّᆭၧಖ ଁ଀ॢࡗ൞၂ᇕध૳֥৘୑đ໡ૌ ႋ֒؟ࡆ০Ⴈ
39. Simple is better than complex class HelloWorld { public static void main(String args[]) { System.out.println("Hello World!"); } }
40. Simple is better than complex print "Hello World!"
41. Readability counts
42. Readability counts • ఼ᇅॶ෪ࣉđીႵ{}ބend
43. Readability counts • ఼ᇅॶ෪ࣉđીႵ{}ބend • ીႵٮࢳ֥ሳژ (except "@" for decorators)
44. Readability counts • ఼ᇅॶ෪ࣉđીႵ{}ބend • ીႵٮࢳ֥ሳژ (except "@" for decorators) if limit is not None and len(ids)>limit: ids = random.sample(ids, limit)
45. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)
46. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2)
47. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2)
48. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2) b = [] for x in a: b.append(x*2)
49. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2) b = [] for x in a: b.append(x*2)
50. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) b = [x*2 for x in a]
51. http://twitter.com/hongqn/status/9883515681 http://twitter.com/robbinfan/status/9879724095
52. Ⴕ๭Ⴕᆇཌྷ Python C http://www.flickr.com/photos/nicksieger/281055485/ http://www.flickr.com/photos/nicksieger/281055530/
53. ु๭҂ඪ߅ Ruby http://www.flickr.com/photos/nicksieger/280661836/
54. ु๭҂ඪ߅ Java http://www.flickr.com/photos/nicksieger/280662707/
55. ০ႨPython֥ე࿽ห ྟࡥ߄षؿ
56. σ২ਬ
57. σ২ਬ • svnᇏЌӻಌസ஥ᇂđषؿᆀߌ࣢ބཌഈ ߌ࣢οླห২஥ᇂ
58. σ২ਬ • svnᇏЌӻಌസ஥ᇂđषؿᆀߌ࣢ބཌഈ ߌ࣢οླห২஥ᇂ • ஥ᇂᇏླေگކࢲܒඔऌ(ೂlist)
59. σ২ਬ • svnᇏЌӻಌസ஥ᇂđषؿᆀߌ࣢ބཌഈ ߌ࣢οླห২஥ᇂ • ஥ᇂᇏླေگކࢲܒඔऌ(ೂlist) • ؟۱஥ᇂ໓ࡱ + ҆ඇൈሱ׮ކѩĤ
60. σ২ਬ • svnᇏЌӻಌസ஥ᇂđषؿᆀߌ࣢ބཌഈ ߌ࣢οླห২஥ᇂ • ஥ᇂᇏླေگކࢲܒඔऌ(ೂlist) • ؟۱஥ᇂ໓ࡱ + ҆ඇൈሱ׮ކѩĤ • щཿ஥ᇂ໓ࡱ۬ൔparserĤ
61. MEMCACHED_ADDR = ['localhost:11211'] from local_config import * config.py
62. MEMCACHED_ADDR = ['localhost:11211'] from local_config import * config.py MEMCACHED_ADDR = [ 'frodo:11211', 'sam:11211', 'pippin:11211', 'merry:11211', ] local_config.py
63. MEMCACHED_ADDR = ['localhost:11211'] from local_config import * config.py MEMCACHED_ADDR = [ 'frodo:11211', 'sam:11211', 'pippin:11211', 'merry:11211', ] local_config.py ໓ࡱ଀ުሗ҂ູ.pyൈđ ္ॖ൐Ⴈexec
64. σ২၂ • ଖ်ུ૫сྶႚႵଖ۱ಃཋҌି٠໙
65. class GroupUI(object): def new_topic(self, request): if self.group.can_post(request.user): return new_topic_ui(self.group) else: request.response.set_status(403, "Forbidden") return error_403_ui(msg="ӮູཬቆӮჴҌିؿ๎") def join(self, request): if self.group.can_join(request.user): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
66. class GroupUI(object): @check_permission('post', msg="ӮູཬቆӮჴҌିؿ๎") def new_topic(self, request): return new_topic_ui(self.group) @check_permission('join', msg="҂ିࡆೆཬቆ") def join(self, request): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
67. decorator def print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _ @print_before_exec def double(x): print x*2 double(10)
68. decorator def print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _ @print_before_exec def double(x): print x*2 double(10) ൻԛğ decorated 20
69. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
70. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
71. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
72. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
73. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
74. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
75. class GroupUI(object): @check_permission('post', msg="ӮູཬቆӮჴҌିؿ๎") def new_topic(self, request): return new_topic_ui(self.group) @check_permission('join', msg="҂ିࡆೆཬቆ") def join(self, request): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
76. σ২ؽ • ൐Ⴈཨ༏ؒਙၳ҄טႨݦඔ
77. def send_notification_mail(email, subject, body): msg = MSG_SEND_MAIL + '\0' + email + '\0' + subject + '\0' + body mq.put(msg) def async_worker(): msg = mq.get() msg = msg.split('\0') cmd = msg[0] if cmd == MSG_SEND_MAIL: email, subject, body = msg[1:] fromaddr = 'no-reply@douban.com' email_body = make_email_body(fromaddr, email, subject, body) smtp = smtplib.SMTP('mail') smtp.sendmail(fromaddr, email, email_body) elif cmd == MSG_xxxx: ... elif cmd == MSG_yyyy: ...
78. @async def send_notification_mail(email, subject, body): fromaddr = 'no-reply@douban.com' email_body = make_email_body(fromaddr, email, subject, body) smtp = smtplib.SMTP('mail') smtp.sendmail(fromaddr, email, email_body)
79. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
80. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
81. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
82. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
83. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
84. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
85. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
86. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
87. σ২೘ • cacheݦඔᄎྛࢲݔ(SQL, گᄖᄎෘ, etc)
88. def get_latest_review_id(): review_id = mc.get('latest_review_id') if review_id is None: review_id = exc_sql("select max(id) from review") mc.set('latest_review_id', review_id) return review_id
89. @cache('latest_review_id') def get_latest_review_id(): return exc_sql("select max(id) from review")
90. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
91. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
92. def get_review(id): key = 'review:%s' % id review = mc.get(key) if review is None: # cache miss id, author_id, text = exc_sql("select id, author_id, text from review where id=%s", id) review = Review(id, author_id, text) mc.set(key, review) return review ೂݔcache keyླေ׮෿ളӮ଻Ĥ
93. ླေ׮෿ളӮ֥cache keyھೂޅཿ decoratorĤ @cache('review:{id}') def get_review(id): id, author_id, text = exc_sql("select id, author_id, text from review where id=%s", id) return Review(id, author_id, text)
94. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
95. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
96. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
97. inspect.getargspec >>> import inspect >>> def f(a, b=1, c=2): ... pass ... >>> inspect.getargspec(f) ArgSpec(args=['a', 'b', 'c'], varargs=None, keywords=None, defaults=(1, 2)) >>> >>> >>> def f(a, b=1, c=2, *args, **kwargs): ... pass ... >>> inspect.getargspec(f) ArgSpec(args=['a', 'b', 'c'], varargs='args', keywords='kwargs', defaults=(1, 2))
98. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
99. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco hint: • str.format in python 2.6: '{id}'.format(id=1) => '1' • dict(zip(['a', 'b', 'c'], [1, 2, 3])) => {'a': 1, 'b': 2, 'c': 3}
100. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco hint: • str.format in python 2.6: '{id}'.format(id=1) => '1' • dict(zip(['a', 'b', 'c'], [1, 2, 3])) => {'a': 1, 'b': 2, 'c': 3}
101. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
102. σ২ඹ • feedᄇ؀ఖ๝ൈཁൕ؟۱feed֥໓ᅣđο entry_idކѩஆ྽b
103. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
104. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
105. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
106. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10] ඔऌ९Ұ࿘ྛඔ = len(self.feeds) * limit
107. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10] শٮ֥Entry.getඔ = len(self.feeds-1) * limit
108. iterator and generator def fib(): x, y = 1, 1 while True: yield x x, y = y, x+y def odd(seq): return (n for n in seq if n%2) def less_than(seq, upper_limit): for number in seq: if number >= upper_limit: break yield number print sum(odd(less_than(fib(), 4000000)))
109. itertools • count([n]) --> n, n+1, n+2 • cycle(p) --> p0, p1, ... plast, p0, p1, ... • repeat(elem [,n]) --> elem, elem, elem, ... endless or up to n times • izip(p, q, ...) --> (p[0], q[0]), (p[1], q[1]), ... • islice(seq, [start,] stop [, step]) --> elements from seq[start:stop:step] • ... and more ...
110. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
111. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
112. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
113. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
114. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
115. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit)) ඔऌ९Ұ࿘ྛඔ = len(self.feeds) * 5 ~ len(self.feeds)*5 + limit -5
116. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit)) শٮ֥Entry.getඔ = 0
117. decorator ބ generator ൞ࡥ߄ս઒֥০ఖ
118. σ২໴ • Ⴊ߄҂ॖэؓའّ྽ਙ߄ൈࡗ
119. class User(object): def __init__(self, id, username, screen_name, sig): self.id = id self.username = username self.screen_name = screen_name self.sig = sig user = User('1002211', 'hongqn', 'hongqn', "ϘϘ҃aϘϘ҃Ϙ҃ Ϙ҃Ć")
120. $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' \ > 'loads(s)' 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop cPickle vs. marshal
121. $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' \ > 'loads(s)' 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop cPickle vs. marshal 7П෎؇ิശ
122. $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' \ > 'loads(s)' 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop cPickle vs. marshal 7П෎؇ิശ
123. $ python -c ' > import cPickle, marshal > from user import user > print "pickle:", len(cPickle.dumps(user, 2)) > print "marshal:", len(marshal.dumps((user.id, \ > user.username, user.screen_name, user.sig), 2))' pickle: 129 marshal: 74 cPickle vs. marshal timeit 43%ॢࡗࢫസ
124. $ python -c ' > import cPickle, marshal > from user import user > print "pickle:", len(cPickle.dumps(user, 2)) > print "marshal:", len(marshal.dumps((user.id, \ > user.username, user.screen_name, user.sig), 2))' pickle: 129 marshal: 74 cPickle vs. marshal timeit 43%ॢࡗࢫസ
125. namedtuple from collections import namedtuple User = namedtuple('User', 'id username screen_name sig') user = User('1002211', 'hongqn', 'hongqn', sig="ϘϘ҃aϘϘ҃ Ϙ҃Ϙ҃Ć") user.username -> 'hongqn'
126. __metaclass__ class User(tuple): __metaclass__ = NamedTupleMetaClass __attrs__ = ['id', 'username', 'screen_name', 'sig'] user = User('1002211', 'hongqn', 'hongqn', sig="ϘϘ҃aϘϘ҃ Ϙ҃Ϙ҃Ć") s = marshal.dumps(user.__marshal__()) User.__load_marshal__(marshal.loads(s))
127. from operator import itemgetter class NamedTupleMetaClass(type): def __new__(mcs, name, bases, dict): assert bases == (tuple,) for i, a in enumerate(dict['__attrs__']): dict[a] = property(itemgetter(i)) dict['__slots__'] = () dict['__marshal__'] = tuple dict['__load_marshal__'] = classmethod(tuple.__new__) dict['__getnewargs__'] = lambda self: tuple(self) argtxt = repr(tuple(attrs)).replace("'", "")[1:-1] template = """def newfunc(cls, %(argtxt)s): return tuple.__new__(cls, (%(argtxt)s))""" % locals() namespace = {} exec template in namespace dict['__new__'] = namespace['newfunc'] return type.__new__(mcs, name, bases, dict)
128. Warning!
129. σ২ੂ • ࡥ߄request.get_environ(key)֥ཿم • e.g. request.get_environ('REMOTE_ADDR') --> request.remote_addr
130. descriptor • ၂۱ऎႵ__get__, __set__ࠇᆀ __delete__ٚم֥ؓའ class Descriptor(object): def __get__(self, instance, owner): return 'descriptor' class Owner(object): attr = Descriptor() owner = Owner() owner.attr --> 'descriptor'
131. ӈႨ֥descriptor • classmethod • staticmethod • property class C(object): def get_x(self): return self._x def set_x(self, x): self._x = x x = property(get_x, set_x)
132. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default) class HTTPRequest(quixote.http_request.HTTPRequest): for key in ['HTTP_REFERER', 'REMOTE_ADDR', 'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']: locals()[key.lower()] = environ_getter(key) del key locals()
133. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default) class HTTPRequest(quixote.http_request.HTTPRequest): for key in ['HTTP_REFERER', 'REMOTE_ADDR', 'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']: locals()[key.lower()] = environ_getter(key) del key
134. σ২௾ • ಞ urllib.urlopen ሱ׮০Ⴈsocksս৘ي఺
135. Monkey Patch
136. import httplib orig_connect = httplib.HTTPConnection.connect def _patched_connect(self): if HOSTS_BLOCKED.match(self.host): return _connect_via_socks_proxy(self) else: return orig_connect(self) def _connect_via_socks_proxy(self): ... httplib.HTTPConnection.connect = _patched_connect
137. ൐ႨPythonൈླေᇿ ၩ֥໙ี
138. ൐ႨPythonൈླေᇿ ၩ֥໙ี • Pythonic!
139. ൐ႨPythonൈླေᇿ ၩ֥໙ี • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html
140. ൐ႨPythonൈླေᇿ ၩ֥໙ี • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding
141. ൐ႨPythonൈླေᇿ ၩ֥໙ี • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding • GIL (Global Interpreter Lock)
142. ൐ႨPythonൈླေᇿ ၩ֥໙ี • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding • GIL (Global Interpreter Lock) • Garbage Collection
143. षؿߌ࣢ • щࠠఖ: Vim / Emacs / Ulipad • ϱЧܵ৘: subversion / mercurial / git • wiki/հ༂۵ሶ/ս઒ᛍফ: Trac • ӻ࿃ࠢӮ: Bitten
144. Python Implementations
145. Python Implementations • CPython http://www.python.org/
146. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/
147. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/
148. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/
149. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/ • Jython http://www.jython.org/
150. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/ • Jython http://www.jython.org/ • PyPy http://pypy.org/
151. ۋ྆ݓࡅđۋ྆նࡅ Q & A