Slide 1

Slide 1 text

1 Programming with Python Adv. Topics Mosky

Slide 2

Slide 2 text

2 Mosky: ● The examples and the PDF version are available at: – j.mp/mosky-programming-with-python. ● It is welcome to give me any advice of this slide or ask me the answers of the challenges. – mosky.tw

Slide 3

Slide 3 text

3 Topics ● Basic Topics – Python 2 or 3? – Environment – hello.py – Common Types – Flow Control – File I/O – Documentation – Scope ● Adv. Topics – Module and Package – Typing – Comprehension – Functional Technique – Object-oriented Prog. – Useful Libraries ● Final Project – A Blog System

Slide 4

Slide 4 text

4 Module and Package Write you own module and package!

Slide 5

Slide 5 text

5 Module and Package ● A Python file is just a Python module: – import module_a # module_a.py ● A folder which has __init__.py is just a Python package: – import package_x # __init__.py – import package_x.module_b # package_x/module_b.py – from . import module_c # (in package_x.moudle_b) package_x/module_c.py – $ python -m package_x.module_b ● Do not name your file as any built-in module. – ex. sys.py

Slide 6

Slide 6 text

6 Module and Package (cont.) ● The tree: – . ├── moudle_a.py └── package_x ├── __init__.py ├── module_b.py └── module_c.py

Slide 7

Slide 7 text

7 The import Statement ● A module is only imported at the first import. ● import module module.val = 'modified' – The module is affected by this modification. ● from module import val val = 'modified' – The module is not affected by this modification. – It does a shallow copy.

Slide 8

Slide 8 text

8 Typing static? dynamic? weak? strong?

Slide 9

Slide 9 text

9 Static Typing ● Checking types in compile time. ● Usually, it is required to give a type to a variable. ● Python is not static typing. long x short y double z c2 c1 c3 c4 c5 c6 c7 c8

Slide 10

Slide 10 text

10 Dynamic Typing ● Checking types in run time. ● A variable just points to an object. ● Python is dynamic typing. x y object A object b object c NOTE: This is an animation and it is not correct in the PDF version.

Slide 11

Slide 11 text

11 Duck Typing

Slide 12

Slide 12 text

12 Duck Typing (cont.) ● A style of dynamic typing. ● Happy coding without the template, the generics … etc. ● If it is necessary to check type: – if hasattr(x, '__iter__'): ● adapt the type inputed – assert not hasattr(x, '__iter__'), 'x must be iterable' ● notify the programmer – if isinstance(x, basestring): ● the worst choice

Slide 13

Slide 13 text

13 Duck Typing (cont.) ● String and integer both support += operator. ● Write the code with elasticity. #!/usr/bin/env python # -*- coding: utf-8 -*- # file: ex_dyn.py def dynsum(*seq): r = seq[0] for item in seq[1:]: r += item return r if __name__ == '__main__': print dynsum(1, 2, 3) print dynsum('x', 'y', 'z')

Slide 14

Slide 14 text

14 Duck Typing (cont.) ● BUT, it will confuse you when your project is going to big. – Name your variables with hint of type. ● item vs. items ● employee vs. employee_name ● args vs. kargs – Documentation does matter.

Slide 15

Slide 15 text

15 Weak Typing ● It converts the type if you do an operation which is not supported by the original type. ● In JavaScript: – 1 + '1' → '11' – 1 == '1' → true ● Python is not weak typing!

Slide 16

Slide 16 text

16 Strong Typing ● Only do the operations which are supported by the original type. – 1 + '1' → TypeError – 1 == '1' → False ● Python is strong typing!

Slide 17

Slide 17 text

17 Comprehension Compact your statements.

Slide 18

Slide 18 text

18 List Comprehension [i for i in range(10)] [i ** 2 for i in range(10)] [f(i) for i in range(10)] [i for i in range(10) if i % 2 == 0] [i for i in range(10) if not i % 2 == 0] [i for i in range(10) if g(i)]

Slide 19

Slide 19 text

19 List Comprehension (cont.) List comprehension: [ (i, j) for i in range(3) for j in range(3) ] is equal to: r = [] for i in range(3): for j in range(3): r.append((i, j))

Slide 20

Slide 20 text

20 List Comprehension (cont.) List comprehension: [ [ (i, j) for i in range(3) ] for j in range(3) ] is equal to: r = [] for i in range(3): t = [] for j in range(3): t.append((i, j)) r.append(t)

Slide 21

Slide 21 text

21 Generator Comprehension ● Generator comprehension: – The examples: ● (i for i in range(10)) ● f(i for i in range(10)) – It is like xrange. – Lazy evaluation → Save memory.

Slide 22

Slide 22 text

22 Other Comprehensions Python 3 only: ● set comprehension: – {i for i in range(10)} ● dict comprehension: – {i:i for i in range(10)} But we can do so with below statements: ● set comprehension: – set(i for i in range(10)) ● dict comprehension: – dict((i, i) for i in range(10))

Slide 23

Slide 23 text

23 Functional Technique Think in the functional way.

Slide 24

Slide 24 text

24 The any/all Function ● def all_even(seq): return all(i % 2 == 0 for i in seq) ● all(type(item) is int for item in inputs) ● any(i < 0 for i in inputs)

Slide 25

Slide 25 text

25 Challenge 3-3: The Primes (cont.) – limit: in one line. ● hint: use any or all [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]

Slide 26

Slide 26 text

26 The zip Function ● matrix = [ [1, 2], [3, 4], ] ● zip(*matrix)

Slide 27

Slide 27 text

27 The zip Function (cont.) ● result = [ ('I am A.', 'email_A'), ('I am B.', 'email_B'), ] ● emails = zip(*result)[1]

Slide 28

Slide 28 text

28 First-class Functions #!/usr/bin/env python # -*- coding: utf-8 -*- # file: ex_do.py from operator import add, mul def do(action, x, y): return action(x, y) if __name__ == '__main__': print do(add, 10, 20) print do(mul, 10, 20) ● passing functions as arguments.

Slide 29

Slide 29 text

29 The lambda Expression ● lambda [args]: [expression] ● It defines an anonymous function. ● It only allows a single expression. ● f = lambda x: g(x)+h(x) ● do(lambda x, y: (x+y)*(x+y), 10, 20)

Slide 30

Slide 30 text

30 Use sort with Lambda ● d = dict(a=300, b=200, c=100) ● keys = d.keys() ● keys.sort(key=lambda k: d[k]) ● for k in keys: print k, d[k]

Slide 31

Slide 31 text

31 Use sort with Lambda (cont.) ● names = ['Andy', 'Bob', 'Cindy'] ● scores = [70, 100, 95] ● table = zip(names, scores) ● table.sort(key=lambda pair: pair[1]) ● for name, score in table: print name, score

Slide 32

Slide 32 text

32 The map Function ● map(lambda x: x**2, range(10)) ● map(int, '1 2 3'.split(' ')) ● map(ord, 'String') ● map(open, []) ● map(str.split, open())

Slide 33

Slide 33 text

33 The map Function (cont.) ● from operator import mul ● a = (1, 2) ● b = (3, 4) ● sum(map(mul, a, b))

Slide 34

Slide 34 text

34 The filter Function ● filter(lambda i: i % 2 == 0, range(10)) ● filter(str.isdigit, strings) ● filter(lambda s: s.endswith('.py'), file_names)

Slide 35

Slide 35 text

35 Comprehension vs. map/filter ● [i ** 2 for i in range(10)] ● map(lambda i: i ** 2, range(10)) ● [i ** 2 for i in range(10) if i % 2 == 0] ● map(lambda i: i ** 2, filter( lambda i: i % 2 == 0, range(10) ))

Slide 36

Slide 36 text

36 Comprehension vs. map/filter (cont.) ● [ord(c) for c in 'ABC'] ● map(ord, 'ABC')

Slide 37

Slide 37 text

37 Comprehension vs. map/filter (cont.) Compare the speeds of them: 1. map/filter (with built-in function) 2. comprehension 3. map/filter

Slide 38

Slide 38 text

38 The reduce Function ● # from functools import reduce # py3 ● from operator import add ● seq = [-1, 0, 1] ● reduce(add, s) ● seq = ['reduce ', 'the ', 'lines.'] ● reduce(add, s)

Slide 39

Slide 39 text

39 The partial Function ● from functools import partial ● from operator import add ● rdsum = partial(reduce, add) ● rdsum([-1, 0, 1]) ● rdsum(['reduce ', 'the ', 'lines.'])

Slide 40

Slide 40 text

40 The partial Function (cont.) ● from functools import partial ● from fractions import gcd as _gcd ● _gcd(6, 14) ● gcd = partial(reduce, _gcd) ● gcd([6, 14, 26])

Slide 41

Slide 41 text

41 Closure ● from math import log ● def mklog(n): return lambda x: log(x, n) ● log10 = mklog(10) ● log10(100) # n = 10

Slide 42

Slide 42 text

42 Closure (cont.) ● setattr(DictLike, attrname, # it is a colsure (lambda x: property( lambda self: self.__getitem__(x), lambda self, v: self.__setitem__(x, v), lambda self: self.__delitem__(x) ) )(attrname) )

Slide 43

Slide 43 text

43 The yield Statement ● def mkgen(n): for i in range(n): yield i ** 2 ● It is a generator. ● gen = mkgen(10) ● for i in gen: print i

Slide 44

Slide 44 text

44 Decorator ● def deco(f): def f_wrapper(*args, **kargs): print 'DEBUG: ', args, kargs return f(*args, **kargs) return f_wrapper ● @deco # is equal to add = deco(add) def add(x, y): return x+y ● add(1, 2)

Slide 45

Slide 45 text

45 Object-oriented Programming is also available.

Slide 46

Slide 46 text

46 The class Statement class Example(object): class_attribute = 1 def method(self, …): pass example = Example() print example

Slide 47

Slide 47 text

47 The Class in Python (cont.) ● Everything in Python is object. – Class is an object, too. ● All class inherit the object → new-style classes – Use new-style classes. It provides more features. – Python 3: auto inherit the object. ● Supports multiple inheritance. – Searching attributes/methods is like BFS.

Slide 48

Slide 48 text

48 Bound and Unbound Method ● unbound method – def m(self, ...) – C.m(c, ...) ● bound method (instance method) – c.m(...)

Slide 49

Slide 49 text

49 Class Method and Static Method ● class method – @classmethod def m(cls, …) – C.m() – c.m() ● static method – @staticmethod def m(...) – C.m() – c.m()

Slide 50

Slide 50 text

50 The Data Model of Python ● Special methods – __init__ – __str__ – __repr__ – __getitem__ x[key] → – __setitem__ x[key] = value → – __delitem__ del x[key] → … ● ref: docs.python.org/2/reference/datamodel.html

Slide 51

Slide 51 text

51 Protocol ● It like interface, but it is only described in doc. ● The examples: – iterator protocol ● object which supports __iter__ and next – readable ● object which supports read – ...

Slide 52

Slide 52 text

52 The employee class ● see examples/ex_empolyee.py .

Slide 53

Slide 53 text

53 Do Math with Classes ● see examples/ex_do_math_with_classes/ .

Slide 54

Slide 54 text

54 Challenge 6: Give a Raise ● Give your employee a raise. – without limit – limit: prevent modifying salary by attribute. ● hint: use property cindy = Empolyee(...) cindy.add_salary(1000) print cindy.salary

Slide 55

Slide 55 text

55 Useful Libraries import antigravity

Slide 56

Slide 56 text

56 Useful Libraries The built-in libraies: – random – datetime – re – hashlib/hmac – decimal – glob – collections – subprocess – multiprocessing – gc – pickle – json – pprint – gzip – timeit – logging – unitest – doctest – pdb ...

Slide 57

Slide 57 text

57 Useful Libraries (cont.) The third-party libraries on PyPI: – Requests – Use it instead of the poor built-in urllib. – lxml – Do you need to parse HTML? Use it! – PyYAML – YAML is a the best of data serialization standards. – PIL – Python Image Library – NumPy and SciPy – are for mathematics, science, and engineering. – SymPy – is for symbolic mathematic – Bottle, Flask or Django – are the web frameworks. – Sphinx – helps you to create documentation. ...

Slide 58

Slide 58 text

58 Challenge 7: Iterable Interger ● Make integer iterable. – without limit – limit 1: don't use string – limit 2: use collection ● hint: use property x = IterableInt(10) for b in x: print b 0 1 0 1

Slide 59

Slide 59 text

59 Final Project : A Blog System

Slide 60

Slide 60 text

60 Final Project : A Blog System ● The cookbook: – Flask – A micro web framework. – Use pickle to store the posts. – Optional: ● A database instead of pickle. (ex. MySQL, PostgreSQL, ...) ● A cache layer. (ex. memcached, redis, ...) ● A message queue for async jobs. (ex. RabbitMQ, ...)

Slide 61

Slide 61 text

61 It is the end. Contact me? http://mosky.tw/