Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Programming with Python - Adv.

Programming with Python -ย Adv.

Python is a great programming language. It is a complete tutorial of using this programming language.

This slides is split into two parts, and it is the second part. Another part is at: https://speakerdeck.com/mosky/programming-with-python-basic.

Mosky Liu

May 20, 2013
Tweet

More Decks by Mosky Liu

Other Decks in Programming

Transcript

  1. 1
    Programming with Python
    Adv. Topics
    Mosky

    View Slide

  2. 2
    Mosky:
    โ—
    The examples and the PDF version are available at:
    โ€“ j.mp/mosky-programming-with-python.
    โ—
    It is welcome to give me any advice of this slide or
    ask me the answers of the challenges.
    โ€“ mosky.tw

    View Slide

  3. 3
    Topics
    โ—
    Basic Topics
    โ€“ Python 2 or 3?
    โ€“ Environment
    โ€“ hello.py
    โ€“ Common Types
    โ€“ Flow Control
    โ€“ File I/O
    โ€“ Documentation
    โ€“ Scope
    โ—
    Adv. Topics
    โ€“ Module and Package
    โ€“ Typing
    โ€“ Comprehension
    โ€“ Functional Technique
    โ€“ Object-oriented Prog.
    โ€“ Useful Libraries
    โ—
    Final Project
    โ€“ A Blog System

    View Slide

  4. 4
    Module and Package
    Write you own module and package!

    View Slide

  5. 5
    Module and Package
    โ—
    A Python file is just a Python module:
    โ€“ import module_a # module_a.py
    โ—
    A folder which has __init__.py is just a Python package:
    โ€“ import package_x # __init__.py
    โ€“ import package_x.module_b # package_x/module_b.py
    โ€“ from . import module_c
    # (in package_x.moudle_b) package_x/module_c.py
    โ€“ $ python -m package_x.module_b
    โ—
    Do not name your file as any built-in module.
    โ€“ ex. sys.py

    View Slide

  6. 6
    Module and Package (cont.)
    โ—
    The tree:
    โ€“ .
    โ”œโ”€โ”€ moudle_a.py
    โ””โ”€โ”€ package_x
    โ”œโ”€โ”€ __init__.py
    โ”œโ”€โ”€ module_b.py
    โ””โ”€โ”€ module_c.py

    View Slide

  7. 7
    The import Statement
    โ—
    A module is only imported at the first import.
    โ—
    import module
    module.val = 'modified'
    โ€“ The module is affected by this modification.
    โ—
    from module import val
    val = 'modified'
    โ€“ The module is not affected by this modification.
    โ€“ It does a shallow copy.

    View Slide

  8. 8
    Typing
    static? dynamic? weak? strong?

    View Slide

  9. 9
    Static Typing
    โ—
    Checking types in compile time.
    โ—
    Usually, it is required to give a type to a variable.
    โ—
    Python is not static typing.
    long x short y
    double z
    c2
    c1 c3 c4 c5 c6 c7 c8

    View Slide

  10. 10
    Dynamic Typing
    โ—
    Checking types in run time.
    โ—
    A variable just points to an object.
    โ—
    Python is dynamic typing.
    x
    y
    object A
    object b
    object c
    NOTE: This is an animation and it is not correct in the PDF version.

    View Slide

  11. 11
    Duck Typing

    View Slide

  12. 12
    Duck Typing (cont.)
    โ—
    A style of dynamic typing.
    โ—
    Happy coding without the template, the generics โ€ฆ etc.
    โ—
    If it is necessary to check type:
    โ€“ if hasattr(x, '__iter__'):
    โ—
    adapt the type inputed
    โ€“ assert not hasattr(x, '__iter__'), 'x must be iterable'
    โ—
    notify the programmer
    โ€“ if isinstance(x, basestring):
    โ—
    the worst choice

    View Slide

  13. 13
    Duck Typing (cont.)
    โ—
    String and integer both
    support += operator.
    โ—
    Write the code with
    elasticity.
    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    # file: ex_dyn.py
    def dynsum(*seq):
    r = seq[0]
    for item in seq[1:]:
    r += item
    return r
    if __name__ == '__main__':
    print dynsum(1, 2, 3)
    print dynsum('x', 'y', 'z')

    View Slide

  14. 14
    Duck Typing (cont.)
    โ—
    BUT, it will confuse you when your project is
    going to big.
    โ€“ Name your variables with hint of type.
    โ—
    item vs. items
    โ—
    employee vs. employee_name
    โ—
    args vs. kargs
    โ€“ Documentation does matter.

    View Slide

  15. 15
    Weak Typing
    โ—
    It converts the type if you do an operation which is
    not supported by the original type.
    โ—
    In JavaScript:
    โ€“ 1 + '1'
    โ†’ '11'
    โ€“ 1 == '1'
    โ†’ true
    โ—
    Python is not weak typing!

    View Slide

  16. 16
    Strong Typing
    โ—
    Only do the operations which are supported by the
    original type.
    โ€“ 1 + '1'
    โ†’ TypeError
    โ€“ 1 == '1'
    โ†’ False
    โ—
    Python is strong typing!

    View Slide

  17. 17
    Comprehension
    Compact your statements.

    View Slide

  18. 18
    List Comprehension
    [i for i in range(10)]
    [i ** 2 for i in range(10)]
    [f(i) for i in range(10)]
    [i for i in range(10) if i % 2 == 0]
    [i for i in range(10) if not i % 2 == 0]
    [i for i in range(10) if g(i)]

    View Slide

  19. 19
    List Comprehension (cont.)
    List comprehension:
    [
    (i, j)
    for i in range(3)
    for j in range(3)
    ]
    is equal to:
    r = []
    for i in range(3):
    for j in range(3):
    r.append((i, j))

    View Slide

  20. 20
    List Comprehension (cont.)
    List comprehension:
    [
    [
    (i, j)
    for i in range(3)
    ]
    for j in range(3)
    ]
    is equal to:
    r = []
    for i in range(3):
    t = []
    for j in range(3):
    t.append((i, j))
    r.append(t)

    View Slide

  21. 21
    Generator Comprehension
    โ—
    Generator comprehension:
    โ€“ The examples:
    โ—
    (i for i in range(10))
    โ—
    f(i for i in range(10))
    โ€“ It is like xrange.
    โ€“ Lazy evaluation โ†’ Save memory.

    View Slide

  22. 22
    Other Comprehensions
    Python 3 only:
    โ—
    set comprehension:
    โ€“ {i for i in range(10)}
    โ—
    dict comprehension:
    โ€“ {i:i for i in range(10)}
    But we can do so with below statements:
    โ—
    set comprehension:
    โ€“ set(i for i in range(10))
    โ—
    dict comprehension:
    โ€“ dict((i, i) for i in range(10))

    View Slide

  23. 23
    Functional Technique
    Think in the functional way.

    View Slide

  24. 24
    The any/all Function
    โ—
    def all_even(seq):
    return all(i % 2 == 0 for i in seq)
    โ—
    all(type(item) is int for item in inputs)
    โ—
    any(i < 0 for i in inputs)

    View Slide

  25. 25
    Challenge 3-3: The Primes (cont.)
    โ€“ limit: in one line.
    โ—
    hint: use any or all
    [2, 3, 5, 7, 11, 13,
    17, 19, 23, 29, 31,
    37, 41, 43, 47, 53,
    59, 61, 67, 71, 73,
    79, 83, 89, 97]

    View Slide

  26. 26
    The zip Function
    โ—
    matrix = [
    [1, 2],
    [3, 4],
    ]
    โ—
    zip(*matrix)

    View Slide

  27. 27
    The zip Function (cont.)
    โ—
    result = [
    ('I am A.', 'email_A'),
    ('I am B.', 'email_B'),
    ]
    โ—
    emails = zip(*result)[1]

    View Slide

  28. 28
    First-class Functions
    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    # file: ex_do.py
    from operator import add, mul
    def do(action, x, y):
    return action(x, y)
    if __name__ == '__main__':
    print do(add, 10, 20)
    print do(mul, 10, 20)
    โ—
    passing functions as
    arguments.

    View Slide

  29. 29
    The lambda Expression
    โ—
    lambda [args]: [expression]
    โ—
    It defines an anonymous function.
    โ—
    It only allows a single expression.
    โ—
    f = lambda x: g(x)+h(x)
    โ—
    do(lambda x, y: (x+y)*(x+y), 10, 20)

    View Slide

  30. 30
    Use sort with Lambda
    โ—
    d = dict(a=300, b=200, c=100)
    โ—
    keys = d.keys()
    โ—
    keys.sort(key=lambda k: d[k])
    โ—
    for k in keys:
    print k, d[k]

    View Slide

  31. 31
    Use sort with Lambda (cont.)
    โ—
    names = ['Andy', 'Bob', 'Cindy']
    โ—
    scores = [70, 100, 95]
    โ—
    table = zip(names, scores)
    โ—
    table.sort(key=lambda pair: pair[1])
    โ—
    for name, score in table:
    print name, score

    View Slide

  32. 32
    The map Function
    โ—
    map(lambda x: x**2, range(10))
    โ—
    map(int, '1 2 3'.split(' '))
    โ—
    map(ord, 'String')
    โ—
    map(open, [])
    โ—
    map(str.split, open())

    View Slide

  33. 33
    The map Function (cont.)
    โ—
    from operator import mul
    โ—
    a = (1, 2)
    โ—
    b = (3, 4)
    โ—
    sum(map(mul, a, b))

    View Slide

  34. 34
    The filter Function
    โ—
    filter(lambda i: i % 2 == 0, range(10))
    โ—
    filter(str.isdigit, strings)
    โ—
    filter(lambda s: s.endswith('.py'),
    file_names)

    View Slide

  35. 35
    Comprehension vs. map/filter
    โ—
    [i ** 2 for i in range(10)]
    โ—
    map(lambda i: i ** 2, range(10))
    โ—
    [i ** 2 for i in range(10) if i % 2 == 0]
    โ—
    map(lambda i: i ** 2, filter(
    lambda i: i % 2 == 0,
    range(10)
    ))

    View Slide

  36. 36
    Comprehension vs. map/filter (cont.)
    โ—
    [ord(c) for c in 'ABC']
    โ—
    map(ord, 'ABC')

    View Slide

  37. 37
    Comprehension vs. map/filter (cont.)
    Compare the speeds of them:
    1. map/filter (with built-in function)
    2. comprehension
    3. map/filter

    View Slide

  38. 38
    The reduce Function
    โ—
    # from functools import reduce # py3
    โ—
    from operator import add
    โ—
    seq = [-1, 0, 1]
    โ—
    reduce(add, s)
    โ—
    seq = ['reduce ', 'the ', 'lines.']
    โ—
    reduce(add, s)

    View Slide

  39. 39
    The partial Function
    โ—
    from functools import partial
    โ—
    from operator import add
    โ—
    rdsum = partial(reduce, add)
    โ—
    rdsum([-1, 0, 1])
    โ—
    rdsum(['reduce ', 'the ', 'lines.'])

    View Slide

  40. 40
    The partial Function (cont.)
    โ—
    from functools import partial
    โ—
    from fractions import gcd as _gcd
    โ—
    _gcd(6, 14)
    โ—
    gcd = partial(reduce, _gcd)
    โ—
    gcd([6, 14, 26])

    View Slide

  41. 41
    Closure
    โ—
    from math import log
    โ—
    def mklog(n):
    return lambda x: log(x, n)
    โ—
    log10 = mklog(10)
    โ—
    log10(100) # n = 10

    View Slide

  42. 42
    Closure (cont.)
    โ—
    setattr(DictLike, attrname,
    # it is a colsure
    (lambda x:
    property(
    lambda self: self.__getitem__(x),
    lambda self, v: self.__setitem__(x, v),
    lambda self: self.__delitem__(x)
    )
    )(attrname)
    )

    View Slide

  43. 43
    The yield Statement
    โ—
    def mkgen(n):
    for i in range(n):
    yield i ** 2
    โ—
    It is a generator.
    โ—
    gen = mkgen(10)
    โ—
    for i in gen:
    print i

    View Slide

  44. 44
    Decorator
    โ—
    def deco(f):
    def f_wrapper(*args, **kargs):
    print 'DEBUG: ', args, kargs
    return f(*args, **kargs)
    return f_wrapper
    โ—
    @deco # is equal to add = deco(add)
    def add(x, y):
    return x+y
    โ—
    add(1, 2)

    View Slide

  45. 45
    Object-oriented Programming
    is also available.

    View Slide

  46. 46
    The class Statement
    class Example(object):
    class_attribute = 1
    def method(self, โ€ฆ):
    pass
    example = Example()
    print example

    View Slide

  47. 47
    The Class in Python (cont.)
    โ—
    Everything in Python is object.
    โ€“ Class is an object, too.
    โ—
    All class inherit the object โ†’ new-style classes
    โ€“ Use new-style classes. It provides more features.
    โ€“ Python 3: auto inherit the object.
    โ—
    Supports multiple inheritance.
    โ€“ Searching attributes/methods is like BFS.

    View Slide

  48. 48
    Bound and Unbound Method
    โ—
    unbound method
    โ€“ def m(self, ...)
    โ€“ C.m(c, ...)
    โ—
    bound method (instance method)
    โ€“ c.m(...)

    View Slide

  49. 49
    Class Method and Static Method
    โ—
    class method
    โ€“ @classmethod
    def m(cls, โ€ฆ)
    โ€“ C.m()
    โ€“ c.m()
    โ—
    static method
    โ€“ @staticmethod
    def m(...)
    โ€“ C.m()
    โ€“ c.m()

    View Slide

  50. 50
    The Data Model of Python
    โ—
    Special methods
    โ€“ __init__
    โ€“ __str__
    โ€“ __repr__
    โ€“ __getitem__ x[key]
    โ†’
    โ€“ __setitem__ x[key] = value
    โ†’
    โ€“ __delitem__ del x[key]
    โ†’
    โ€ฆ
    โ—
    ref:
    docs.python.org/2/reference/datamodel.html

    View Slide

  51. 51
    Protocol
    โ—
    It like interface, but it is only described in doc.
    โ—
    The examples:
    โ€“ iterator protocol
    โ—
    object which supports __iter__ and next
    โ€“ readable
    โ—
    object which supports read
    โ€“ ...

    View Slide

  52. 52
    The employee class
    โ—
    see examples/ex_empolyee.py .

    View Slide

  53. 53
    Do Math with Classes
    โ—
    see examples/ex_do_math_with_classes/ .

    View Slide

  54. 54
    Challenge 6: Give a Raise
    โ—
    Give your employee a
    raise.
    โ€“ without limit
    โ€“ limit: prevent
    modifying salary by
    attribute.
    โ—
    hint: use property
    cindy = Empolyee(...)
    cindy.add_salary(1000)
    print cindy.salary

    View Slide

  55. 55
    Useful Libraries
    import antigravity

    View Slide

  56. 56
    Useful Libraries
    The built-in libraies:
    โ€“ random
    โ€“ datetime
    โ€“ re
    โ€“ hashlib/hmac
    โ€“ decimal
    โ€“ glob
    โ€“ collections
    โ€“ subprocess
    โ€“ multiprocessing
    โ€“ gc
    โ€“ pickle
    โ€“ json
    โ€“ pprint
    โ€“ gzip
    โ€“ timeit
    โ€“ logging
    โ€“ unitest
    โ€“ doctest
    โ€“ pdb
    ...

    View Slide

  57. 57
    Useful Libraries (cont.)
    The third-party libraries on PyPI:
    โ€“ Requests โ€“ Use it instead of the poor built-in urllib.
    โ€“ lxml โ€“ Do you need to parse HTML? Use it!
    โ€“ PyYAML โ€“ YAML is a the best of data serialization standards.
    โ€“ PIL โ€“ Python Image Library
    โ€“ NumPy and SciPy โ€“ are for mathematics, science, and engineering.
    โ€“ SymPy โ€“ is for symbolic mathematic
    โ€“ Bottle, Flask or Django โ€“ are the web frameworks.
    โ€“ Sphinx โ€“ helps you to create documentation.
    ...

    View Slide

  58. 58
    Challenge 7: Iterable Interger
    โ—
    Make integer iterable.
    โ€“ without limit
    โ€“ limit 1: don't use string
    โ€“ limit 2: use collection
    โ—
    hint: use property
    x = IterableInt(10)
    for b in x:
    print b
    0
    1
    0
    1

    View Slide

  59. 59
    Final Project : A Blog System

    View Slide

  60. 60
    Final Project : A Blog System
    โ—
    The cookbook:
    โ€“ Flask โ€“ A micro web framework.
    โ€“ Use pickle to store the posts.
    โ€“ Optional:
    โ—
    A database instead of pickle. (ex. MySQL, PostgreSQL, ...)
    โ—
    A cache layer. (ex. memcached, redis, ...)
    โ—
    A message queue for async jobs. (ex. RabbitMQ, ...)

    View Slide

  61. 61
    It is the end.
    Contact me? http://mosky.tw/

    View Slide