$30 off During Our Annual Pro Sale. View Details »

Making with Python 3

Making with Python 3

Invited keynote. PyCon Taiwan 2013. Taipei.

David Beazley

May 25, 2013
Tweet

More Decks by David Beazley

Other Decks in Programming

Transcript

  1. Making with Python 3
    David Beazley (@dabeaz)
    PyCon Taiwan 2013

    View Slide

  2. Hello! 你好
    Thanks for inviting me!

    View Slide

  3. Question 1:
    Do you like Python programming?
    >>> print('Hello World')
    Hello World!
    >>>

    View Slide

  4. Question 2:
    Do you like making things?

    View Slide

  5. Question 2:
    Do you like making things?
    Of course you do!

    View Slide

  6. Making Things is Fun!
    It's how I got into Python programming
    It's about solving day-to-day problems

    View Slide

  7. Supercomputing
    Scriptable Scientific Software
    [email protected]
    Sample Session
    Python

    View Slide

  8. Data hacking
    Toys

    View Slide

  9. Data hacking example
    Find worst area
    of Chicago to
    ride my bike
    Based on number
    of open potholes
    in street

    View Slide

  10. Science

    View Slide

  11. Plotting (in wood)

    View Slide

  12. Actually, Toys

    View Slide

  13. Actually, Toys

    View Slide

  14. Premise: IT Should Be Fun!
    Programming that is

    View Slide

  15. Question: Is Python 3 Fun?
    Does it have the same "feel" as Python 2?
    Can you solve problems with it?
    Can you make things with it?
    Is it something that you want to use?

    View Slide

  16. Some History
    Python 3000 discussed for a long time
    Python 3 first release: December, 2008
    Expectation: Might take >5 years for users

    View Slide

  17. Brief Summary
    Python 3 "fixes" the design flaws in Python 2
    But breaks backwards compatibility

    View Slide

  18. Poll
    Are you using Python 3?

    View Slide

  19. My Interest
    I like Python (language and community)
    Want to see it improve
    However, I'm only a user (it's a tool)
    I am not a core developer

    View Slide

  20. Using Python 3
    Have been using Python 3 for various
    personal projects since about 2010
    Trying to use it for all new projects
    However, mostly just playing around

    View Slide

  21. Book Writing
    (2009) (2013)

    View Slide

  22. Python Cookbook
    Just finished Python Cookbook, 3rd Ed.
    A deep dive into Python 3.3
    No regard for backwards compatibility
    Be as modern as possible

    View Slide

  23. Thinking about Python 3
    Python 3 looks almost the same
    However, it's quite different in many ways
    I like it a lot
    But, not for reasons you might think

    View Slide

  24. Python 2 < "Three"

    View Slide

  25. Please Explain
    >>> 2 < "Three"
    True
    >>>

    View Slide

  26. Please Explain
    >>> x = 10
    >>> y = 11
    >>> avgxy = (x + y) / 2
    >>> avgxy
    10
    >>>
    Or this...

    View Slide

  27. Please Explain
    >>> s = "Hello"
    >>> t = u"World"
    >>> s + t
    u'HelloWorld'
    >>>
    And this...

    View Slide

  28. Please Explain
    >>> name = "ACME"
    >>> price = 123.45
    >>> print name, price
    ACME 123.45
    >>> print name + ',' + str(price)
    ACME,123.45
    >>>
    Ugh...

    View Slide

  29. Please Explain
    >>> name = "ACME"
    >>> price = 123.45
    >>> print name, price
    ACME 123.45
    >>> print '%s,%0.2f'%(name, price)
    ACME,123.45
    >>>
    Ugh...

    View Slide

  30. Please Explain
    >>> name = "ACME"
    >>> price = 123.45
    >>> print name, price
    ACME 123.45
    >>> print ','.join([name,str(price)])
    ACME,123.45
    >>>
    Ugh...

    View Slide

  31. Please Explain
    s = ["N/A", "12", "34"]
    try:
    v = int(s[n])
    except ValueError, IndexError:
    v = 0.0
    Finally...

    View Slide

  32. Only a Small Sample
    Python 2 has many "features" like this
    Has grown organically
    Not always clean or consistent

    View Slide

  33. Language Evolution
    Good ideas often take time to mature
    Sometimes in unexpected ways

    View Slide

  34. Example: Iteration
    Custom iterators == awesome
    >>> c = countdown(5)
    >>> for x in c:
    ... print x
    ...
    5
    4
    3
    2
    1
    >>>

    View Slide

  35. Example: Iteration
    Early days: hacking __getitem__()
    class countdown:
    def __init__(self, start):
    self.n = start
    def __getitem__(self, index):
    if index >= self.n:
    raise IndexError()
    return self.n - index

    View Slide

  36. Example: Iteration
    Iteration protocol
    class countdown:
    def __init__(self, start):
    self.n = start
    def __iter__(self):
    return self
    def next(self):
    if self.n <= 0:
    raise StopIteration()
    n = self.n
    self.n -= 1
    return n

    View Slide

  37. Example: Iteration
    Generators
    def countdown(n):
    while n > 0:
    yield n
    n -= 1
    It's now something quite elegant

    View Slide

  38. Getting Walled In
    Sometimes a good idea can only go so far
    Limited by existing language features
    Constrained by backwards compatibility

    View Slide

  39. The Story of Sets
    print ' , '.join([name, str(shares), str(price)])

    View Slide

  40. Before Sets
    Dictionaries with dummy values
    a = {
    'Guido' : None,
    'Barry' : None,
    'Tim' : None,
    'Paula' : None
    }
    You write clumsy code that manipulates keys

    View Slide

  41. Before Sets
    def in_common(a, b):
    return [ key for key in a
    if key in b ]

    View Slide

  42. After Sets
    A lot of code gets simplified
    a = set(['Guido', 'Barry',
    'Tim', 'Paula'])
    b = set(['Dave', 'Paula',
    'Thomas', 'Lewis'])
    all = a | b
    in_common = a & b
    a_not_b = a - b

    View Slide

  43. Sets and Dicts
    There is a close relationship
    Sets : Unordered collection of items
    Dicts : Unordered set of keys mapped to values
    Underlying implementation almost identical

    View Slide

  44. Sets and Dicts
    Yet, sets and dicts don't play together
    a = {
    'Guido': 1,
    'Barry': 2,
    'Tim': 3,
    'Paula': 4
    }
    b = {
    'Dave': 5,
    'Paula': 6,
    'Thomas': 7,
    'Lewis': 8
    }
    Find keys in common
    >>> a.keys() & b.keys()
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: unsupported operand type(s) for &: 'list'
    and 'list'
    >>>

    View Slide

  45. Sets and Dicts
    Yet, sets and dicts don't play together
    a = {
    'Guido': 1,
    'Barry': 2,
    'Tim': 3,
    'Paula': 4
    }
    b = {
    'Dave': 5,
    'Paula': 6,
    'Thomas': 7,
    'Lewis': 8
    }
    Find keys in common (workaround)
    >>> set(a.keys()) & set(b.keys())
    set(['Paula'])
    >>>

    View Slide

  46. Sets and Dicts
    It all just "works" in Python 3 though
    >>> a.keys() & b.keys()
    {'Paula'}
    >>>
    Hey, cool!

    View Slide

  47. Python 3 In a Nutshell
    It's a more polished Python
    All of the parts just fit together better
    Think of it as a more finely tuned machine
    Made possible by breaking backwards compat

    View Slide

  48. Python 3 Examples
    Comparison
    >>> 2 < "Three"
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: unorderable types: int() <
    str()
    >>>

    View Slide

  49. Python 3 Examples
    Division
    >>> x = 10
    >>> y = 11
    >>> avgxy = (x + y) / 2
    >>> avgxy
    10.5
    >>>

    View Slide

  50. Python 3 Examples
    Printing
    >>> name = "ACME"
    >>> price = 123.45
    >>> print(name, price)
    ACME 123.45
    >>> print(name, price, sep=',')
    ACME,123.45
    >>>

    View Slide

  51. Cool Stuff
    Unpacking
    >>> values = ('Dave', 37, 'Chicago')
    >>> name, *rest = values
    >>> name
    'Dave'
    >>> rest
    [37, 'Chicago']
    >>>

    View Slide

  52. Cool Stuff
    Keyword-only arguments
    def recv(maxbytes, *, block=True):
    ...
    recv(1024, block=False) # OK
    recv(1024, False) # ERROR

    View Slide

  53. Cool Stuff
    Generator subroutines
    def countdown(n):
    while n > 0:
    yield n
    n -= 1
    def spam():
    yield from countdown(5)

    View Slide

  54. Cool Stuff
    Generator subroutines are basis for this...

    View Slide

  55. Commentary
    In Python 3, I find myself using fewer "hacks"
    and workarounds (basically none)
    Again, the parts just fit together a bit better
    It's nice

    View Slide

  56. More Control

    View Slide

  57. Features for Frameworks
    There are a lot of power features
    Mostly aimed at more "serious" development
    Libraries/Frameworks

    View Slide

  58. Exception Chains
    try:
    raise ValueError("Must be > 0")
    except Exception as e:
    raise RuntimeError("Failed") from e

    View Slide

  59. Exception Chains
    Traceback (most recent call last):
    File "", line 2, in
    ValueError: Must be > 0
    The above exception was the direct cause
    of the following exception:
    Traceback (most recent call last):
    File "", line 4, in
    RuntimeError: Failed

    View Slide

  60. Metaclasses
    from abc import ABCMeta, abstractmethod
    class Stream(metaclass=ABCMeta):
    @abstractmethod
    def recv(self, maxsize):
    pass
    @abstractmethod
    def send(self, data):
    pass

    View Slide

  61. Metaclasses
    from enum import Enum
    class Colors(Enum):
    red = 1
    blue = 2
    green = 3
    yellow = 4
    magenta = 5
    Coming : Python 3.4 (PEP 435)
    Uses some serious metaclass magic

    View Slide

  62. Import Hooks
    Look at sys.meta_path sometime...
    Can take over the import statement
    Can create custom importers
    Diabolical!

    View Slide

  63. C Interoperability
    Standardized interface for exposing memory
    regions and arrays (memoryviews)
    Gets array processing libraries (e.g., numpy),
    I/O libraries, and related tools to play nice

    View Slide

  64. Commentary
    Building big frameworks isn't about "fun"
    But, correctness matters
    Python 3 helps a lot (fewer "hacks")
    A different talk though...

    View Slide

  65. Annoyances

    View Slide

  66. Unicode/Bytes
    It's a huge headache (still)
    Rules of thumb:
    All text input must be decoded
    All text output must be encoded
    Python 3 forces you to be precise about it
    There are semantic differences

    View Slide

  67. Strings vs. Bytes
    >>> s = "Hello"
    >>> s[0]
    'H'
    >>> s[0:4]
    'Hell'
    >>>
    Major headaches for people porting libraries
    >>> s = b"Hello"
    >>> s[0]
    72
    >>> s[0:4]
    b'Hell'
    >>>

    View Slide

  68. Serialization Woes
    >>> s = "Hello"
    >>> json.dumps(s)
    '"Hello"'
    >>> pickle.dumps(s)
    b'\x80\x03X\x05\x00\x00\x00Helloq\x00.
    >>>
    Not entirely consistent across libraries
    Text
    Bytes
    Frankly, a mess

    View Slide

  69. Unicode Insanity
    >>> f = open('jalape\xf1o.txt', 'w')
    >>> g = open(b'jalape\xf1o.txt', 'w')
    >>> import os
    >>> os.listdir('.')
    ['jalape%F1o.txt', 'jalapeño.txt']
    >>> os.listdir(b'.')
    [b'jalape%F1o.txt', b'jalapen\xcc\x83o.txt
    >>>
    Really crazy things in system calls

    View Slide

  70. Error Messages
    >>> data = b"Hello World"
    >>> 'Hell' in data
    Traceback (most recent call last):
    File "", line 1, in
    TypeError: Type str doesn't support
    the buffer API
    >>>
    WhAt?!?

    View Slide

  71. New Style Formatting
    print("{0} has {1} messages".format('Da
    Always results in lines too wide in editor
    Kind of silly complaint (I know)

    View Slide

  72. What to Make of Python 3?
    Overall, a really nice language
    Better appreciated by doing new projects
    Especially if you're unchained from past
    Porting still a potential problem (sigh)

    View Slide

  73. But is it Fun?
    On the whole, I think so
    More so if not porting old code
    Kind of fun to play with advanced features

    View Slide

  74. Recent Fun
    Using Python 3 to control a CNC mill
    Stepper
    Motors (3)
    Arduino w/
    Grblshield
    Laptop
    (USB)
    Rotary
    cutter

    View Slide

  75. Software
    import serial
    ser = serial.Serial(
    '/dev/tty.usbmodem641', 9600)
    def command(cmd):
    ser.send(cmd.encode('ascii')+b'\n')
    resp = ser.readline()
    if resp != b'Ok\n':
    raise RuntimeError(resp)
    It's just serial ports... use pyserial
    Simple command/response protocol

    View Slide

  76. GCode
    G1 Z10
    G1 X0 Y0
    G1 Z-2
    G1 X50 Y10
    G1 X20 Y40
    G1 X0 Y0
    G1 Z0
    Movement controlled by simple commands
    It's a lot like plotting graphics
    (0,0)
    (50,10)
    (20,40)

    View Slide

  77. Whirling Knives
    You're in the physical world
    Plotting with 25000 RPM end mill

    View Slide

  78. Toy Making (for kids)

    View Slide

  79. A Thought
    Much of the joy in making things is in using
    precision tools that work
    This is the real reason to try Python 3
    Also: An opportunity

    View Slide

  80. That is All!
    Thanks for inviting me!
    Hope I've inspired you to look at Python 3
    Twitter : @dabeaz
    Questions?

    View Slide