Making with Python 3

Making with Python 3

Invited keynote. PyCon Taiwan 2013. Taipei.

70c42f4cf225f1455a7e01379bbd4d48?s=128

David Beazley

May 25, 2013
Tweet

Transcript

  1. Making with Python 3 David Beazley (@dabeaz) PyCon Taiwan 2013

  2. Hello! 你好 Thanks for inviting me!

  3. Question 1: Do you like Python programming? >>> print('Hello World')

    Hello World! >>>
  4. Question 2: Do you like making things?

  5. Question 2: Do you like making things? Of course you

    do!
  6. Making Things is Fun! It's how I got into Python

    programming It's about solving day-to-day problems
  7. Supercomputing Scriptable Scientific Software beazley@cs.uchicago.edu Sample Session Python

  8. Data hacking Toys

  9. Data hacking example Find worst area of Chicago to ride

    my bike Based on number of open potholes in street
  10. Science

  11. Plotting (in wood)

  12. Actually, Toys

  13. Actually, Toys

  14. Premise: IT Should Be Fun! Programming that is

  15. Question: Is Python 3 Fun? Does it have the same

    "feel" as Python 2? Can you solve problems with it? Can you make things with it? Is it something that you want to use?
  16. Some History Python 3000 discussed for a long time Python

    3 first release: December, 2008 Expectation: Might take >5 years for users
  17. Brief Summary Python 3 "fixes" the design flaws in Python

    2 But breaks backwards compatibility
  18. Poll Are you using Python 3?

  19. My Interest I like Python (language and community) Want to

    see it improve However, I'm only a user (it's a tool) I am not a core developer
  20. Using Python 3 Have been using Python 3 for various

    personal projects since about 2010 Trying to use it for all new projects However, mostly just playing around
  21. Book Writing (2009) (2013)

  22. Python Cookbook Just finished Python Cookbook, 3rd Ed. A deep

    dive into Python 3.3 No regard for backwards compatibility Be as modern as possible
  23. Thinking about Python 3 Python 3 looks almost the same

    However, it's quite different in many ways I like it a lot But, not for reasons you might think
  24. Python 2 < "Three"

  25. Please Explain >>> 2 < "Three" True >>>

  26. Please Explain >>> x = 10 >>> y = 11

    >>> avgxy = (x + y) / 2 >>> avgxy 10 >>> Or this...
  27. Please Explain >>> s = "Hello" >>> t = u"World"

    >>> s + t u'HelloWorld' >>> And this...
  28. Please Explain >>> name = "ACME" >>> price = 123.45

    >>> print name, price ACME 123.45 >>> print name + ',' + str(price) ACME,123.45 >>> Ugh...
  29. Please Explain >>> name = "ACME" >>> price = 123.45

    >>> print name, price ACME 123.45 >>> print '%s,%0.2f'%(name, price) ACME,123.45 >>> Ugh...
  30. Please Explain >>> name = "ACME" >>> price = 123.45

    >>> print name, price ACME 123.45 >>> print ','.join([name,str(price)]) ACME,123.45 >>> Ugh...
  31. Please Explain s = ["N/A", "12", "34"] try: v =

    int(s[n]) except ValueError, IndexError: v = 0.0 Finally...
  32. Only a Small Sample Python 2 has many "features" like

    this Has grown organically Not always clean or consistent
  33. Language Evolution Good ideas often take time to mature Sometimes

    in unexpected ways
  34. Example: Iteration Custom iterators == awesome >>> c = countdown(5)

    >>> for x in c: ... print x ... 5 4 3 2 1 >>>
  35. Example: Iteration Early days: hacking __getitem__() class countdown: def __init__(self,

    start): self.n = start def __getitem__(self, index): if index >= self.n: raise IndexError() return self.n - index
  36. Example: Iteration Iteration protocol class countdown: def __init__(self, start): self.n

    = start def __iter__(self): return self def next(self): if self.n <= 0: raise StopIteration() n = self.n self.n -= 1 return n
  37. Example: Iteration Generators def countdown(n): while n > 0: yield

    n n -= 1 It's now something quite elegant
  38. Getting Walled In Sometimes a good idea can only go

    so far Limited by existing language features Constrained by backwards compatibility
  39. The Story of Sets print ' , '.join([name, str(shares), str(price)])

  40. Before Sets Dictionaries with dummy values a = { 'Guido'

    : None, 'Barry' : None, 'Tim' : None, 'Paula' : None } You write clumsy code that manipulates keys
  41. Before Sets def in_common(a, b): return [ key for key

    in a if key in b ]
  42. After Sets A lot of code gets simplified a =

    set(['Guido', 'Barry', 'Tim', 'Paula']) b = set(['Dave', 'Paula', 'Thomas', 'Lewis']) all = a | b in_common = a & b a_not_b = a - b
  43. Sets and Dicts There is a close relationship Sets :

    Unordered collection of items Dicts : Unordered set of keys mapped to values Underlying implementation almost identical
  44. Sets and Dicts Yet, sets and dicts don't play together

    a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common >>> a.keys() & b.keys() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for &: 'list' and 'list' >>>
  45. Sets and Dicts Yet, sets and dicts don't play together

    a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common (workaround) >>> set(a.keys()) & set(b.keys()) set(['Paula']) >>>
  46. Sets and Dicts It all just "works" in Python 3

    though >>> a.keys() & b.keys() {'Paula'} >>> Hey, cool!
  47. Python 3 In a Nutshell It's a more polished Python

    All of the parts just fit together better Think of it as a more finely tuned machine Made possible by breaking backwards compat
  48. Python 3 Examples Comparison >>> 2 < "Three" Traceback (most

    recent call last): File "<stdin>", line 1, in <module> TypeError: unorderable types: int() < str() >>>
  49. Python 3 Examples Division >>> x = 10 >>> y

    = 11 >>> avgxy = (x + y) / 2 >>> avgxy 10.5 >>>
  50. Python 3 Examples Printing >>> name = "ACME" >>> price

    = 123.45 >>> print(name, price) ACME 123.45 >>> print(name, price, sep=',') ACME,123.45 >>>
  51. Cool Stuff Unpacking >>> values = ('Dave', 37, 'Chicago') >>>

    name, *rest = values >>> name 'Dave' >>> rest [37, 'Chicago'] >>>
  52. Cool Stuff Keyword-only arguments def recv(maxbytes, *, block=True): ... recv(1024,

    block=False) # OK recv(1024, False) # ERROR
  53. Cool Stuff Generator subroutines def countdown(n): while n > 0:

    yield n n -= 1 def spam(): yield from countdown(5)
  54. Cool Stuff Generator subroutines are basis for this...

  55. Commentary In Python 3, I find myself using fewer "hacks"

    and workarounds (basically none) Again, the parts just fit together a bit better It's nice
  56. More Control

  57. Features for Frameworks There are a lot of power features

    Mostly aimed at more "serious" development Libraries/Frameworks
  58. Exception Chains try: raise ValueError("Must be > 0") except Exception

    as e: raise RuntimeError("Failed") from e
  59. Exception Chains Traceback (most recent call last): File "<stdin>", line

    2, in <module> ValueError: Must be > 0 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 4, in <module> RuntimeError: Failed
  60. Metaclasses from abc import ABCMeta, abstractmethod class Stream(metaclass=ABCMeta): @abstractmethod def

    recv(self, maxsize): pass @abstractmethod def send(self, data): pass
  61. Metaclasses from enum import Enum class Colors(Enum): red = 1

    blue = 2 green = 3 yellow = 4 magenta = 5 Coming : Python 3.4 (PEP 435) Uses some serious metaclass magic
  62. Import Hooks Look at sys.meta_path sometime... Can take over the

    import statement Can create custom importers Diabolical!
  63. C Interoperability Standardized interface for exposing memory regions and arrays

    (memoryviews) Gets array processing libraries (e.g., numpy), I/O libraries, and related tools to play nice
  64. Commentary Building big frameworks isn't about "fun" But, correctness matters

    Python 3 helps a lot (fewer "hacks") A different talk though...
  65. Annoyances

  66. Unicode/Bytes It's a huge headache (still) Rules of thumb: All

    text input must be decoded All text output must be encoded Python 3 forces you to be precise about it There are semantic differences
  67. Strings vs. Bytes >>> s = "Hello" >>> s[0] 'H'

    >>> s[0:4] 'Hell' >>> Major headaches for people porting libraries >>> s = b"Hello" >>> s[0] 72 >>> s[0:4] b'Hell' >>>
  68. Serialization Woes >>> s = "Hello" >>> json.dumps(s) '"Hello"' >>>

    pickle.dumps(s) b'\x80\x03X\x05\x00\x00\x00Helloq\x00. >>> Not entirely consistent across libraries Text Bytes Frankly, a mess
  69. Unicode Insanity >>> f = open('jalape\xf1o.txt', 'w') >>> g =

    open(b'jalape\xf1o.txt', 'w') >>> import os >>> os.listdir('.') ['jalape%F1o.txt', 'jalapeño.txt'] >>> os.listdir(b'.') [b'jalape%F1o.txt', b'jalapen\xcc\x83o.txt >>> Really crazy things in system calls
  70. Error Messages >>> data = b"Hello World" >>> 'Hell' in

    data Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Type str doesn't support the buffer API >>> WhAt?!?
  71. New Style Formatting print("{0} has {1} messages".format('Da Always results in

    lines too wide in editor Kind of silly complaint (I know)
  72. What to Make of Python 3? Overall, a really nice

    language Better appreciated by doing new projects Especially if you're unchained from past Porting still a potential problem (sigh)
  73. But is it Fun? On the whole, I think so

    More so if not porting old code Kind of fun to play with advanced features
  74. Recent Fun Using Python 3 to control a CNC mill

    Stepper Motors (3) Arduino w/ Grblshield Laptop (USB) Rotary cutter
  75. Software import serial ser = serial.Serial( '/dev/tty.usbmodem641', 9600) def command(cmd):

    ser.send(cmd.encode('ascii')+b'\n') resp = ser.readline() if resp != b'Ok\n': raise RuntimeError(resp) It's just serial ports... use pyserial Simple command/response protocol
  76. GCode G1 Z10 G1 X0 Y0 G1 Z-2 G1 X50

    Y10 G1 X20 Y40 G1 X0 Y0 G1 Z0 Movement controlled by simple commands It's a lot like plotting graphics (0,0) (50,10) (20,40)
  77. Whirling Knives You're in the physical world Plotting with 25000

    RPM end mill
  78. Toy Making (for kids)

  79. A Thought Much of the joy in making things is

    in using precision tools that work This is the real reason to try Python 3 Also: An opportunity
  80. That is All! Thanks for inviting me! Hope I've inspired

    you to look at Python 3 Twitter : @dabeaz Questions?