Making with Python 3

Invited keynote. PyCon Taiwan 2013. Taipei.

Transcript

1. Making with Python 3 David Beazley (@dabeaz) PyCon Taiwan 2013
2. Hello! 你好 Thanks for inviting me!
3. Question 1: Do you like Python programming? >>> print('Hello World') Hello World! >>>
4. Question 2: Do you like making things?
5. Question 2: Do you like making things? Of course you do!
6. Making Things is Fun! It's how I got into Python programming It's about solving day-to-day problems
7. Supercomputing Scriptable Scientific Software beazley@cs.uchicago.edu Sample Session Python
8. Data hacking Toys
9. Data hacking example Find worst area of Chicago to ride my bike Based on number of open potholes in street
10. Science
11. Plotting (in wood)
12. Actually, Toys
13. Actually, Toys
14. Premise: IT Should Be Fun! Programming that is
15. Question: Is Python 3 Fun? Does it have the same "feel" as Python 2? Can you solve problems with it? Can you make things with it? Is it something that you want to use?
16. Some History Python 3000 discussed for a long time Python 3 first release: December, 2008 Expectation: Might take >5 years for users
17. Brief Summary Python 3 "fixes" the design flaws in Python 2 But breaks backwards compatibility
18. Poll Are you using Python 3?
19. My Interest I like Python (language and community) Want to see it improve However, I'm only a user (it's a tool) I am not a core developer
20. Using Python 3 Have been using Python 3 for various personal projects since about 2010 Trying to use it for all new projects However, mostly just playing around
21. Book Writing (2009) (2013)
22. Python Cookbook Just finished Python Cookbook, 3rd Ed. A deep dive into Python 3.3 No regard for backwards compatibility Be as modern as possible
23. Thinking about Python 3 Python 3 looks almost the same However, it's quite different in many ways I like it a lot But, not for reasons you might think
24. Python 2 < "Three"
25. Please Explain >>> 2 < "Three" True >>>
26. Please Explain >>> x = 10 >>> y = 11 >>> avgxy = (x + y) / 2 >>> avgxy 10 >>> Or this...
27. Please Explain >>> s = "Hello" >>> t = u"World" >>> s + t u'HelloWorld' >>> And this...
28. Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print name + ',' + str(price) ACME,123.45 >>> Ugh...
29. Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print '%s,%0.2f'%(name, price) ACME,123.45 >>> Ugh...
30. Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print ','.join([name,str(price)]) ACME,123.45 >>> Ugh...
31. Please Explain s = ["N/A", "12", "34"] try: v = int(s[n]) except ValueError, IndexError: v = 0.0 Finally...
32. Only a Small Sample Python 2 has many "features" like this Has grown organically Not always clean or consistent
33. Language Evolution Good ideas often take time to mature Sometimes in unexpected ways
34. Example: Iteration Custom iterators == awesome >>> c = countdown(5) >>> for x in c: ... print x ... 5 4 3 2 1 >>>
35. Example: Iteration Early days: hacking __getitem__() class countdown: def __init__(self, start): self.n = start def __getitem__(self, index): if index >= self.n: raise IndexError() return self.n - index
36. Example: Iteration Iteration protocol class countdown: def __init__(self, start): self.n = start def __iter__(self): return self def next(self): if self.n <= 0: raise StopIteration() n = self.n self.n -= 1 return n
37. Example: Iteration Generators def countdown(n): while n > 0: yield n n -= 1 It's now something quite elegant
38. Getting Walled In Sometimes a good idea can only go so far Limited by existing language features Constrained by backwards compatibility
39. The Story of Sets print ' , '.join([name, str(shares), str(price)])
40. Before Sets Dictionaries with dummy values a = { 'Guido' : None, 'Barry' : None, 'Tim' : None, 'Paula' : None } You write clumsy code that manipulates keys
41. Before Sets def in_common(a, b): return [ key for key in a if key in b ]
42. After Sets A lot of code gets simplified a = set(['Guido', 'Barry', 'Tim', 'Paula']) b = set(['Dave', 'Paula', 'Thomas', 'Lewis']) all = a | b in_common = a & b a_not_b = a - b
43. Sets and Dicts There is a close relationship Sets : Unordered collection of items Dicts : Unordered set of keys mapped to values Underlying implementation almost identical
44. Sets and Dicts Yet, sets and dicts don't play together a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common >>> a.keys() & b.keys() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for &: 'list' and 'list' >>>
45. Sets and Dicts Yet, sets and dicts don't play together a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common (workaround) >>> set(a.keys()) & set(b.keys()) set(['Paula']) >>>
46. Sets and Dicts It all just "works" in Python 3 though >>> a.keys() & b.keys() {'Paula'} >>> Hey, cool!
47. Python 3 In a Nutshell It's a more polished Python All of the parts just fit together better Think of it as a more finely tuned machine Made possible by breaking backwards compat
48. Python 3 Examples Comparison >>> 2 < "Three" Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unorderable types: int() < str() >>>
49. Python 3 Examples Division >>> x = 10 >>> y = 11 >>> avgxy = (x + y) / 2 >>> avgxy 10.5 >>>
50. Python 3 Examples Printing >>> name = "ACME" >>> price = 123.45 >>> print(name, price) ACME 123.45 >>> print(name, price, sep=',') ACME,123.45 >>>
51. Cool Stuff Unpacking >>> values = ('Dave', 37, 'Chicago') >>> name, *rest = values >>> name 'Dave' >>> rest [37, 'Chicago'] >>>
52. Cool Stuff Keyword-only arguments def recv(maxbytes, *, block=True): ... recv(1024, block=False) # OK recv(1024, False) # ERROR
53. Cool Stuff Generator subroutines def countdown(n): while n > 0: yield n n -= 1 def spam(): yield from countdown(5)
54. Cool Stuff Generator subroutines are basis for this...
55. Commentary In Python 3, I find myself using fewer "hacks" and workarounds (basically none) Again, the parts just fit together a bit better It's nice
56. More Control
57. Features for Frameworks There are a lot of power features Mostly aimed at more "serious" development Libraries/Frameworks
58. Exception Chains try: raise ValueError("Must be > 0") except Exception as e: raise RuntimeError("Failed") from e
59. Exception Chains Traceback (most recent call last): File "<stdin>", line 2, in <module> ValueError: Must be > 0 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 4, in <module> RuntimeError: Failed
60. Metaclasses from abc import ABCMeta, abstractmethod class Stream(metaclass=ABCMeta): @abstractmethod def recv(self, maxsize): pass @abstractmethod def send(self, data): pass
61. Metaclasses from enum import Enum class Colors(Enum): red = 1 blue = 2 green = 3 yellow = 4 magenta = 5 Coming : Python 3.4 (PEP 435) Uses some serious metaclass magic
62. Import Hooks Look at sys.meta_path sometime... Can take over the import statement Can create custom importers Diabolical!
63. C Interoperability Standardized interface for exposing memory regions and arrays (memoryviews) Gets array processing libraries (e.g., numpy), I/O libraries, and related tools to play nice
64. Commentary Building big frameworks isn't about "fun" But, correctness matters Python 3 helps a lot (fewer "hacks") A different talk though...
65. Annoyances
66. Unicode/Bytes It's a huge headache (still) Rules of thumb: All text input must be decoded All text output must be encoded Python 3 forces you to be precise about it There are semantic differences
67. Strings vs. Bytes >>> s = "Hello" >>> s[0] 'H' >>> s[0:4] 'Hell' >>> Major headaches for people porting libraries >>> s = b"Hello" >>> s[0] 72 >>> s[0:4] b'Hell' >>>
68. Serialization Woes >>> s = "Hello" >>> json.dumps(s) '"Hello"' >>> pickle.dumps(s) b'\x80\x03X\x05\x00\x00\x00Helloq\x00. >>> Not entirely consistent across libraries Text Bytes Frankly, a mess
69. Unicode Insanity >>> f = open('jalape\xf1o.txt', 'w') >>> g = open(b'jalape\xf1o.txt', 'w') >>> import os >>> os.listdir('.') ['jalape%F1o.txt', 'jalapeño.txt'] >>> os.listdir(b'.') [b'jalape%F1o.txt', b'jalapen\xcc\x83o.txt >>> Really crazy things in system calls
70. Error Messages >>> data = b"Hello World" >>> 'Hell' in data Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: Type str doesn't support the buffer API >>> WhAt?!?
71. New Style Formatting print("{0} has {1} messages".format('Da Always results in lines too wide in editor Kind of silly complaint (I know)
72. What to Make of Python 3? Overall, a really nice language Better appreciated by doing new projects Especially if you're unchained from past Porting still a potential problem (sigh)
73. But is it Fun? On the whole, I think so More so if not porting old code Kind of fun to play with advanced features
74. Recent Fun Using Python 3 to control a CNC mill Stepper Motors (3) Arduino w/ Grblshield Laptop (USB) Rotary cutter
75. Software import serial ser = serial.Serial( '/dev/tty.usbmodem641', 9600) def command(cmd): ser.send(cmd.encode('ascii')+b'\n') resp = ser.readline() if resp != b'Ok\n': raise RuntimeError(resp) It's just serial ports... use pyserial Simple command/response protocol
76. GCode G1 Z10 G1 X0 Y0 G1 Z-2 G1 X50 Y10 G1 X20 Y40 G1 X0 Y0 G1 Z0 Movement controlled by simple commands It's a lot like plotting graphics (0,0) (50,10) (20,40)
77. Whirling Knives You're in the physical world Plotting with 25000 RPM end mill
78. Toy Making (for kids)
79. A Thought Much of the joy in making things is in using precision tools that work This is the real reason to try Python 3 Also: An opportunity
80. That is All! Thanks for inviting me! Hope I've inspired you to look at Python 3 Twitter : @dabeaz Questions?