Making with Python 3 - Speaker Deck

Slide 1

Slide 1 text

Making with Python 3 David Beazley (@dabeaz) PyCon Taiwan 2013

Slide 2

Slide 2 text

Hello! 你好 Thanks for inviting me!

Slide 3

Slide 3 text

Question 1: Do you like Python programming? >>> print('Hello World') Hello World! >>>

Slide 4

Slide 4 text

Question 2: Do you like making things?

Slide 5

Slide 5 text

Question 2: Do you like making things? Of course you do!

Slide 6

Slide 6 text

Making Things is Fun! It's how I got into Python programming It's about solving day-to-day problems

Slide 7

Slide 7 text

Supercomputing Scriptable Scientiﬁc Software beazley@cs.uchicago.edu Sample Session Python

Slide 8

Slide 8 text

Data hacking Toys

Slide 9

Slide 9 text

Data hacking example Find worst area of Chicago to ride my bike Based on number of open potholes in street

Slide 10

Slide 10 text

Science

Slide 11

Slide 11 text

Plotting (in wood)

Slide 12

Slide 12 text

Actually, Toys

Slide 13

Slide 13 text

Actually, Toys

Slide 14

Slide 14 text

Premise: IT Should Be Fun! Programming that is

Slide 15

Slide 15 text

Question: Is Python 3 Fun? Does it have the same "feel" as Python 2? Can you solve problems with it? Can you make things with it? Is it something that you want to use?

Slide 16

Slide 16 text

Some History Python 3000 discussed for a long time Python 3 ﬁrst release: December, 2008 Expectation: Might take >5 years for users

Slide 17

Slide 17 text

Brief Summary Python 3 "ﬁxes" the design ﬂaws in Python 2 But breaks backwards compatibility

Slide 18

Slide 18 text

Poll Are you using Python 3?

Slide 19

Slide 19 text

My Interest I like Python (language and community) Want to see it improve However, I'm only a user (it's a tool) I am not a core developer

Slide 20

Slide 20 text

Using Python 3 Have been using Python 3 for various personal projects since about 2010 Trying to use it for all new projects However, mostly just playing around

Slide 21

Slide 21 text

Book Writing (2009) (2013)

Slide 22

Slide 22 text

Python Cookbook Just ﬁnished Python Cookbook, 3rd Ed. A deep dive into Python 3.3 No regard for backwards compatibility Be as modern as possible

Slide 23

Slide 23 text

Thinking about Python 3 Python 3 looks almost the same However, it's quite different in many ways I like it a lot But, not for reasons you might think

Slide 24

Slide 24 text

Python 2 < "Three"

Slide 25

Slide 25 text

Please Explain >>> 2 < "Three" True >>>

Slide 26

Slide 26 text

Please Explain >>> x = 10 >>> y = 11 >>> avgxy = (x + y) / 2 >>> avgxy 10 >>> Or this...

Slide 27

Slide 27 text

Please Explain >>> s = "Hello" >>> t = u"World" >>> s + t u'HelloWorld' >>> And this...

Slide 28

Slide 28 text

Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print name + ',' + str(price) ACME,123.45 >>> Ugh...

Slide 29

Slide 29 text

Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print '%s,%0.2f'%(name, price) ACME,123.45 >>> Ugh...

Slide 30

Slide 30 text

Please Explain >>> name = "ACME" >>> price = 123.45 >>> print name, price ACME 123.45 >>> print ','.join([name,str(price)]) ACME,123.45 >>> Ugh...

Slide 31

Slide 31 text

Please Explain s = ["N/A", "12", "34"] try: v = int(s[n]) except ValueError, IndexError: v = 0.0 Finally...

Slide 32

Slide 32 text

Only a Small Sample Python 2 has many "features" like this Has grown organically Not always clean or consistent

Slide 33

Slide 33 text

Language Evolution Good ideas often take time to mature Sometimes in unexpected ways

Slide 34

Slide 34 text

Example: Iteration Custom iterators == awesome >>> c = countdown(5) >>> for x in c: ... print x ... 5 4 3 2 1 >>>

Slide 35

Slide 35 text

Example: Iteration Early days: hacking __getitem__() class countdown: def __init__(self, start): self.n = start def __getitem__(self, index): if index >= self.n: raise IndexError() return self.n - index

Slide 36

Slide 36 text

Example: Iteration Iteration protocol class countdown: def __init__(self, start): self.n = start def __iter__(self): return self def next(self): if self.n <= 0: raise StopIteration() n = self.n self.n -= 1 return n

Slide 37

Slide 37 text

Example: Iteration Generators def countdown(n): while n > 0: yield n n -= 1 It's now something quite elegant

Slide 38

Slide 38 text

Getting Walled In Sometimes a good idea can only go so far Limited by existing language features Constrained by backwards compatibility

Slide 39

Slide 39 text

The Story of Sets print ' , '.join([name, str(shares), str(price)])

Slide 40

Slide 40 text

Before Sets Dictionaries with dummy values a = { 'Guido' : None, 'Barry' : None, 'Tim' : None, 'Paula' : None } You write clumsy code that manipulates keys

Slide 41

Slide 41 text

Before Sets def in_common(a, b): return [ key for key in a if key in b ]

Slide 42

Slide 42 text

After Sets A lot of code gets simpliﬁed a = set(['Guido', 'Barry', 'Tim', 'Paula']) b = set(['Dave', 'Paula', 'Thomas', 'Lewis']) all = a | b in_common = a & b a_not_b = a - b

Slide 43

Slide 43 text

Sets and Dicts There is a close relationship Sets : Unordered collection of items Dicts : Unordered set of keys mapped to values Underlying implementation almost identical

Slide 44

Slide 44 text

Sets and Dicts Yet, sets and dicts don't play together a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common >>> a.keys() & b.keys() Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for &: 'list' and 'list' >>>

Slide 45

Slide 45 text

Sets and Dicts Yet, sets and dicts don't play together a = { 'Guido': 1, 'Barry': 2, 'Tim': 3, 'Paula': 4 } b = { 'Dave': 5, 'Paula': 6, 'Thomas': 7, 'Lewis': 8 } Find keys in common (workaround) >>> set(a.keys()) & set(b.keys()) set(['Paula']) >>>

Slide 46

Slide 46 text

Sets and Dicts It all just "works" in Python 3 though >>> a.keys() & b.keys() {'Paula'} >>> Hey, cool!

Slide 47

Slide 47 text

Python 3 In a Nutshell It's a more polished Python All of the parts just ﬁt together better Think of it as a more ﬁnely tuned machine Made possible by breaking backwards compat

Slide 48

Slide 48 text

Python 3 Examples Comparison >>> 2 < "Three" Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() < str() >>>

Slide 49

Slide 49 text

Python 3 Examples Division >>> x = 10 >>> y = 11 >>> avgxy = (x + y) / 2 >>> avgxy 10.5 >>>

Slide 50

Slide 50 text

Python 3 Examples Printing >>> name = "ACME" >>> price = 123.45 >>> print(name, price) ACME 123.45 >>> print(name, price, sep=',') ACME,123.45 >>>

Slide 51

Slide 51 text

Cool Stuff Unpacking >>> values = ('Dave', 37, 'Chicago') >>> name, *rest = values >>> name 'Dave' >>> rest [37, 'Chicago'] >>>

Slide 52

Slide 52 text

Cool Stuff Keyword-only arguments def recv(maxbytes, *, block=True): ... recv(1024, block=False) # OK recv(1024, False) # ERROR

Slide 53

Slide 53 text

Cool Stuff Generator subroutines def countdown(n): while n > 0: yield n n -= 1 def spam(): yield from countdown(5)

Slide 54

Slide 54 text

Cool Stuff Generator subroutines are basis for this...

Slide 55

Slide 55 text

Commentary In Python 3, I ﬁnd myself using fewer "hacks" and workarounds (basically none) Again, the parts just ﬁt together a bit better It's nice

Slide 56

Slide 56 text

More Control

Slide 57

Slide 57 text

Features for Frameworks There are a lot of power features Mostly aimed at more "serious" development Libraries/Frameworks

Slide 58

Slide 58 text

Exception Chains try: raise ValueError("Must be > 0") except Exception as e: raise RuntimeError("Failed") from e

Slide 59

Slide 59 text

Exception Chains Traceback (most recent call last): File "", line 2, in ValueError: Must be > 0 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 4, in RuntimeError: Failed

Slide 60

Slide 60 text

Metaclasses from abc import ABCMeta, abstractmethod class Stream(metaclass=ABCMeta): @abstractmethod def recv(self, maxsize): pass @abstractmethod def send(self, data): pass

Slide 61

Slide 61 text

Metaclasses from enum import Enum class Colors(Enum): red = 1 blue = 2 green = 3 yellow = 4 magenta = 5 Coming : Python 3.4 (PEP 435) Uses some serious metaclass magic

Slide 62

Slide 62 text

Import Hooks Look at sys.meta_path sometime... Can take over the import statement Can create custom importers Diabolical!

Slide 63

Slide 63 text

C Interoperability Standardized interface for exposing memory regions and arrays (memoryviews) Gets array processing libraries (e.g., numpy), I/O libraries, and related tools to play nice

Slide 64

Slide 64 text

Commentary Building big frameworks isn't about "fun" But, correctness matters Python 3 helps a lot (fewer "hacks") A different talk though...

Slide 65

Slide 65 text

Annoyances

Slide 66

Slide 66 text

Unicode/Bytes It's a huge headache (still) Rules of thumb: All text input must be decoded All text output must be encoded Python 3 forces you to be precise about it There are semantic differences

Slide 67

Slide 67 text

Strings vs. Bytes >>> s = "Hello" >>> s[0] 'H' >>> s[0:4] 'Hell' >>> Major headaches for people porting libraries >>> s = b"Hello" >>> s[0] 72 >>> s[0:4] b'Hell' >>>

Slide 68

Slide 68 text

Serialization Woes >>> s = "Hello" >>> json.dumps(s) '"Hello"' >>> pickle.dumps(s) b'\x80\x03X\x05\x00\x00\x00Helloq\x00. >>> Not entirely consistent across libraries Text Bytes Frankly, a mess

Slide 69

Slide 69 text

Unicode Insanity >>> f = open('jalape\xf1o.txt', 'w') >>> g = open(b'jalape\xf1o.txt', 'w') >>> import os >>> os.listdir('.') ['jalape%F1o.txt', 'jalapeño.txt'] >>> os.listdir(b'.') [b'jalape%F1o.txt', b'jalapen\xcc\x83o.txt >>> Really crazy things in system calls

Slide 70

Slide 70 text

Error Messages >>> data = b"Hello World" >>> 'Hell' in data Traceback (most recent call last): File "", line 1, in TypeError: Type str doesn't support the buffer API >>> WhAt?!?

Slide 71

Slide 71 text

New Style Formatting print("{0} has {1} messages".format('Da Always results in lines too wide in editor Kind of silly complaint (I know)

Slide 72

Slide 72 text

What to Make of Python 3? Overall, a really nice language Better appreciated by doing new projects Especially if you're unchained from past Porting still a potential problem (sigh)

Slide 73

Slide 73 text

But is it Fun? On the whole, I think so More so if not porting old code Kind of fun to play with advanced features

Slide 74

Slide 74 text

Recent Fun Using Python 3 to control a CNC mill Stepper Motors (3) Arduino w/ Grblshield Laptop (USB) Rotary cutter

Slide 75

Slide 75 text

Software import serial ser = serial.Serial( '/dev/tty.usbmodem641', 9600) def command(cmd): ser.send(cmd.encode('ascii')+b'\n') resp = ser.readline() if resp != b'Ok\n': raise RuntimeError(resp) It's just serial ports... use pyserial Simple command/response protocol

Slide 76

Slide 76 text

GCode G1 Z10 G1 X0 Y0 G1 Z-2 G1 X50 Y10 G1 X20 Y40 G1 X0 Y0 G1 Z0 Movement controlled by simple commands It's a lot like plotting graphics (0,0) (50,10) (20,40)

Slide 77

Slide 77 text

Whirling Knives You're in the physical world Plotting with 25000 RPM end mill

Slide 78

Slide 78 text

Toy Making (for kids)

Slide 79

Slide 79 text

A Thought Much of the joy in making things is in using precision tools that work This is the real reason to try Python 3 Also: An opportunity

Slide 80

Slide 80 text

That is All! Thanks for inviting me! Hope I've inspired you to look at Python 3 Twitter : @dabeaz Questions?