Understanding Python attributes

Slide 1

Slide 1 text

Understanding attributes (Or: They're not nearly as boring as you think!) Reuven M. Lerner • PyCon US 2022 [email protected] • @reuvenmlerner

Slide 2

Slide 2 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Corporate • Video • Hybrid • Weekly Python Exercise • Books: • Python Workout • Pandas Workout • Come see my booth in the expo hall for swag + discounts! I teach Python! 2

Slide 3

Slide 3 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes x = 100 • This does not mean: Put the value of 100 in the memory location called “x” • This does mean: The name “x” should refer to the integer 100 Let’s assign to a variable! 3

Slide 4

Slide 4 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class MyClass: # do-nothing class pass x = MyClass() # create an instance, assign to x x.y = 100 # assign 100 to x.y • y is not a variable. It’s an attribute. Let’s get fancier 4

Slide 5

Slide 5 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Every object in Python has attributes • They’re basically a private dictionary — but you use a dot (.) rather than [] (square brackets). • The attribute doesn’t exist on the variable, but rather on the object that the variable refers to. Variables vs. attributes 5

Slide 6

Slide 6 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • We retrieve values from attributes quite a bit: • sys.version • str.upper('abc') • random.randint(0, 100) • Just like variables, attributes can contain any Python object — both data and functions. Reading from attributes 6

Slide 7

Slide 7 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • With rare exception, you can set any attribute on any object x.y = 100 >>> sys.version = '4.00' >>> sys.version '4.00' >>> sys.version = '4.20' >>> sys.version '4.20' Setting attributes 7

Slide 8

Slide 8 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • In fact, the __init__ method is meant to set attributes! class Person: def __init__(self, name): self.name = name p1 = Person('name1') p2 = Person('name2') We set attributes all the time 8

Slide 9

Slide 9 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes When __init__ is running 9

Slide 10

Slide 10 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes After __init__ returns 10

Slide 11

Slide 11 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Person: def __init__(self, name): self.name = name p1 = Person('name1') p2 = Person('name2') • Let’s keep track of the population as we add people! What’s missing from this program? 11

Slide 12

Slide 12 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes population = 0 class Person: def __init__(self, name): population += 1 self.name = name def greet(self): return f'Hello, {self.name}!' print(f'Before, {population=}') p1 = Person(‘name1’) p2 = Person(‘name2’) print(f'After, {population=}’) Idea: A global variable 12

Slide 13

Slide 13 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes UnboundLocalError 13

Slide 14

Slide 14 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes population = 0 class Person: def __init__(self, name): population += 1 self.name = name def greet(self): return f'Hello, {self.name}!' print(f'Before, {population=}') p1 = Person('name1') p2 = Person('name2') print(f'After, {population=}') The problem? 14

Slide 15

Slide 15 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes population = 0 class Person: def __init__(self, name): global population population += 1 self.name = name def greet(self): return f'Hello, {self.name}!' print(f'Before, {population=}') p1 = Person('name1') p2 = Person('name2') print(f'After, {population=}') print(p1.greet()) print(p2.greet()) “global” to the rescue! 15

Slide 16

Slide 16 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Everything in Python is an object • Every object has attributes • We can set and retrieve attributes on every object • What if we set an attribute on the Person class? A better alternative 16

Slide 17

Slide 17 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Person: def __init__(self, name): Person.population += 1 self.name = name def greet(self): return f'Hello, {self.name}!' Person.population = 0 print(f'Before, {Person.population=}') p1 = Person('name1') p2 = Person('name2') print(f'After, {Person.population=}') print(p1.greet()) print(p2.greet()) Class attribute 17

Slide 18

Slide 18 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • First we’re creating the Person class • Then, after we’re done, we add the “population” attribute • It works, but this can’t be the best way to do it This is pretty ugly 18

Slide 19

Slide 19 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes Quiz time! 19

Slide 20

Slide 20 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes print('A') class Person: print('B') def __init__(self, name): print('C') self.name = name print('D') print('E') p1 = Person('name1') p2 = Person('name2') What will this code print? 20

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Function and class de f initions look similar, but they aren’t • A function body doesn’t execute when it’s de f ined • A class body, though, must execute when it’s de f ined class Person: def __init__(self, name): self.name = name p1 = Person('name1') p2 = Person('name2') Why? 22 Before this can happen This has to happen

Slide 23

Slide 23 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • But wait — def normally does two things: • Creates a function object • Assigns the object to a variable, the function’s name • But here, def is de f ining __init__, which is… what?   What about de f ? 23

Slide 24

Slide 24 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes Methods are class attributes 24

Slide 25

Slide 25 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Functions de f ined in a class body are actually class attributes • Inside of the class, it looks like a variable • Outside of the class, it looks like an attribute on the class • Where have we seen this before? Modules! • When we import a module, any global variables in the module are available as attributes on the module object. • So any functions we de f ine inside a class aren’t variables, but class attributes. • And any assignments we make in a class don’t create variables, but rather class attributes. Classes are f ile-less modules 25

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • People who come from other languages sometimes call class attributes “static variables.” • Don’t do this! • Static variables are shared across the class and instances • But a class attribute is a name that exists only on the class • Python doesn’t have the concept of a “shared attribute.” • However, many different attributes (and variables) might refer to the same object. Static variables?!? 27

Slide 28

Slide 28 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Person: population = 0 def __init__(self, name): Person.population += 1 self.name = name print(f'Before, {Person.population=}') p1 = Person('name1') p2 = Person('name2') print(f'After, {Person.population=}') print(f'After, {p1.population=}') print(f’After, {p2.population=}’) Let’s see that they’re different 28 2 2 2

Slide 29

Slide 29 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • It worked. • And we got the same values. • Maybe they are actually shared? Uh oh 29

Slide 30

Slide 30 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Nope! • We’ve stumbled onto the attribute lookup rule in Python, which I call ICPO: • I: First Python looks on the instance in the expression. • C: If it doesn’t f ind the attribute on the speci f ied instance, it looks on it’s class. • P: Not on the class? Python looks on the class’s parent • It keeps looking up and up, on each class’s parent • O: The f inal stop is object, the top of Python’s class hierarchy ICPO 30

Slide 31

Slide 31 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Person: population = 0 # not a variable definition! def __init__(self, name): Person.population += 1 self.name = name print(f'Before, {Person.population=}') p1 = Person('name1') p2 = Person('name2') print(f'After, {Person.population=}') print(f'After, {p1.population=}') print(f'After, {p2.population=}') Let’s walk through this 31 I: Does Person have the attribute “population”? YES, 2 I: Does p1 have the attribute “population”? NO. C: Does Person have the attribute “population”? YES, 2 I: Does p2 have the attribute “population”? NO. C: Does Person have the attribute “population”? YES, 2

Slide 32

Slide 32 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes ICPO explains method lookup 32

Slide 33

Slide 33 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Person: def __init__(self, name): self.name = name def greet(self): return f'Hello, {self.name}!' p1 = Person('name1') p2 = Person('name2') print(p1.greet()) print(p2.greet()) A new version of Person 33 I: Does p1 have the attribute “greet”? NO. C: Does Person have the attribute “greet”? YES! I: Does p2 have the attribute “greet”? NO. C: Does Person have the attribute “greet”? YES!

Slide 34

Slide 34 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Methods are class attributes… • …but we normally call them via the instance • ICPO explains how this can be. ICPO explains method lookup 34

Slide 35

Slide 35 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Employee, which is almost exactly like Person • Employees are created with two attributes, name and id_number • Otherwise, they’re the same as people We need a new class 35

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee: def __init__(self, name, id_number): self.name = name self.id_number = id_number def greet(self): return f'Hello, {self.name}!' e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) Let’s de f ine Employee! 37

Slide 38

Slide 38 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes Hello, name1 Hello, name2 Hello, emp1 Hello, emp2 Does it work? 38

Slide 39

Slide 39 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee(Person): def __init__(self, name, id_number): self.name = name self.id_number = id_number def greet(self): return f'Hello, {self.name}!' e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) Try again, but with inheritance 39 This inserts “Person” into the ICPO attribute search

Slide 40

Slide 40 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • The “greet” method is exactly the same in both Person and Employee • So we don’t need to implement it in Employee! Let’s use inheritance 40

Slide 41

Slide 41 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee(Person): def __init__(self, name, id_number): self.name = name self.id_number = id_number e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) Better use of inheritance 41

Slide 42

Slide 42 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Do we need to set self.name in Employee.__init__? • After all, we set it in Person.__init__, right? What about name? 42

Slide 43

Slide 43 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee(Person): def __init__(self, name, id_number): self.id_number = id_number e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) What happens now? 43 Assume it’ll be set in Person.__init__

Slide 44

Slide 44 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes AttributeError: 'Employee' object has no attribute ‘name' • Huh? Let’s check, using “vars”: >>> print(vars(e1)) {'id_number': 1} >>> print(vars(e2)) {'id_number': 2} • The “name” attribute was never set! It doesn’t go well… 44

Slide 45

Slide 45 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • When we created each instance of Employee: • I: We asked the new instance, “Do you have __init__?” No. • C: We asked Employee, “Do you have __init__?” Yes. • Employee.__init__ ran, setting self.id_number • Because Employee.__init__ was found, Person.__init__ never ran • Class de f initions in Python aren’t merged! If Person.__init__ doesn’t run, then the attributes it adds aren’t added • We end up with nameless employees… • …which ends badly when “greet” asks for self.name The problem? 45

Slide 46

Slide 46 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee(Person): def __init__(self, name, id_number): Person.__init__(self, name) self.id_number = id_number e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) Solution: Explicitly call Person.__init__ 46 # Hello, emp1! # Hello, emp2!

Slide 47

Slide 47 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class Employee(Person): def __init__(self, name, id_number): super().__init__(name) self.id_number = id_number e1 = Employee('emp1', 1) e2 = Employee('emp2', 2) print(e1.greet()) print(e2.greet()) Better: Use super 47 # Hello, emp1! # Hello, emp2!

Slide 48

Slide 48 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • What happens when I print(p1) or print(e1)? >>> print(p1) <__main__.Person object at 0x10902c5e0> >>> print(e1) <__main__.Employee object at 0x108f14910> Printing our objects 48

Slide 49

Slide 49 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Calling “print” invokes str, which calls the __str__ method. • With p1: • I: Does p1 have __str__? No. • C: Does p1’s class, Person, have __str__? No. • O: Does object have __str__? Yes! • With e1: • I: Does e1 have __str__? No. • C: Does e1’s class, Employee, have __str__? No. • P: Does e1’s class’s parent, Person, have __str__? No. • O: Does object have __str__? Yes! Why so ugly? 49

Slide 50

Slide 50 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Attribute lookup is the reason that operator overloading works • De f ine __str__ in your class, and it is found earlier than “object” Operator overloading! 50

Slide 51

Slide 51 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Python loves to be explicit, and to work without magic. • But there’s a big piece of magic that we all know, and don’t complain about: Method rewriting. >>> s = 'abcd' >>> s.upper() 'ABCD' >>> str.upper(s) 'ABCD' Something is still missing 51

Slide 52

Slide 52 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • We know that methods are stored in class attributes • We know that we can retrieve a class attribute either via the class or via the instance • But somehow, we’re getting different behavior depending on how we retrieve it! Something is weird here 52

Slide 53

Slide 53 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> Person.greet >>> p1.greet > And indeed: 53

Slide 54

Slide 54 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • Normally, when I retrieve a class attribute, I get the object that is stored in the attribute. • But if: • The attribute’s class has a __get__ method • Then the result of __get__ is returned instead Descriptors 54

Slide 55

Slide 55 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class LoudNumber: def __init__(self, n): print(f'In LoudNumber.__init__, {n=}') self.n = n def __get__(self, *args): print(f'In LoudNumber.__get__, {self.n=}') return self.n • If I assign an instance of LoudNumber to a variable, nothing special happens. • But if I assign it to a class attribute, magic happens. Example 55

Slide 56

Slide 56 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> class Person: age = LoudNumber(30) In LoudNumber.__init__, n=30 >>> p = Person() Use it in a class attribute 56

Slide 57

Slide 57 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes 57

Slide 58

Slide 58 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> p.age In LoudNumber.__get__, self.n=30 30 >>> Person.age In LoudNumber.__get__, self.n=30 30 Retrieve the value 58

Slide 59

Slide 59 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • I retrieve the attribute • I don’t use parentheses • Despite this, a method has run! Notice 59

Slide 60

Slide 60 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes class LoudNumber: def __init__(self, n): print(f'In LoudNumber.__init__, {n=}') self.n = n def __get__(self, obj, objtype): print(f'Also: {obj=}, {objtype=}') print(f'In LoudNumber.__get__, {self.n=}') return self.n What are __get__’s parameters? 60 From what instance is this attribute being retrieved? In what class is this attribute stored?

Slide 61

Slide 61 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> class Person: age = LoudNumber(30) In LoudNumber.__init__, n=30 De f ine the class again 61

Slide 62

Slide 62 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> p = Person() >>> p.age Also: obj=<__main__.Person object at 0x108f15240>, objtype= In LoudNumber.__get__, self.n=30 30 • We retrieve via p, the instance — that’s obj • The class attribute is de f ined on Person — that’s objtype What happens now? 62

Slide 63

Slide 63 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> Person.age Also: obj=None, objtype= In LoudNumber.__get__, self.n=30 30 • We retrieve via Person, the the class — so obj is None! • The class attribute is de f ined on Person — that’s objtype And from the class? 63

Slide 64

Slide 64 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • When we retrieve a method via the class, Python returns the original function that we de f ined. • It can tell, because obj is set to None • Thus, we need to supply an instance as the f irst argument • When we retrieve a method via the instance, Python does its magic rewriting • It takes obj, the instance on which we ran the method • It then calls the original function that we de f ined • It sticks obj into the f irst location — what is assigned to self, and returns a partial function Methods are descriptors 64

Slide 65

Slide 65 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • A partial function is a function with some arguments “pre- loaded” into it from functools import partial def add(a, b): return a + b add5 = partial(add, 5) add5(10) Partial? 65

Slide 66

Slide 66 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes >>> Person.greet >>> p1.greet > Remember this? 66

Slide 67

Slide 67 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • We now understand that methods are class attributes • But where is our original function? • Stored on another class attribute, __dict__ • The method name (as a string) is the key • The original function is the value • Based on whether obj is None (i.e., we’re running things via the instance or the class), Python either returns the original function or the function with our f irst argument inserted there Where is our original function? 67

Slide 68

Slide 68 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • When we ask for a.b in Python, a lot is going on! • Python looks for our attribute via ICPO • When we f inally f ind an attribute, it might be a descriptor • We use descriptors every day without knowing it • They allow us to use methods via either the instance or the class — whichever we prefer. Whew! 68

Slide 69

Slide 69 text

Reuven M. Lerner • @reuvenmlerner • https://lerner.co.il Understanding Attributes • E-mail me: [email protected] • Follow me on Twitter: @reuvenmlerner • Free, weekly newsletters about Python: • About Python: BetterDevelopersWeekly.com • Come see me at my booth, for T shirts and a raffle of my books! Questions or comments? 69