Slide 1

Slide 1 text

u s i n g & b u i l d i n g PYTHON SET PRACTICE Learn great API design ideas from Python's set types. Luciano Ramalho @standupdev

Slide 2

Slide 2 text

FLUENT PYTHON 2 Available in 9 languages: •Chinese (simplified) •Chinese (traditional) •English •French •Russian •Japanese •Korean •Polish •Portuguese
 2nd ed: I’m working on it!

Slide 3

Slide 3 text

AGENDA Motivation Overview of Python Sets Learning from the set API The __magic__ behind a set class 3

Slide 4

Slide 4 text

MOTIVATION Some common use cases for sets 4

Slide 5

Slide 5 text

USE CASE #1 5 display product if all words in the query appear in the product description.

Slide 6

Slide 6 text

HAND-ROLLED SOLUTION #1 I’ve written code like this in Go, which lacks built-in sets: 6

Slide 7

Slide 7 text

HAND-ROLLED SOLUTION #2 More readable, but still inefficient: 7

Slide 8

Slide 8 text

8 What if… Later! I am too busy coding nested loops! www.workcompass.com/

Slide 9

Slide 9 text

USE CASE #1 9 www.workcompass.com/ display product if all words in the query appear in the product description. coffee grinder manual stainless steel

Slide 10

Slide 10 text

USE CASE #1 10 Q ⊂ D www.workcompass.com/ display product if all words in the query appear in the product description. coffee grinder manual stainless steel

Slide 11

Slide 11 text

USE CASE #2 11 Mark all products previously favorited, except those already in the shopping cart.

Slide 12

Slide 12 text

USE CASE #2 12 F ∖ C Mark all products previously favorited, except those already in the shopping cart.

Slide 13

Slide 13 text

LOGIC AND SETS A close relationship 13

Slide 14

Slide 14 text

Nobody has yet discovered a branch of mathematics that has successfully resisted formalization into set theory. Thomas Forster
 Logic Induction and Sets, p. 167 14

Slide 15

Slide 15 text

LOGIC CONJUNCTION IS INTERSECTION x belongs to the intersection of A with B. is the same as: x belongs to A and
 x also belongs to B. Math notation: x ∈ (A ∩ B) ⟺ (x ∈ A) ∧ (x ∈ B) In computing: AND 15

Slide 16

Slide 16 text

LOGIC DISJUNCTION: UNION x belongs to the union of A and B. is the same as: x belongs to A or
 x belongs to B. Math notation: x ∈ (A ∪ B) ⟺ (x ∈ A) ∨ (x ∈ B) In computing: OR 16

Slide 17

Slide 17 text

SYMMETRIC DIFFERENCE x belongs to A or
 x belongs to B but
 does not belong to both Is the same as: x belongs to the union of A with B less the intersection of A with B. Math notation:
 In computing: XOR 17 x ∈ (A ∆ B) ⟺ (x ∈ A) ⊻ (x ∈ B)

Slide 18

Slide 18 text

DIFFERENCE x belongs to A but
 does not belong to B. is the same as: elements of A minus elements of B Math notation: x ∈ (A ∖ B) ⟺ (x ∈ A) ∧ (x ∉ B) 18

Slide 19

Slide 19 text

SETS IN SEVERAL LANGUAGES 19

Slide 20

Slide 20 text

SETS IN SEVERAL STANDARD LIBRARIES Some languages/platform APIs that implement sets in their standard libraries 20 Java Set interface: < 10 methods; 8 implementations Python set, frozenset: > 10 methods and operators .Net (C# etc.) ISet interface: > 10 methods; 2 implementations JavaScript (ES6) Set: < 10 methods Ruby Set: > 10 methods and operators Python, .Net and Ruby offer rich set APIs

Slide 21

Slide 21 text

SETS IN PYTHON The built-in types 21

Slide 22

Slide 22 text

BUILDING A SET FROM A SERIES OF NUMBERS Using a set comprehension: 22

Slide 23

Slide 23 text

ANOTHER SET, FOR THE EXAMPLES 23

Slide 24

Slide 24 text

ELEMENT CONTAINMENT: THE IN OPERATOR O(1) in sets, because they use a hash table to hold elements. Implemented by the __contains__ special method: 24

Slide 25

Slide 25 text

FUNDAMENTAL SET OPERATIONS 25 Intersection Union Symmetric difference (a.k.a. XOR) Difference

Slide 26

Slide 26 text

SET COMPARISONS Subset and superset testing. In math: ⊂, ⊆, ⊃, ⊇. 26

Slide 27

Slide 27 text

DE MORGAN’S LAW: #1 27

Slide 28

Slide 28 text

DE MORGAN’S LAW: #2 28

Slide 29

Slide 29 text

SET METHODS Going beyond what operators can do. 29

Slide 30

Slide 30 text

SET OPERATORS AND METHODS (1) 30

Slide 31

Slide 31 text

SET OPERATORS AND METHODS (2) Differences: 31

Slide 32

Slide 32 text

SET TESTS All of these return a bool: 32

Slide 33

Slide 33 text

ADDITIONAL METHODS These have nothing to do with math, and all to do with practical computing: 33

Slide 34

Slide 34 text

ABSTRACT SET INTERFACES These interfaces are all defined in collections.abc. set and frozenset both implement Set set also implements MutableSet 34

Slide 35

Slide 35 text

OPERATOR OVERLOADING Not as bad as they say 35

Slide 36

Slide 36 text

COMPARISON OPERATORS 36

Slide 37

Slide 37 text

37

Slide 38

Slide 38 text

THE BEAUTY OF DOUBLE DISPATCH 38

Slide 39

Slide 39 text

EXAMPLE IMPLEMENTATION A set for non-negative integers 39

Slide 40

Slide 40 text

UINTSET: A SET CLASS FOR NON-NEGATIVE INTEGERS Inspired by the intset example in chapter 6 of The Go Programming Language by A. Donovan and B. Kernighan An empty set is represented by zero.
 A set of integers {a, b, c} is represented by on bits in an integer at offsets a, b, and c. Source code: 40 https://github.com/standupdev/uintset

Slide 41

Slide 41 text

REPRESENTING SETS OF INTEGERS AS BIT PATTERNS 41 This set: UintSet({13, 14, 22, 28, 38, 53, 64, 76, 94, 102, 107, 121, 136, 143, 150, 157, 169, 173, 187, 201, 213, 216, 234, 247, 257, 268, 283, 288, 290})

Slide 42

Slide 42 text

REPRESENTING SETS OF INTEGERS AS BIT PATTERNS 42 This set: UintSet({13, 14, 22, 28, 38, 53, 64, 76, 94, 102, 107, 121, 136, 143, 150, 157, 169, 173, 187, 201, 213, 216, 234, 247, 257, 268, 283, 288, 290}) Is represented by this integer 2502158007702946921897431281681230116680925854234644385938703 363396454971897652283727872

Slide 43

Slide 43 text

REPRESENTING SETS OF INTEGERS AS BIT PATTERNS 43 This set: UintSet({13, 14, 22, 28, 38, 53, 64, 76, 94, 102, 107, 121, 136, 143, 150, 157, 169, 173, 187, 201, 213, 216, 234, 247, 257, 268, 283, 288, 290}) Is represented by this integer 2502158007702946921897431281681230116680925854234644385938703 363396454971897652283727872
 Which has this bit pattern: 1010000100000000000000100000000001000000000100000000000010000 0000000000000100100000000000100000000000001000000000000010001 0000000000010000001000000100000010000000000000010000000000000 1000010000000100000000000000000100000000000100000000001000000 00000000100000000010000010000000110000000000000

Slide 44

Slide 44 text

REPRESENTING SETS OF INTEGERS AS BIT PATTERNS 44 This set: UintSet({290})
 
 Is represented by this integer 1989292945639146568621528992587283360401824603189390869761855 907572637988050133502132224
 Which has this bit pattern: 1000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000

Slide 45

Slide 45 text

REPRESENTING SETS OF INTEGERS AS BIT PATTERNS (2) 45 UintSet() → 0 │0│ └─┘ UintSet({0}) → 1 │1│ └─┘ UintSet({1}) → 2 │1│0│ └─┴─┘ UintSet({0, 1, 2, 4, 8}) → 279 │1│0│0│0│1│0│1│1│1│ └─┴─┴─┴─┴─┴─┴─┴─┴─┘ UintSet({0, 1, 2, 3, 4, 5, 6, 7, 8, 9}) → 1023 │1│1│1│1│1│1│1│1│1│1│ └─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘ UintSet({10}) → 1024 │1│0│0│0│0│0│0│0│0│0│0│ └─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘ UintSet({0, 2, 4, 6, 8, 10, 12, 14, 16, 18}) → 349525
 │1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│ └─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘ UintSet({1, 3, 5, 7, 9, 11, 13, 15, 17, 19}) → 699050
 
 │1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│0│1│0│ └─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘

Slide 46

Slide 46 text

SAMPLE METHOD: INTERSECTION OPERATOR & 46

Slide 47

Slide 47 text

SAMPLE METHOD: INTERSECTION WITH ITERABLES 47

Slide 48

Slide 48 text

DIVE INTO THE CODE 48 https://github.com/standupdev/uintset

Slide 49

Slide 49 text

CONCLUSION 49

Slide 50

Slide 50 text

KEY TAKEAWAYS 1. Set operations allow simpler, faster solutions for many tasks.
 2. Python’s set classes are lessons in idiomatic API design.
 3. A set class provides good context for operator overloading. 50

Slide 51

Slide 51 text

THANK YOU! COME SEE ME AT THE EXPO HALL A deeper look at the code for UintSet •Today, 11:45 at the JetBrains/PyCharm booth Fluent Python book signing
 —handing out free copies! •Today, 4:00 at the O’Reilly booth 51

Slide 52

Slide 52 text

Luciano Ramalho
 @ramalhoorg | @standupdev
 [email protected] THANK YOU!