Quicksort, Timsort, Powersort

Slide 1

Slide 1 text

Quicksort, Timsort, Powersort Algorithmic ideas, engineering tricks, and trivia behind CPython’s new sorting algorithm Sebastian Wild www.wild-inter.net PyCon US 2023 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19

Slide 2

Slide 2 text

Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19

Slide 3

Slide 3 text

1 Sort of a list 1 Sort of a list Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19

Slide 4

Slide 4 text

Sorting We use sorting to to make searching faster (binary search!) clean data (remove dups, get canonical form of things) to present data neatly for users as building block in algorithms (database join, sweepline, ...) ... built-in functions in Python: my_list.sort() and sorted(my_list) key parameter to specify sorting criterion prefer pairwise comparator function? ⇝ functools.cmp_to_key Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 1 / 19

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Stable Sorting unsorted input Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19

Slide 7

Slide 7 text

Stable Sorting unsorted input sorted by First Name Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19

Slide 8

Slide 8 text

Stable Sorting unsorted input sorted by First Name sorted by Last Name Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19

Slide 9

Slide 9 text

CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 10

Slide 10 text

CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 11

Slide 11 text

CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 12

Slide 12 text

CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ 2.3.1 – 3.10.2 2003 Timsort custom mergesort by Tim, [P01] almost 20 years unchanged ✓ (except: caching keys, special pure-type comparisons) [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 13

Slide 13 text

CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ 2.3.1 – 3.10.2 2003 Timsort custom mergesort by Tim, [P01] almost 20 years unchanged ✓ (except: caching keys, special pure-type comparisons) 3.11.1 – ∞? 2022 Powersort Timsort with Powersort merge policy [MW18] ✓ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 14

Slide 14 text

Slide 15

Slide 15 text

2 Timsort 2 Timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19

Slide 16

Slide 16 text

Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) “Pick any two!” stable fast, few cmps in place ≈ honest smart investment banker Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19

Slide 20

Slide 20 text

Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) “Pick any two!” stable fast, few cmps in place ≈ honest smart investment banker Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19

Slide 21

Slide 21 text

x MERGE SÖRT MERGE SÖRT idea-instructions.com/merge-sort/ v1.2, CC by-nc-sa 4.0 1x nx 1 2 3 4 MERGE SÖRT Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 5 / 19

Slide 22

Slide 22 text

Mergesort++ Timsort 1 finding existing runs run = any (weakly) increasing or (strictly) decreasing range 2 galloping merges use exponential searches to find position in other run 3 minimum run lengths with binary insertion sort choose 32 ⩽ minrun ⩽ 64, so that ⌈ n minrun ⌉ is 2k or slightly smaller fill runs to minrun elements 4 merge policy decides when to merge which runs 5 keeping runs on a fixed-size runstack Tim Peters et al.: listsort.txt, CPython sources Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 6 / 19

Slide 23

Slide 23 text

Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

3 Beware, Stackoverflow! 3 Beware, Stackoverflow! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Invariant trouble Recall this? Goal: runs on stack grow like Fibonacci: Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] not true(!) for naughty pattern of run lengths KeY project for formal verification in Java de Gouw, de Boer, Bubel, Hähnle, Rot, Steinhöfel: Verifying OpenJDK’s Sort Method for Generic Collections, J Autom Reasoning 2019 In Java: Arrays.sort(Object[]) could throw ArrayIndexOutOfBoundException for specific input of size 67,108,864 Timsort in Java since 2009, but issue never reported? first (incorrectly!) patched in 2013, then a second time in 2015 ... for CPython: patched in 2015 mostly a theoretical issue (needs ⩾ 249 elements) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 8 / 19

Slide 34

Slide 34 text

Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 40

Slide 40 text

Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 41

Slide 41 text

Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 42

Slide 42 text

Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 43

Slide 43 text

Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Enough! Let’s fix this merge policy problem once and for all, shall we? Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 44

Slide 44 text

Slide 45

Slide 45 text

4 Merge policies 4 Merge policies Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19

Slide 46

Slide 46 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 58

Slide 58 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 59

Slide 59 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 1 3 4 5 6 8 10 14 16 18 20 21 22 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 60

Slide 60 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 1 3 4 5 6 8 10 14 16 18 20 21 22 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 63

Slide 63 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 64

Slide 64 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 65

Slide 65 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 66

Slide 66 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 67

Slide 67 text

Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 71 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19

Slide 68

Slide 68 text

Mergesort meets Binary Search Trees 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Merge cost = total area of Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19

Slide 69

Slide 69 text

Mergesort meets Binary Search Trees 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Merge cost = total area of = total length of paths to all array entries Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19

Slide 70

Slide 70 text

Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19

Slide 71

Slide 71 text

Slide 72

Slide 72 text

Slide 73

Slide 73 text

Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 80

Slide 80 text

Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Slide 89

Slide 89 text

Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 90

Slide 90 text

Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 91

Slide 91 text

Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 92

Slide 92 text

Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 93

Slide 93 text

Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19

Slide 94

Slide 94 text

Run-Boundary Powers are Local 4 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 Computation of powers only depends on two adjacent runs. Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 14 / 19

Slide 95

Slide 95 text

Slide 96

Slide 96 text

5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 14 / 19

Slide 97

Slide 97 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 98

Slide 98 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 run1 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 99

Slide 99 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 run1 run2 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 100

Slide 100 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 run1 run2 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 101

Slide 101 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 run1 run2 a – 0 b – 3 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 102

Slide 102 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 103

Slide 103 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 104

Slide 104 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 105

Slide 105 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 106

Slide 106 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 run2 ab – 0 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 107

Slide 107 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 108

Slide 108 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 109

Slide 109 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 110

Slide 110 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 111

Slide 111 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 run2 abc – 0 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 112

Slide 112 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 run1 run2 abc – 0 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 113

Slide 113 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 run1 run2 abc – 0 d – 1 e – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 114

Slide 114 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 run1 run2 abc – 0 d – 1 e – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 115

Slide 115 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 run1 run2 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 116

Slide 116 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 117

Slide 117 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 118

Slide 118 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 119

Slide 119 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d ef 0 1 2 abc – 0 d – 1 ef – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 120

Slide 120 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d ef 0 1 2 abc – 0 d – 1 ef – 2 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 121

Slide 121 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc def 0 1 abc – 0 def – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 122

Slide 122 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc def 0 1 abc – 0 def – 1 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 123

Slide 123 text

Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abcdef 0 abcdef – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19

Slide 124

Slide 124 text

Slide 125

Slide 125 text

Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19

Slide 126

Slide 126 text

Slide 127

Slide 127 text

Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19

Slide 128

Slide 128 text

Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19

Slide 129

Slide 129 text

Some performance data Summary claims: 1 Typical improvement from Powersort: 0-5% fewer comparisons; occasionally 20–30%. Data (One can contrive inputs where Powersort does worse; seems inevitable) 2 No running time regressions: never measurably slower in actual running time Data 3 Sometimes substantially faster (20–40%) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 17 / 19

Slide 130

Slide 130 text

Bonus: Multiway powersort Timsort has been highly successful export from Python ... In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19

Slide 131

Slide 131 text

Slide 132

Slide 132 text

Slide 133

Slide 133 text

Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) * (My PhD was on how this affects Quicksort) Wild: Dual-Pivot Quicksort and Beyond: Analysis of Multiway Partitioning and Its Practical Potential, PhD thesis 2016 ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19

Slide 134

Slide 134 text

Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) * (My PhD was on how this affects Quicksort) Wild: Dual-Pivot Quicksort and Beyond: Analysis of Multiway Partitioning and Its Practical Potential, PhD thesis 2016 ⇝ Easy to do with Powersort while keeping adaptivity! Cawley Gelling, Nebel, Smith, Wild: Multiway Powersort, ALENEX 2023 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19

Slide 135

Slide 135 text

Conclusion Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 136

Slide 136 text

Conclusion Summary Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 137

Slide 137 text

Conclusion Summary Timsort is stable sorting method of choice Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 138

Slide 138 text

Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 139

Slide 139 text

Slide 140

Slide 140 text

Slide 141

Slide 141 text

Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Goals 1 Powersort in other libraries using Timsort? numpy & pandas OpenJDK Android Java runtime V8 JavaScript engine GNU STL std::stable_sort 2 How much does adaptive sorting help? What are typical inputs like in difference applications? Is sorting used at all? Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 142

Slide 142 text

Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Goals 1 Powersort in other libraries using Timsort? numpy & pandas OpenJDK Android Java runtime V8 JavaScript engine GNU STL std::stable_sort 2 How much does adaptive sorting help? What are typical inputs like in difference applications? Is sorting used at all? Please help me with your data I only care about relative ordering ! Instructions to contribute: tiny.cc/sort-lake-city Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19

Slide 143

Slide 143 text

Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 20 / 19

Slide 144

Slide 144 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 145

Slide 145 text

Slide 146

Slide 146 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 147

Slide 147 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 148

Slide 148 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 149

Slide 149 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 150

Slide 150 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 151

Slide 151 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 152

Slide 152 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 153

Slide 153 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 154

Slide 154 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 155

Slide 155 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 156

Slide 156 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 157

Slide 157 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 158

Slide 158 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 159

Slide 159 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 160

Slide 160 text

The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19

Slide 161

Slide 161 text

Abstract cost measures back 1 1.05 1.1 1.15 1.2 1 1.05 1.1 Powersort better merge cost #comparisons Scatter plot of relative costs Timsort Powersort Timsort better Worse on 2-3%, but if so, only slightly. On average, 3% fewer cmps and 5% less merge cost. Input: Runs of expected length √n, n = 105 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 22 / 19

Slide 162

Slide 162 text

Abstract cost measures back 1 1.05 1.1 1.15 1.2 1 1.05 1.1 Powersort better merge cost #comparisons Scatter plot of relative costs Timsort Powersort Timsort better Worse on 2-3%, but if so, only slightly. On average, 3% fewer cmps and 5% less merge cost. Input: Tim’s mixture of long and short runs Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 22 / 19

Slide 163

Slide 163 text

Running times back 0 0.5 1 1.5 bad-example-cmps bad-example-cmps-1m bad-example-cmps-1m-2 bad-example-mc random-permutations random-permutations2 random-sqrtn-runs random-sqrtn-runs2 random-sqrtn-runs3 words-of-bible Powersort faster Timsort faster Timsort faster Timsort faster Average running times Timsort Powersort CPython 3.11 with Powersort resp. Timsort selection of some “random” inputs all using int (cheap comparisons) machine-dependent, but qualitatively stable Machine 1 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 23 / 19

Slide 164

Slide 164 text

Running times back 0 0.5 1 1.5 bad-example-cmps bad-example-cmps-1m bad-example-cmps-1m2 bad-example-mc random-permutations random-permutations2 random-sqrtn-runs random-sqrtn-runs2 random-sqrtn-runs3 words-of-bible Powersort faster Timsort faster Timsort faster Timsort faster Average running times Timsort Powersort CPython 3.11 with Powersort resp. Timsort selection of some “random” inputs all using int (cheap comparisons) machine-dependent, but qualitatively stable Machine 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 23 / 19

Slide 165

Slide 165 text

4-way Powersort 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 24 / 19

Slide 166

Slide 166 text

4-way Powersort 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 2 2 3 1 1 2 1 2 2 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 24 / 19

Slide 167

Slide 167 text

Icons made by Freepik, Gregor Cresnar, Those Icons, Smashicons, Good Ware, Pause08, and Madebyoliver from www.flaticon.com. Vector graphics from Pressfoto, brgfx, macrovector and Jannoon028 on freepik.com Other photos from www.pixabay.com. Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 25 / 19