Quicksort, Timsort, Powersort

Algorithmic ideas, engineering tricks, and trivia behind CPython’s new sorting algorithm

These are the slides for my talk at PyCon US 2023.

Transcript

1. Quicksort, Timsort, Powersort Algorithmic ideas, engineering tricks, and trivia behind CPython’s new sorting algorithm Sebastian Wild www.wild-inter.net PyCon US 2023 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19
2. Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19
3. 1 Sort of a list 1 Sort of a list Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 0 / 19
4. Sorting We use sorting to to make searching faster (binary search!) clean data (remove dups, get canonical form of things) to present data neatly for users as building block in algorithms (database join, sweepline, ...) ... built-in functions in Python: my_list.sort() and sorted(my_list) key parameter to specify sorting criterion prefer pairwise comparator function? ⇝ functools.cmp_to_key Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 1 / 19
5. Sorting We use sorting to to make searching faster (binary search!) clean data (remove dups, get canonical form of things) to present data neatly for users as building block in algorithms (database join, sweepline, ...) ... built-in functions in Python: my_list.sort() and sorted(my_list) key parameter to specify sorting criterion prefer pairwise comparator function? ⇝ functools.cmp_to_key Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 1 / 19
6. Stable Sorting unsorted input Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19
7. Stable Sorting unsorted input sorted by First Name Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19
8. Stable Sorting unsorted input sorted by First Name sorted by Last Name Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 2 / 19
9. CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
10. CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
11. CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
12. CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ 2.3.1 – 3.10.2 2003 Timsort custom mergesort by Tim, [P01] almost 20 years unchanged ✓ (except: caching keys, special pure-type comparisons) [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
13. CPython Sorting History Python Version (Year) Sorting method Remarks Stable? 0.9 – 1.4 1991 qsort call to C library, general purpose Quicksort, [BM93] ✗ 1.5 – 1.6 1998 custom Quicksort inspired by Tim Peters, “NEWSORT” ✗ 2.0 – 2.2 2000 Samplesort Quicksort with clever pivot choice, [FM70] ✗ 2.3.1 – 3.10.2 2003 Timsort custom mergesort by Tim, [P01] almost 20 years unchanged ✓ (except: caching keys, special pure-type comparisons) 3.11.1 – ∞? 2022 Powersort Timsort with Powersort merge policy [MW18] ✓ [BM93] Bentley & McIlroy: Engineering a sort function, Softw. Prac. Exp. 1993 [FM70] Frazer & McKellar: Samplesort: A Sampling Approach to Minimal Storage Tree Sorting, J. ACM 1970 [P01] Tim Peters et al.: listsort.txt, CPython sources [MW18] Munro & Wild: Nearly-Optimal Mergesorts: Fast, Practical Sorting Methods That Optimally Adapt to Existing Runs, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
14. Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
15. 2 Timsort 2 Timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 3 / 19
16. Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19
17. Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19
18. Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19
19. Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) “Pick any two!” stable fast, few cmps in place ≈ honest smart investment banker Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19
20. Sorting Trade-offs Quicksort is generally fast and in place, but not stable Mergesort is generally (quite) fast and stable, but not in place stable and in place is tricky (possible, but relatively slow) “Pick any two!” stable fast, few cmps in place ≈ honest smart investment banker Python has lots of objects and pointers anyways ... “in-place” property least painful to sacrifice ⇝ Mergesort! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 4 / 19
21. x MERGE SÖRT MERGE SÖRT idea-instructions.com/merge-sort/ v1.2, CC by-nc-sa 4.0 1x nx 1 2 3 4 MERGE SÖRT Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 5 / 19
22. Mergesort++ Timsort 1 finding existing runs run = any (weakly) increasing or (strictly) decreasing range 2 galloping merges use exponential searches to find position in other run 3 minimum run lengths with binary insertion sort choose 32 ⩽ minrun ⩽ 64, so that ⌈ n minrun ⌉ is 2k or slightly smaller fill runs to minrun elements 4 merge policy decides when to merge which runs 5 keeping runs on a fixed-size runstack Tim Peters et al.: listsort.txt, CPython sources Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 6 / 19
23. Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
24. Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
25. Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
26. Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
27. Timsort merge policy (original) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.append((i,j-i)); i = j 6 while Rule A/B/C applicable 7 merge X,Y resp. Y,Z 8 while len(runs) > 1: 9 merge Y,Z Z Y X . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . extend_run detects next run & boosts to minrun if necessary Goal: runs on stack grow like Fibonacci numbers Invariant: ∀j : runs[j] ⩾ runs[j + 1] + runs[j + 2] ⇝ max stack height ≈ logφ (n) Why exactly these rules? Tim Peters: “first thing I tried that ‘worked well (a) small runs stack, (b) balanced on equal runs, (c) O(n log n) worst case* ’” full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
28. Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
29. 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 7 / 19
30. Invariant trouble Recall this? Goal: runs on stack grow like Fibonacci: Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] not true(!) for naughty pattern of run lengths KeY project for formal verification in Java de Gouw, de Boer, Bubel, Hähnle, Rot, Steinhöfel: Verifying OpenJDK’s Sort Method for Generic Collections, J Autom Reasoning 2019 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 8 / 19
31. Invariant trouble Recall this? Goal: runs on stack grow like Fibonacci: Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] not true(!) for naughty pattern of run lengths KeY project for formal verification in Java de Gouw, de Boer, Bubel, Hähnle, Rot, Steinhöfel: Verifying OpenJDK’s Sort Method for Generic Collections, J Autom Reasoning 2019 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 8 / 19
32. Invariant trouble Recall this? Goal: runs on stack grow like Fibonacci: Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] not true(!) for naughty pattern of run lengths KeY project for formal verification in Java de Gouw, de Boer, Bubel, Hähnle, Rot, Steinhöfel: Verifying OpenJDK’s Sort Method for Generic Collections, J Autom Reasoning 2019 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 8 / 19
33. Invariant trouble Recall this? Goal: runs on stack grow like Fibonacci: Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] not true(!) for naughty pattern of run lengths KeY project for formal verification in Java de Gouw, de Boer, Bubel, Hähnle, Rot, Steinhöfel: Verifying OpenJDK’s Sort Method for Generic Collections, J Autom Reasoning 2019 In Java: Arrays.sort(Object[]) could throw ArrayIndexOutOfBoundException for specific input of size 67,108,864 Timsort in Java since 2009, but issue never reported? first (incorrectly!) patched in 2013, then a second time in 2015 ... for CPython: patched in 2015 mostly a theoretical issue (needs ⩾ 249 elements) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 8 / 19
34. Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19
35. Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19
36. Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19
37. Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19
38. Timsort merge policy (patched) 1 def timsort(lst): 2 i = 0; runs = [] 3 while i < len(lst): 4 j = extend_run(lst, i) 5 runs.apppend((i,j)); i = j 6 while Rule A/B/C/D applicable 7 merge corresponding runs 8 while len(runs) > 1: 9 merge topmost 2 runs Z Y X W . . . top runs Rule A: Z > X ⇝ merge(X, Y) Z Y X . . . Z X+Y . . . Rule B: Z ≥ Y ⇝ merge(Y, Z) ¬ A Z Y . . . Y+Z . . . Rule C: Y + Z ≥ X ⇝ merge(Y, Z) ¬ A, ¬ B Z Y X Z . . . Y+Z X . . . Rule D: X + Y ≥ W ⇝ merge(Y, Z) ¬ A, ¬ B, ¬ C Z Y X W Y . . . Y+Z X W . . . Need to add a Rule D ⇝ Invariant: ∀j = 0, . . . , len(runs) − 3 : runs[j] ⩾ runs[j + 1] + runs[j + 2] ✓ running time indeed O(n log n) ...very complicated to prove Auger, Jugé, Nicaud, Pivoteau: On the Worst-Case Complexity of TimSort, ESA 2018 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 9 / 19
39. Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
40. Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
41. Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
42. Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
43. Timsort bad case Timsort’s merge policy mostly works OK ...but does stupid things on certain inputs: 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 63 64 62 61 58 59 60 57 54 55 56 52 53 50 51 49 46 47 48 44 45 42 43 39 40 41 37 38 36 34 35 33 30 31 32 28 29 26 27 23 24 25 21 22 20 18 19 15 16 17 13 14 11 12 8 9 10 6 7 5 4 2 3 1 intuitive problem: regularly very unbalanced merges Buss, Knop: Strategies for Stable Merge Sorting, SODA 2019 can do much better (merge cost 321 vs. 371) As n increases, 50% higher merge cost than standard mergesort Enough! Let’s fix this merge policy problem once and for all, shall we? Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
44. Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
45. 4 Merge policies 4 Merge policies Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 10 / 19
46. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
47. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
48. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
49. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
50. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
51. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
52. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
53. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
54. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
55. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
56. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
57. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
58. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
59. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 1 3 4 5 6 8 10 14 16 18 20 21 22 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
60. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 2 9 12 13 19 2 9 12 13 15 17 19 2 7 9 11 12 13 15 17 19 1 4 5 8 10 14 21 23 1 3 4 5 6 8 10 14 16 18 20 21 22 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
61. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
62. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
63. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
64. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
65. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
66. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
67. Merge policies from first principles Run-adaptive mergesort Concep interleaved in code tually two steps: 1 Find runs in input. 2 Merge them in some order determined by merge policy . Here: only binary merges ⇝ 2 becomes: merge 2 runs, repeat until single run only stable sorts ⇝ merge 2 adjacent runs ⇝ Merge order = merge tree: ⇝ Merge policy = algorithm to choose merge tree Merge costs cost of merge := size of output ≈ memory transfers ⩾ #cmps total cost = total area of Different merge policies yield different merge costs! 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 42 2 4 6 8 10 12 14 16 3 5 1 9 7 17 11 13 15 0 merge costs: 71 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 11 / 19
68. Mergesort meets Binary Search Trees 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Merge cost = total area of Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
69. Mergesort meets Binary Search Trees 15 17 12 19 2 9 13 7 11 1 4 8 10 14 23 5 21 3 6 16 18 20 22 Merge cost = total area of = total length of paths to all array entries Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
70. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
71. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
72. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
73. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
74. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
75. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
76. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
77. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
78. Mergesort meets Binary Search Trees 2 3 2 2 6 2 6 Merge cost = total area of = total length of paths to all array entries = weighted external path w leaf weight(w) · depth(w) length ⇝ optimal merge tree = optimal BST for leaf weights L run lengths 1 , . . . , Lr ⇝ merge cost ⩾ Hn with H = r i=1 Li n log2 n Li How to compute good merge trees? nearly-optimal ...70s are calling BST merge simple (greedy) linear-time methods! almost optimal (⩽ H + 2) ⇝ Powersort based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to perfect binary tree Find run boundary closest to middle Recurse on both sides But: Can’t use recursion! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 12 / 19
79. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
80. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
81. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
82. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
83. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
84. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
85. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
86. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
87. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
88. Run-Boundary Powers 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
89. Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
90. Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
91. Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
92. Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
93. Run-Boundary Powers 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 (virtual) perfect balanced binary tree midpoint intervals “snap” to closest virtual tree node ⇝ assigns each run boundary a depth = its power ⇝ merge tree follows virtual tree Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 13 / 19
94. Run-Boundary Powers are Local 4 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 Computation of powers only depends on two adjacent runs. Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 14 / 19
95. Outline 1 Sort of a list 1 Sort of a list 2 Timsort 2 Timsort 3 Beware, Stackoverflow! 3 Beware, Stackoverflow! 4 Merge policies 4 Merge policies 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 14 / 19
96. 5 Powersort 5 Powersort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 14 / 19
97. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
98. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 run1 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
99. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 run1 run2 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
100. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 run1 run2 a – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
101. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 run1 run2 a – 0 b – 3 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
102. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
103. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
104. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
105. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) a b c d e f 0 3 2 run1 run2 a – 0 b – 3 c – 2 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
106. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 run2 ab – 0 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
107. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
108. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
109. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
110. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) ab c d e f 0 2 1 run1 run2 ab – 0 c – 2 d – 1 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
111. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 run2 abc – 0 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
112. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 run1 run2 abc – 0 d – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
113. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 run1 run2 abc – 0 d – 1 e – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
114. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 run1 run2 abc – 0 d – 1 e – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
115. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 run1 run2 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
116. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
117. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
118. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d e f 0 1 2 4 abc – 0 d – 1 e – 2 f – 4 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
119. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d ef 0 1 2 abc – 0 d – 1 ef – 2 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
120. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc d ef 0 1 2 abc – 0 d – 1 ef – 2 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
121. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc def 0 1 abc – 0 def – 1 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
122. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) merge-down phase abc def 0 1 abc – 0 def – 1 run stack merge 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
123. Powersort 1 def powersort(lst) 2 i = 0; n = len(lst) 3 runs = []; 4 j = extend_run(lst, i) 5 runs.append((i,j-i,0)); i = j 6 while i < n: 7 j = extend_run(lst, i) 8 p = power(runs[-1], (i,j-i), n) 9 while p <= runs[-1][2] 10 merge_topmost_2(lst, runs) 11 runs.append((i,j-i,p)); i = j 12 while len(runs) >= 2: 13 merge_topmost_2(lst, runs) abcdef 0 abcdef – 0 run stack 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 24 25 26 27 28 21 22 23 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 3 1 2 full code: tiny.cc/timsort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 15 / 19
124. Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19
125. Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19
126. Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19
127. Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19
128. Computing powers Computing the power of (run boundary between) two runs = normalized midpoint interval power = min ℓ s.t. contains c · 2−ℓ Bi 0 1 2−p ai bi ℓi−1 ℓi Pi ⩽ p 1 def power(run1, run2, n): 2 i1, n1 = run1 3 i2, n2 = run2 4 a = (i1 + n1/2) / n 5 b = (i2 + n2/2) / n 6 l = 0 7 while math.floor(a * 2**l) == \ 8 math.floor(b * 2**l): 9 l += 1 10 return l 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 3⁄ 8 5⁄ 8 1⁄ 8 7⁄ 8 1⁄ 16 3⁄ 16 5⁄ 16 7⁄ 16 9⁄ 16 11⁄ 16 13⁄ 16 15⁄ 16 3 2 1 2 4 4 4 3 3 4 4 2 2 4 4 3 3 4 4 1 1 4 4 3 3 4 4 2 2 4 4 3 3 4 4 More efficient implementation possible (avoid multiplication / division in loop) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 16 / 19
129. Some performance data Summary claims: 1 Typical improvement from Powersort: 0-5% fewer comparisons; occasionally 20–30%. Data (One can contrive inputs where Powersort does worse; seems inevitable) 2 No running time regressions: never measurably slower in actual running time Data 3 Sometimes substantially faster (20–40%) Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 17 / 19
130. Bonus: Multiway powersort Timsort has been highly successful export from Python ... In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19
131. Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19
132. Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19
133. Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) * (My PhD was on how this affects Quicksort) Wild: Dual-Pivot Quicksort and Beyond: Analysis of Multiway Partitioning and Its Practical Potential, PhD thesis 2016 ⇝ Easy to do with Powersort while keeping adaptivity! Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19
134. Bonus: Multiway powersort Timsort has been highly successful export from Python ... = now using Powersort merge policy In other contexts, comparisons can be much cheaper ⇝ need to economize on memory transfers* ⇝ can profit from multiway merging (instead of 2 runs at a time) * (My PhD was on how this affects Quicksort) Wild: Dual-Pivot Quicksort and Beyond: Analysis of Multiway Partitioning and Its Practical Potential, PhD thesis 2016 ⇝ Easy to do with Powersort while keeping adaptivity! Cawley Gelling, Nebel, Smith, Wild: Multiway Powersort, ALENEX 2023 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 18 / 19
135. Conclusion Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
136. Conclusion Summary Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
137. Conclusion Summary Timsort is stable sorting method of choice Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
138. Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
139. Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
140. Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Goals 1 Powersort in other libraries using Timsort? numpy & pandas OpenJDK Android Java runtime V8 JavaScript engine GNU STL std::stable_sort Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
141. Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Goals 1 Powersort in other libraries using Timsort? numpy & pandas OpenJDK Android Java runtime V8 JavaScript engine GNU STL std::stable_sort 2 How much does adaptive sorting help? What are typical inputs like in difference applications? Is sorting used at all? Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
142. Conclusion Summary Timsort is stable sorting method of choice its original merge policy wasn’t ideal complicated correctness proof / analysis blind spots in performance Powersort fixes this! K And they lived merrily every after. k Goals 1 Powersort in other libraries using Timsort? numpy & pandas OpenJDK Android Java runtime V8 JavaScript engine GNU STL std::stable_sort 2 How much does adaptive sorting help? What are typical inputs like in difference applications? Is sorting used at all? Please help me with your data I only care about relative ordering ! Instructions to contribute: tiny.cc/sort-lake-city Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 19 / 19
143. Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 20 / 19
144. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
145. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
146. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
147. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
148. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
149. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
150. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
151. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
152. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
153. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
154. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
155. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
156. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
157. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
158. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
159. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
160. The Bisection Method Powersort is based on the bisection method Mehlhorn: A best possible bound for the weighted path length of binary search trees, SIAM J Comp 1977 Intuition: “Round” to a perfectly balanced binary tree Pretend array is interval [0, 1] Find run boundaries closest to middle Recurse on both sides 1⁄ 2 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 8 split out of range! no node created 1⁄ 2 1⁄ 4 3⁄ 4 1⁄ 8 7⁄ 815⁄ 16 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 21 / 19
161. Abstract cost measures back 1 1.05 1.1 1.15 1.2 1 1.05 1.1 Powersort better merge cost #comparisons Scatter plot of relative costs Timsort Powersort Timsort better Worse on 2-3%, but if so, only slightly. On average, 3% fewer cmps and 5% less merge cost. Input: Runs of expected length √n, n = 105 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 22 / 19
162. Abstract cost measures back 1 1.05 1.1 1.15 1.2 1 1.05 1.1 Powersort better merge cost #comparisons Scatter plot of relative costs Timsort Powersort Timsort better Worse on 2-3%, but if so, only slightly. On average, 3% fewer cmps and 5% less merge cost. Input: Tim’s mixture of long and short runs Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 22 / 19
163. Running times back 0 0.5 1 1.5 bad-example-cmps bad-example-cmps-1m bad-example-cmps-1m-2 bad-example-mc random-permutations random-permutations2 random-sqrtn-runs random-sqrtn-runs2 random-sqrtn-runs3 words-of-bible Powersort faster Timsort faster Timsort faster Timsort faster Average running times Timsort Powersort CPython 3.11 with Powersort resp. Timsort selection of some “random” inputs all using int (cheap comparisons) machine-dependent, but qualitatively stable Machine 1 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 23 / 19
164. Running times back 0 0.5 1 1.5 bad-example-cmps bad-example-cmps-1m bad-example-cmps-1m2 bad-example-mc random-permutations random-permutations2 random-sqrtn-runs random-sqrtn-runs2 random-sqrtn-runs3 words-of-bible Powersort faster Timsort faster Timsort faster Timsort faster Average running times Timsort Powersort CPython 3.11 with Powersort resp. Timsort selection of some “random” inputs all using int (cheap comparisons) machine-dependent, but qualitatively stable Machine 2 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 23 / 19
165. 4-way Powersort 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 24 / 19
166. 4-way Powersort 3 4 5 2 1 3 2 4 3 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 1 5 4 5 3 5 4 5 2 5 4 5 3 5 4 5 2 2 3 1 1 2 1 2 2 44 45 46 47 48 49 50 41 42 43 40 39 33 34 35 36 37 38 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 11 12 13 14 15 16 10 7 8 9 1 2 3 4 5 6 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 1 3 3 3 2 3 3 3 2 3 3 3 2 3 3 3 Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 24 / 19
167. Icons made by Freepik, Gregor Cresnar, Those Icons, Smashicons, Good Ware, Pause08, and Madebyoliver from www.flaticon.com. Vector graphics from Pressfoto, brgfx, macrovector and Jannoon028 on freepik.com Other photos from www.pixabay.com. Sebastian Wild Quicksort, Timsort, Powersort 2023-04-22 25 / 19