grouping_unclassified_-_tprc_2026.pdf

Classification is a powerful tool for data munging. Easy to do with raw Perl code, and even easier with a Perl module (List::Categorize) or Raku methods (.classify, .categorize) . Sail past SQL’s GROUP BY with Hashes of Arrays (HoA), HoHoA, HoHoHoA … ∞ !

Transcript

1. Grouping: UNCLASSIFIED ( with a waypoint on Counting )
 
 { in 2 + 2 languages } [ in less than 50 minutes ] The Perl and Raku Conference
 
 2026-06-26
2. Grouping: UNCLASSIFIED PLEASE DOWNLOAD THESE SLIDES! 
 http://speakerdeck.com/util
 <bruce.gray@acm.org> <<< >>> >>> <<<
3. What to expect
4. Last time, on Dragon Ball Z Fate Stay/Night Unlimited Blade Works, Abridged
5. Pacing 5
6. Grand Design • Manual code, long-form and minimal • Drop-in builder routines • Modules / Built-in • Example
7. Bruce Gray (Util)
8. Post-talk Feedback++ (Hallway, email, etc)
9. Q & A
10. Perl Header use v5.36; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; use utf8; use open qw<:std :utf8>; 10
11. Counting
12. Data: Array of Pets (species only)
13. Pets: JSON ["dog","cat","dog","cat","dog","dog"] 13
14. Pets Perl my @pets = qw<dog cat dog cat dog dog>; Raku my @pets = <dog cat dog cat dog dog>; Python pets = 'dog cat dog cat dog dog'.split() Javascript const qw = (s) => s[0].trim().split(/\s+/); const pets = qw`dog cat dog cat dog dog`; 14 …
15. Hardcoded : Pseudocode dogs <- 0 cats <- 0 for pet in pets { if pet == 'dog' { dogs += 1 } elsif pet == 'cat' { cats += 1 } else { complain } } 15
16. Hardcoded : Perl use v5.36; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; use utf8; use open qw<:std :utf8>; my @pets = qw<dog cat dog cat dog dog>; my ( $dogs, $cats ) = (0, 0); for my $pet (@pets) { if ($pet eq 'dog') { $dogs++ } elsif ($pet eq 'cat') { $cats++ } else { warn } } print Data::Dumper->Dump([$dogs, $cats], [qw<dogs cats>]); 16
17. Hardcoded : Raku my @pets = <dog cat dog cat dog dog>; my ( $dogs, $cats ) = 0, 0; for @pets { when 'dog' { $dogs++ } when 'cat' { $cats++ } default { warn } } say ( :$dogs, :$cats ); 17
18. Hardcoded : Perl use v5.36; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; use utf8; use open qw<:std :utf8>; my @pets = qw<dog cat dog cat dog dog>; my $dogs = 0+grep { $_ eq 'dog' } @pets; my $cats = 0+grep { $_ eq 'cat' } @pets; for (@pets) { $_ eq 'dog' or $_ eq 'cat' or warn; } print Data::Dumper->Dump([$dogs, $cats], [qw<dogs cats>]); 18
19. Hardcoded : Raku my @pets = <dog cat dog cat dog dog>; my $dogs = +grep 'dog', @pets; my $cats = +grep 'cat', @pets; $_ eq 'dog'|'cat' or warn for @pets; say ( :$dogs, :$cats ); 19
20. Bag
21. "Positive" does not include zero { "foo": "SomeString", "bar": -3, "baz": 1.5, "quux": 0, "quaz": 42, "quin": 1 } 21
22. Input and output in JSON Input: Pets (Array) ["dog","cat","dog","cat","dog","dog"] Output: Bag of Pets (Hash) { "dog": 4, "cat": 2 } 22
23. Softcoded : Pseudocode grouped <- new Hash for pet in pets if pet is not already in grouped grouped{pet} = 0 grouped{pet} = grouped{pet} + 1 print grouped{'dogs'} print grouped{'cats'} 23
24. Softcoded : Pseudocode grouped <- new Hash for pet in pets if pet is not already in grouped grouped{pet} = 0 grouped{pet} = grouped{pet} + 1 print grouped{'dogs'} print grouped{'cats'} 24
25. Bag: Javascript const qw = (s) => s[0].trim().split(/\s+/); const pets = qw`dog cat dog cat dog dog`; const h = {}; for ( const pet of pets ) { if ( !(pet in h) ) h[pet] = []; h[pet]++; } console.log(h); 25
26. Bag: Javascript const qw = (s) => s[0].trim().split(/\s+/); const pets = qw`dog cat dog cat dog dog`; const h = {}; for ( const pet of pets ) { h[pet] ??= 0; h[pet]++; } console.log(h); 26
27. Bag: Javascript const qw = (s) => s[0].trim().split(/\s+/); const pets = qw`dog cat dog cat dog dog`; function bag ( arr ) { const counts = {}; for ( const x of arr ) { counts[x] ??= 0; counts[x]++; } return counts; } console.log(h); 27
28. Bag: Javascript const qw = (s) => s[0].trim().split(/\s+/); const pets = qw`dog cat dog cat dog dog`; function bag ( arr ) { const counts = {}; for ( const x of arr ) { counts[x] ??= 0; counts[x]++; } return counts; } const h = bag(pets); console.log(h); 28
29. Bag: Python pets = 'dog cat dog cat dog dog'.split() h = {} for pet in pets: if pet not in h: h[pet] = 0 h[pet] += 1 print(h) 29
30. Bag: Python pets = 'dog cat dog cat dog dog'.split() h = {} for pet in pets: h.setdefault(pet, 0) h[pet] += 1 print(h) 30
31. Bag: Python pets = 'dog cat dog cat dog dog'.split() def bag (arr): counts = {} for x in arr: counts[x] = counts.setdefault(x, 0) + 1 return counts h = bag(pets) print(h) 31
32. Bag: Python pets = 'dog cat dog cat dog dog'.split() from collections import defaultdict def bag (arr): counts = defaultdict(int) for x in arr: counts[x] += 1 return dict(counts) h = bag(pets) print(h) 32
33. Bag: Python pets = 'dog cat dog cat dog dog'.split() from collections import Counter h = Counter(pets) print(h) 33
34. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for my $pet (@pets) { if ( ! exists $h{$pet} ) { $h{$pet} = 0; } $h{$pet}++; } print Dumper \%h; 34
35. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for my $pet (@pets) { $h{$pet} = 0 if !exists $h{$pet}; $h{$pet}++; } print Dumper \%h; 35
36. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for my $pet (@pets) { $h{$pet} = 0 if !defined $h{$pet}; $h{$pet}++; } print Dumper \%h; 36
37. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for my $pet (@pets) { $h{$pet} //= 0; $h{$pet}++; } print Dumper \%h; 37
38. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for my $pet (@pets) { $h{$pet}++; } print Dumper \%h; 38
39. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; for (@pets) { $h{$_}++; } print Dumper \%h; 39
40. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; $h{$_}++ for (@pets); print Dumper \%h; 40
41. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; $h{$_}++ for @pets; print Dumper \%h; 41
42. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; my %h; $h{$_}++ for @pets; print Dumper \%h; 42
43. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; sub bag (@arr) { my %counts; $counts{$_}++ for @arr; return %counts; } my %h = bag(@pets); print Dumper \%h; 43
44. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; use List::MoreUtils qw<frequency>; my %h = frequency @pets; print Dumper \%h; 44
45. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; use List::MoreUtils qw<frequency>; my %h = frequency @pets; use Set::Bag; print Dumper \%h; say $bag; 45
46. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; use List::MoreUtils qw<frequency>; my %h = frequency @pets; use Set::Bag; my $bag = Set::Bag->new(@pets); print Dumper \%h; say $bag; 46
47. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; use List::MoreUtils qw<frequency>; my %h = frequency @pets; use Set::Bag; my $bag = Set::Bag->new(@pets); print Dumper \%h; say $bag; 47
48. Bag: Perl use v5.36; use utf8; use open qw<:std :utf8>; use Data::Dumper; $Data::Dumper::Sortkeys = 1; $Data::Dumper::Useqq = 1; my @pets = qw<dog cat dog cat dog dog>; use List::MoreUtils qw<frequency>; my %h = frequency @pets; use Set::Bag; my $bag = Set::Bag->new( frequency @pets ); print Dumper \%h; say $bag; 48
49. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h; for @pets -> $pet { if %h{$pet}:!exists { %h{$pet} = 0; } %h{$pet}++; } say %h; 49
50. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h; %h{$_}++ for @pets; say %h; 50
51. Bag: Raku my @pets = <dog cat dog cat dog dog>; sub bag_it (@arr) { my %counts; %counts{$_}++ for @arr; return %counts; } my %h = bag_it(@pets); say %h; 51
52. Bag: Raku my @pets = <dog cat dog cat dog dog>; sub bag_it (@arr) { my %counts; %counts{$_}++ for @arr; return %counts; } my %h = bag_it(@pets); say %h; 52
53. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h = bag @pets; say %h; 53
54. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h1 = bag @pets; my %h2 = @pets.Bag; say (:%h1, :%h2); 54
55. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h1 = bag @pets; my %h2 = @pets.Bag; %h2{'dog'} += 1/2; say (:%h1, :%h2); 55
56. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h1 = bag @pets; my %h2 = @pets.Bag; my $bag_o_pets = @pets.Bag; say (:%h1, :%h2, :$bag_o_pets); 56
57. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h1 = bag @pets; my %h2 = @pets.Bag; my $bag_o_pets = @pets.Bag; #Immutable say (:%h1, :%h2, :$bag_o_pets); 57
58. Bag: Raku my @pets = <dog cat dog cat dog dog>; my %h1 = bag @pets; my %h2 = @pets.Bag; my $bag_o_pets = @pets.BagHash; say (:%h1, :%h2, :$bag_o_pets); 58
59. Be clear on how you use each Bag. A. Build once, use once or more.
 Or, B. Initialize, update gradually, use the totals after all updates.
 Or, C. Initialize, update gradually while using the intermediate values (and maybe the totals after fi nal update too).
60. Counting : Example (Mutable)
61. Bag : Real World : Perl my $age_error; for my $person (@people) { $age_error++ if ...; ... } say "Age error:" $age_error if $age_error; 61
62. Bag : Real World : Perl my $smo_error; my $age_error; for my $person (@people) { $age_error++ if ...; ... } say "Age error:" $age_error if $age_error; 62
63. Bag : Real World : Perl my $smo_error; my $age_error; for my $person (@people) { $age_error++ if ...; $smo_error++ if ...; ... } say "Age error:" $age_error if $age_error; 63
64. Bag : Real World : Perl my $smo_error; my $age_error; for my $person (@people) { $age_error++ if ...; $smo_error++ if ...; ... } say "Age error:" $age_error if $age_error; say "Smo error:" $smo_error if $age_error; 64
65. Bag : Real World : Perl my $smo_error; my $age_error; for my $person (@people) { $age_error++ if ...; $smo_error++ if ...; ... } say "Age error:" $age_error if $age_error; say "Smo error:" $smo_error if $age_error; 65
66. Bag : Real World : Perl my $smo_error; my $age_error; for my $person (@people) { $age_error++ if ...; $smo_error++ if ...; ... } say "Age error:" $age_error if $age_error; say "Smo error:" $smo_error if $smo_error; 66
67. Bag : Real World : Perl my %err; # Bag of error counts for my $person (@people) { $err{age}++ if ...; $err{smo}++ if ...; ... } say "$_ error: $err{$_}" for sort keys %err; 67
68. Counting : Covered • Manual code, long-form and minimal • Drop-in builder routines • Modules / Built-in • Example
69. Grouping
70. Data: Array of Cards
71. Unicode Rocks! 🂡 🂱 🃁 🃑 🂢 🂲 🃂 🃒 🂣 🂳 🃃 🃓 🂤 🂴 🃄 🃔 🂥 🂵 🃅 🃕 🂦 🂶 🃆 🃖 🂧 🂷 🃇 🃗 🂨 🂸 🃈 🃘 🂩 🂹 🃉 🃙 🂪 🂺 🃊 🃚 🂫 🂻 🃋 🃛 🂭 🂽 🃍 🃝 🂮 🂾 🃎 🃞 71
72. Not playing with a full deck 72 🂩 🃉 🃙 🂹 🂪 🃊 🃚 🂺 🂭 🃍 🃝 🂽 🂮 🃎 🃞 🂾
73. Cards: Graphic Output 73 🂩 🂪 🂭 🂮 🂹 🂺 🂽 🂾 🃉 🃊 🃍 🃎 🃙 🃚 🃝 🃞 – : ‖ : ― : — :
74. Cards: JSON Input Output # Input: Flat array of cards [ "9♠", "9♦", "9♣", "9♥", "10♠", "10♦", "10♣", "10♥", "Q♠", "Q♦", "Q♣", "Q♥", "K♠", "K♦", "K♣", "K♥" ] # Output: Hash of Arrays (HoA) { "♠": [ "9♠", "10♠", "Q♠", "K♠" ], "♦": [ "9♦", "10♦", "Q♦", "K♦" ], "♣": [ "9♣", "10♣", "Q♣", "K♣" ], "♥": [ "9♥", "10♥", "Q♥", "K♥" ] } 74
75. Grouping Code : Cards
76. Cards: Deck Suit Rank my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; # Raku sub suit { $^card.substr( *-1) } sub rank { $^card.substr(0,*-1) } my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; # Perl sub suit ($card) { substr($card, -1) } sub rank ($card) { substr($card, 0,-1) } deck = [x+y for x in ('9 10 Q K').split() # Python for y in ('♠ ♦ ♣ ♥').split()] def suit (card): return card[-1] def rank (card): return card[:-1] const deck = ['9♠','9♦','9♣','9♥','10♠','10♦','10♣','10♥', # JS 'Q♠','Q♦','Q♣','Q♥', 'K♠', 'K♦', 'K♣', 'K♥']; function suit (card) { return card.substring( card.length - 1) } function rank (card) { return card.substring(0, card.length - 1) } 76
77. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for my $card (@deck) { my $key = suit($card); push @{ $h{$key} }, $card; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 77
78. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for my $card (@deck) { my $key = suit($card); push @{ $h{$key} }, $card; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 78
79. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for (@deck) { my $key = suit($_); push @{ $h{$key} }, $_; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 79
80. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for (@deck) { push @{ $h{ suit($_) } }, $_; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 80
81. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; push @{$h{suit($_)}}, $_ for @deck; say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 81
82. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for my $card (@deck) { my $key = suit($card); push @{ $h{$key} }, $card; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 82
83. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } my %h; for my $card (@deck) { my $key = suit($card); push @{ $h{$key} }, $card; } say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 83
84. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify (@items) { my %h; for my $card (@deck) { my $key = suit($card); push @{ $h{$key} }, $card; } } # Will be non-functional until we finish the refactor 84
85. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify (@items) { my %ret; for my $card (@deck) { my $key = suit($card); push @{ $ret{$key} }, $card; } } # Will be non-functional until we finish the refactor 85
86. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify (@items) { my %ret; for my $item (@items) { my $key = suit($item); push @{ $ret{$key} }, $item; } } # Will be non-functional until we finish the refactor 86
87. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify (@items) { my %ret; for my $item (@items) { my $key = suit($item); push @{ $ret{$key} }, $item; } } # Will be non-functional until we finish the refactor 87
88. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify ($key_maker, @items) { my %ret; for my $item (@items) { my $key = $key_maker->($item); push @{ $ret{$key} }, $item; } } # Will be non-functional until we finish the refactor; 88
89. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify ($key_maker, @items) { my %ret; for my $item (@items) { my $key = $key_maker->($item); push @{ $ret{$key} }, $item; } return %ret; } # Will be non-functional until we finish the refactor 89
90. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } sub classify ($key_maker, @items) { my %ret; for my $item (@items) { my $key = $key_maker->($item); push @{ $ret{$key} }, $item; } return %ret; } my %h = classify \&suit, @deck; say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 90
91. Cards: Perl use v5.36; use utf8; use open qw<:std :utf8>; my @deck = map {("9$_","10$_","Q$_","K$_")} qw<♠ ♦ ♣ ♥>; sub suit ($card) { substr($card, -1) } sub val ($card) { substr($card, 0,-1) } use List::Categorize qw<categorize>; my %h = categorize { suit($_) } @deck; say "$_ : ", join(" ", @{ $h{$_} }) for sort keys %h; 91
92. Cards: Perl use List::MoreUtils qw<part>; # ILL-ADVISED: Integers only 92
93. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; for @deck -> $card { my $key = suit($card); %h{$key}.push: $card; } .say for %h.sort; 93
94. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; for @deck { my $key = suit($_); %h{$key}.push: $_; } .say for %h.sort; 94
95. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; for @deck { my $key = .&suit; %h{$key}.push: $_; } .say for %h.sort; 95
96. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; for @deck { %h{ .&suit }.push: $_; } .say for %h.sort; 96
97. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; for @deck { %h{ .&suit }.push($_); } .say for %h.sort; 97
98. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; %h{ .&suit }.push($_) for @deck; .say for %h.sort; 98
99. Cards: Raku sub classify_it ( @items, &key_maker ) { my %ret; %ret{.&key_maker}.push: $_ for @items; return %ret; } my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h; %h = classify_it(@deck, &suit); %h = @deck.&classify_it(&suit); .say for %h.sort; 99
100. Cards: Raku my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h = @deck.classify(&suit); .say for %h.sort; 100
101. Cards: Javascript const qw = (s) => s[0].trim().split(/\s+/); const deck = qw`9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥`; function suit (card) { return card.substring( card.length - 1) } function val (card) { return card.substring(0, card.length - 1) } const h = {} for ( const card of deck ) { key = suit(card) h[key] ??= [] h[key].push(card) } console.log(h) 101
102. Cards: Javascript const qw = (s) => s[0].trim().split(/\s+/); const deck = qw`9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥`; function suit (card) { return card.substring( card.length - 1) } function val (card) { return card.substring(0, card.length - 1) } function classify ( array, code ) { const ret = {}; for ( const item of array ) { key = code(item) ret[key] ??= []; ret[key].push(item); } return ret; } console.log( classify(deck, suit) ); 102
103. Cards: Javascript const qw = (s) => s[0].trim().split(/\s+/); const deck = qw`9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥`; function suit (card) { return card.substring( card.length - 1) } function val (card) { return card.substring(0, card.length - 1) } function classify ( array, code ) { const ret = {}; for ( const item of array ) { key = code(item) ret[key] ??= []; ret[key].push(item); } return ret; } console.log( classify(deck, suit) ); console.log( Object.groupBy(deck, suit) ); console.log( Map.groupBy(deck, suit) ); 103
104. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] h = {} for card in deck: k = suit(card) h.setdefault(k, []).append(card) for k, v in h.items(): print(k, v) 104
105. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] def classify ( code, array ): ret = {} for x in array: key = code(x) ret.setdefault(key, []).append(x) return ret h = classify(suit, deck) for k, v in h.items(): print(k, v) 105
106. Cards: remember that Raku? my @deck = <9 10 Q K> X~ <♠ ♦ ♣ ♥>; sub suit ($card) { $card.substr( *-1) } sub val ($card) { $card.substr(0,*-1) } my %h = @deck.classify(&suit); .say for %h.sort; 106
107. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(deck, key=suit) 107
108. Cards: JSON Wanted Output # Output: Hash of Arrays (HoA) { "♠": [ "9♠", "10♠", "Q♠", "K♠" ], "♦": [ "9♦", "10♦", "Q♦", "K♦" ], "♣": [ "9♣", "10♣", "Q♣", "K♣" ], "♥": [ "9♥", "10♥", "Q♥", "K♥" ] } 108
109. Cards: WTF? { "♠": [ "K♠" ], "♦": [ "K♦" ], "♣": [ "K♣" ], "♥": [ "K♥" ] } 109
110. Cards: WTF? { "♠": [ "9♠" ], "♦": [ "9♦" ], "♣": [ "9♣" ], "♥": [ "9♥" ], "♠": [ "10♠" ], "♦": [ "10♦" ], "♣": [ "10♣" ], "♥": [ "10♥" ], "♠": [ "Q♠" ], "♦": [ "Q♦" ], "♣": [ "Q♣" ], "♥": [ "Q♥" ], "♠": [ "K♠" ], "♦": [ "K♦" ], "♣": [ "K♣" ], "♥": [ "K♥" ] } 110
111. Cards: WTF? % echo "foo\nbar\nfoo" | uniq foo bar foo % echo "foo\nbar\nfoo" | sort | uniq bar foo 111
112. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(deck, key=suit) # Wrong! 112
113. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(sorted(deck, key=suit),key=suit) 113
114. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(deck, key=suit) # Wrong! for k, v in h.items(): # Wrong! print(k, v) 114
115. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(sorted(deck, key=suit),key=suit) for k, v in h.items(): # Wrong! print(k, v) 115
116. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(sorted(deck, key=suit),key=suit) #for k, v in h.items(): # Wrong! # print(k, v) for [k, hashref] in h: print(k, list(hashref)) 116
117. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(sorted(deck, key=suit),key=suit) # Change from iterators to simple dict of lists. h = { k: list(v) for k, v in h } for k, v in h.items(): # Works now print(k, v) 117
118. Cards: Python deck = '9♠ 9♦ 9♣ 9♥ 10♠ 10♦ 10♣ 10♥ Q♠ Q♦ Q♣ Q♥ K♠ K♦ K♣ K♥'.split() def suit (card): return card[-1] def val (card): return card[:-1] from itertools import groupby h = groupby(deck, key=suit) # Workaround to allow a JSON dump of `h` to work, to understand `groupby` behavior when unsorted. list_of_lists = [ [k, list(v)] for k, v in h ] import json print(json.dumps(list_of_lists)) 118
119. Grouping Real-world Example: Filled
120. Data: My Big Fat Greek Home Directory
121. raku -e '.say for dir().classify(*.extension).nodemap(*».s.sum).sort(-*.value);' webm => 37991264753 mkv => 12616371486 part => 1281576336 mp4 => 1082287862 ... 121
122. .say for dir().classify(*.extension) .nodemap(*».s.sum) .sort(-*.value); raku -e '.say for dir().classify(*.extension).nodemap(*».s.sum).sort(-*.value);' webm => 37991264753 mkv => 12616371486 part => 1281576336 mp4 => 1082287862 ... 122
123. Extension sizes: Raku raku -e 'for dir().classify(*.extension).sort(-*.value>>.s.sum).head(5) -> (:key($k), :value(@f)) {say @f>>.s.sum.fmt("%15d\t"), $k; say .s.fmt("\t%15d\t"), .path for @f.sort(-*.s).head(5) }' 123
124. Extension sizes: Raku raku -e 'for dir().classify(*.extension).sort(-*.value>>.s.sum).head(5) -> (:key($k), :value(@f)) {say @f>>.s.sum.fmt("%15d\t"), $k; say .s.fmt("\t%15d\t"), .path for @f.sort(-*.s).head(5) }' raku -e 'for dir().classify(*.extension).sort(-*.value>>.s.sum).head(5) -> (:key($k), :value(@f)) {say @f>>.s.sum.fmt("%15d\t"), $k; say .s.fmt("\t%15d\t"), .path for @f.sort(-*.s).head(5) }' 124
125. Extension sizes: Raku raku -e 'for dir().classify(*.extension).sort(-*.value>>.s.sum).head(5) -> (:key($k), :value(@f)) {say @f>>.s.sum.fmt("%15d\t"), $k; say .s.fmt("\t%15d\t"), .path for @f.sort(-*.s).head(5) }' raku -e 'for dir().classify(*.extension).sort(-*.value>>.s.sum).head(5) -> (:key($k), :value(@f)) {say @f>>.s.sum.fmt("%15d\t"), $k; say .s.fmt("\t%15d\t"), .path for @f.sort(-*.s).head(5) }' for dir().classify(*.extension).sort(-*.value».s.sum) { say (.value».s.sum, .key); say "\t", (.s, .path) for .value.sort(-*.s) } 125
126. Extension sizes: Raku for dir().classify(*.extension).sort(-*.value».s.sum) { say (.value».s.sum, .key); say "\t", (.s, .path) for .value.sort(-*.s) } 126
127. Extension sizes: Raku for dir().classify(*.extension).sort(-*.value».s.sum) { say "### {.value».s.sum} {.key}"; say "\t{.s} {.path}" for .value.sort(-*.s) } 127
128. Argentine Tango ### 37881174233 .webm 1154235480 Tango Combination： Sa[Qb5vZ52_ToQ].webm 872053799 4 Mistakes during Ochos[85CknRy7HSc].webm … ### 12349378480 .mkv 399568606 Advance Tango Figures. [FF_lrKBTW7I].mkv 351719890 Tango turn with lapiz [vyKgibz2wKM].mkv … 128
129. `yt-dlp` + `vlc` WTF!
130. Extension sizes: Perl use v5.36; use List::Util qw<sum>; use List::Categorize qw<categorize>; sub ext ($f) { $f =~ /\.(.+?)$/ ? $1 : '<none>' } my @paths = sort { $b->[0] <=> $a->[0] } map { -f $_ ? [-s _, ext($_), $_] : () } glob '.'; my %by_ext = categorize { $_->[1] } @paths; my %size = map { $_ => sum map { $_->[0] } @{$by_ext{$_}} } keys %by_ext; my @exts = sort { $size{$b} <=> $size{$a} } keys %size; for my $ext (@exts) { say "### $size{$ext} $ext"; say "\t$_->[0] $_->[2]" for @{$by_ext{$ext}}; } 130
131. Extension sizes: Javascript const dir = '.'; const fs = require('fs'); const path = require('path'); const sum = (arr) => arr.reduce( ( (acc, x) => acc + x ), 0 ); function get_file_info (f) { try { const stats = fs.statSync(dir + '/' + f); if (stats.isFile()) { return [ stats.size, path.extname(f), f ]; } } catch (err) { console.error(err); } return null; } const file_info = fs.readdirSync(dir).map(get_file_info).filter((x) => x !== null); const h = Object.groupBy( file_info, ([size,ext,file]) => ext ); const size_e = {}; for (const [ext, fis] of Object.entries(h)) { size_e[ext] = sum(fis.map(([s,e,f]) => s)); } for ( const [ext, size] of Object.entries(size_e).toSorted((a,b) => b[1] - a[1]) ) { console.log(`${size}\t${ext}`); for ( const [s,e,f] of h[ext].toSorted((a, b) => b[0] - a[0]) ) { console.log(`\t${s}\t${f}`); } } 131
132. Extension sizes: Raku for dir().classify(*.extension).sort(-*.value».s.sum) { say "### {.value».s.sum} {.key}"; say "\t{.s} {.path}" for .value.sort(-*.s) } 132
133. Extension sizes: Python from os import listdir from os.path import isfile,splitext,getsize files_info = [ [getsize(f), splitext(f)[1], f] for f in listdir('.') if isfile(f) ] def iSizeN (info): return -info[0] def iExt (info): return info[1] from itertools import groupby by_ext_iter = groupby(sorted(files_info, key=iExt), key=iExt) by_ext = { k: list(v) for k, v in by_ext_iter } size = { e : sum(x[0] for x in arr) for e, arr in by_ext.items() } for ext in sorted(size.keys(), key=lambda x: -size[x]): print("### ", size[ext], ext) for t in sorted(by_ext[ext], key=iSizeN): print("\t", t[0], t[2]) 133
134. Extension sizes: Python with comments from os import listdir from os.path import isfile,splitext,getsize # Get all files and their extensions and sizes. files_info = [ [getsize(f), splitext(f)[1], f] for f in listdir('.') if isfile(f) ] # Slots of (negative) size and extension, for sorting. def iSizeN (info): return -info[0] def iExt (info): return info[1] # Group by extension from itertools import groupby by_ext_iter = groupby(sorted(files_info, key=iExt), key=iExt) # Change from iterators to simple dict of lists. by_ext = { k: list(v) for k, v in by_ext_iter } # Make a dict of total sizes per extension. size = { e : sum(x[0] for x in arr) for e, arr in by_ext.items() } # Print heading and detail lines for each group. for ext in sorted(size.keys(), key=lambda x: -size[x]): print("### ", size[ext], ext) for t in sorted(by_ext[ext], key=iSizeN): print("\t", t[0], t[2]) 134
135. Cards : Covered • Manual code, long-form and minimal • Drop-in builder routines • Modules / Built-in • Example
136. Grouping++
137. Data: My Big Fat Greek Harddrive
138. find ~ -type f -size +1G -print0 > a.0 xargs -0 <a.0 md5sum 138
139. MD5 fi ngerprints of many fi les 873f872f5661c4a6e7e30b1f64a6902b /Users/bruce/Downloads/GMT20250227-161215_Recording_640x360.mp4 873f872f5661c4a6e7e30b1f64a6902b /Users/bruce/Downloads/GMT20250227-161215_Recording_640x360 (1).mp4 139
140. MD5: Raku in 1 raku -e 'say "\n", .join("\n") for lines()».split(" ", 2).classify(*.[0], :as{.[1]}).values.grep(*.elems > 1).sort(-*.elems);' md5s 140
141. Perl my %h; while (<>) { chomp; my ($md5, $path) = split ' ', $_, 2; push @{ $h{$md5} }, $path; } my @v = grep { @{$_} > 1 } values %h; @v = sort { $#$b <=> $#$a } @v; for my $aref (@v) { say ''; say for $aref->@*; } 141
142. Perl my %h; while (<>) { chomp; my ($md5, $path) = split ' ', $_, 2; push @{ $h{$md5} }, $path; } 142
143. Perl use List::Categorize qw<categorize>; my @md5_paths; while (<>) { chomp; push @md5_paths, [split ' ', $_, 2]; } my %h = categorize { my ($md5, $path) = @{$_}; $_ = $path; # Transform $md5; # First-level category } @md5_paths; 143
144. MD5 : Raku w/o .classify my %h; for lines() { my ($md5, $path) = .split(" ", 2); %h{$md5}.push: $path; } my @AoAoPaths = %h.values .grep(*.elems > 1) .sort(-*.elems); for @AoAoPaths -> @paths { say ''; .say for @paths; } 144
145. MD5 : Raku raku -e ' say "\n", .join("\n") for lines().map(*.split(" ", 2)) .classify(*.[0], :as{.[1]}) .values .grep(*.elems > 1) .sort(-*.elems); ' file_with_md5s.txt 145
146. Python import fileinput md5_paths = [] for line in fileinput.input(): md5_paths.append( line.rstrip().split(' ', 1) ) get_md5 = lambda x: x[0] from itertools import groupby iter1 = groupby(sorted(md5_paths, key=get_md5), key=get_md5) for [md5, iter2] in iter1: mps = list(iter2) if (len(mps) > 1): print('') for mp in mps: print(mp[1]) # Note the `[1]` 146
147. Javascript # This page intentionally left blank 147
148. Grouping++ : Covered • Manual code, long-form and minimal • Drop-in builder routines • Modules / Built-in • Example
149. Q & A ? (vs. Bonus Topic)
150. Bonus topics • :into • Multi-level • Multi-category • .classify-list, .categorize-list
151. Built-in Language features and modules 151 Count Group Multi-category Multi-level :as :into Raku Bag, BagHash .classify .categorize .classify
 .categorize Y Y Perl Set::Bag, List::MoreUtils#frequency List::Categorize List::Categorize Y JS Object.groupBy() Map.groupBy() Python collections.Counter itertools.groupby
 defaultdict + append
 Pandas:df.groupby Ruby Enumerable#group_by Java groupingBy partitionBy
152. Thank you for coming!
153. Helpful links • https://en.wikipedia.org/wiki/Playing_cards_in_Unicode • Raku • https://docs.raku.org/language/setbagmix • https://docs.raku.org/type/Bag • Perl • https://metacpan.org/pod/List::MoreUtils#frequency-LIST • https://metacpan.org/pod/List::Categorize • https://metacpan.org/pod/Set::Bag • Python • https://docs.python.org/3/library/collections.html#collections.Counter • https://docs.python.org/3/library/itertools.html#itertools.groupby • Javascript • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/groupBy • https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/groupBy • Software used to generate the video clips: • Youtube Downloader • https://github.com/yt-dlp/yt-dlp • VLC player • https://www.videolan.org/vlc/features.html
154. Copyright Information: Images • Camelia • © 2009 by Larry Wall
 http://github.com/perl6/mu/raw/master/misc/camelia.txt • "Catch your breath" clip
 Extracted from https://www.youtube.com/watch?v=Bqli_zdxri8 Fate/Stay Night UBW Abridged - Ep10: All Aias On You
 Downloaded with `yt-dlp`, then clipped with `vlc`:
 vlc -I dummy 'Fate⧸Stay Night UBW Abridged - Ep10： All Aias On You [Bqli_zdxri8].webm' -- start-time=473 --stop-time=482 --sout "#gather:std{access= fi le,dst=breath.webm}" vlc://quit • https://www.youtube.com/shorts/bFPNQLiwfDA
 50 - "OH GOD NO" (UBW:A Ep9) #fate #fatestaynight #anime • https://www.youtube.com/shorts/rzW1_HISa3c
 48 - "Words" (UBW:A Ep9) #fate #fatestaynight #anime • https://www.youtube.com/shorts/ALsGFB2zPKY
 45 - "Manic Pixie Dream Girl" (UBW:A Ep8) #fate #fatestaynight #anime
155. LICENSE • © 2026 Bruce Gray <bruce.gray@acm.org>
 This work is openly licensed under CC BY 4.0.
 https://creativecommons.org/licenses/by/4.0/
156. Past this point are junk and archived slides Not intended to be of any use to you