Refactoring LinkedList::Single for Perl's new OO Model

LinkedList::Single works, but it fell into Perl's trap of not separating the List and iterators. Perl's new OO model gives us a simple, performant solution to objects, Object::Pad gives us a simple way to use the new model, with extensions. This talk describes how I've refactored LinkedList::Single using Object::Pad and how separating the List, Iterators, and Fragments into separate classes simplifies using the list and improves their performance.

Transcript

1. Padding Linked Lists Improving LL::S with Object::Pad Steven Lembark Workhorse Computing lembark@wrkhors.com
2. Object::Pad The Next Generation ™ of Perly OO. Classes, Roles, fields... all the toys everyone wanted. One really nice thing: Objects are closures. Fields are kept alive by reference count. When the object goes away so does its fields.
3. Object::Pad The Next Generation ™ of Perly OO. Classes, Roles, fields... all the toys everyone wanted. One really nice thing: Objects are closures. Fields are kept alive by reference count. When the object goes away so does its fields. Anyone awake last year?
4. Quick Review Object::Pad is a sandbox for new OOP. Basics won’t change much: class name version role name version field variable :param :reader = X
5. Quick Review Object::Pad is a sandbox for new OOP. Why “Pad”? Objects are closures over their fields. Uses Perl’s “pad” structure to hold them. Same place lexical var’s live.
6. Quick Review Object::Pad is a sandbox for new OOP. Why “Pad”? Accessing the pad is fast. The values are private.
7. The Old LinkedList::Single Keeping the entire list alive requires a head. Separate from the list handler itself. sub construct { my $proto my $listh = shift; = bless \( [ [] ] ), ref $proto || $proto; $rootz{ refaddr $listh } = $$listh; $listh }
8. The Old LinkedList::Single $listh is a ref-to-scalar. Assigning $$node walks the list. Only thing blessed is $listh. sub construct { my $proto my $listh = shift; = bless \( [ [] ] ), ref $proto || $proto; $rootz{ refaddr $listh } = $$listh; $listh }
9. The Old LinkedList::Single List entries are [ $next_ref , @node_data ] Walking the list extracts the next node and data. $$node = $rootz{ refaddr $node }[0]; while( @$node ) { ( $$node, @data ) = @$$node; ... }
10. The Old LinkedList::Single $listh is a cursor: It moves along the list. Without separate objects this convolves list and cursor. You have to reset the list each time you use it. Same issues as Perl5’s each().
11. The Old LinkedList::Single It also gets tricky to code with $$node. sub next_node( $node ) { $$node->[0] }
12. The New LinkedList::Single package LinkedList::Single use v5.40; use Carp use Sub::Name use Symbol v2.0.0; qw( croak qw( subname qw( qualify qualify_to_ref ); ); ); sub import( $, $vers = 'v1' ) <-- default is the old module! { $vers =~ m{^ v[12] $}x or croak "Bogus version: '$vers' is not 'v1'/'v2'"; our $pkg = qualify $vers; eval "require $pkg" // croak "Failed require $pkg: $@"; }
13. The New LinkedList::Single use LinkedList::Single qw( v2 ); Loads v2.pm Three classes based on four roles.
14. The New LinkedList::Single It all begins with the NodeMgr role. This contains the $node struct: An arrayref. This is the only data in any of the classes. NodeMgr is the only role that looks inside of $node.
15. The New LinkedList::Single Looks like the old list, but without the “$$”. sentinel_node() returns an empty arrayref. Used to instantiate lists, push new nodes. NodeMgr encapsulates the structure.
16. Using LL::S Starts with a List: my $list = LinkedList::Single->new; New lists are always empty. They are populated in a variety of ways: my $list = LinkedList::Single ->new ->lazy_generate( $node_data_generator );
17. Lists don’t move They provide access to the head node: shift(), unshift(), head() Not exciting, but they do keep the list alive.
18. Lists don’t move shift or head of empty list is an exception. Common throughout: Mistakes are exceptional.
19. Cursors move They do most of the work. Separate from lists, they are lightweight and fast. Move with advance, next( N ). Bulk operations like each, grep, map, first, sort. Separating cursors makes bulk operations restart-able.
20. Cursors move $pass1 and $pass2 are independent cursors. $list isn’t affected by $pass1 or $pass2. $pass2 isn’t affected by the existance of pass1. my $pass1 = $list->each( $handler1 ); my $pass2 = $list->each( $handler2 );
21. Searches return a cursor Empty lists are false. Cursors walked off the list are false. $list or return; my $found = $list->fist( $finder ) or say “Didn’t find what you wanted...”;
22. Bulk operations take closures map, grep, first, each All take a closure. Encapsulates processing. No need to derive anything from LL::S classes.
23. Bulk operations take closures All the data from grep, map returned as a flat list. $select doesn’t need to be OO or derived from LL::S. my $rx = qr{ whatever }x; my $select = sub { $_[0] =~ $rx }; my @data = $list->grep( $select );
24. Bulk operations take closures Node contents may be objects. No relation to LL::S, call their own methods. my $select = sub { $_[0]->is_useful }; my @data = $list->grep( $select );
25. Bulk operations take closures Map can return per-node contents as a ref. my $handler = sub { \@_ }; my @data = $list->map( $handler );
26. Each is unusual: It gets a cursor each gets $self: It can modify the list. Similar to Perl’s each(). Closures simplify the handling: No need to derive classes with new methods. No need for OO at all where it isn’t useful.
27. Each is unusual: It gets a cursor Use an object’s status to drop it from the run queue: my $expire_old = sub( $curs ) { my ( $job ) }; = $curs->data; $curs->drop if $job->expired; $queue->each( $expire_old );
28. The last class: Fragment. This exists for transient lists. Say you try to append a piece of one list to another: $list1->push_list # takes cursor ( $list2->find( $select ) # returns cursor ); Catch: You end up with a multi-headed list!
29. The last class: Fragment. The cursor’s parent node cannot be updated! The old list references the cursor. The new list references the cursor. Oops...
30. The last class: Fragment. The cursor’s parent node cannot be updated! The old list references the cursor. The new list references the cursor. Fix: Fragments truncate their source. This leaves the source node a sentinel. The detached list is headless.
31. ADJUST does the magic This block is run for every class/role at construction. It gets the completely assembled object. Can munge the contents. Common uses: args, validation, adjusting values.
32. ADJUST does the magic LL::S has an ADJUST in NodeMgr. It could have been spread out but this was simpler. Construction gets one argument: $whence. Question is what to do with it...
33. ADJUST for a List $foobar->new( whence => $whatever ) ADJUST :params ( :$whence = undef ) { my $name = $self->META->name; # default is undef if( $self->isa( $list_c ) ) { $whence and croak "Botched $name: extraneous 'whence' argument"; $node } ... = $sentinel_node->();
34. ADJUST for a List $foobar->new( whence => $whatever ) ADJUST :params ( :$whence = undef ) { my $name = $self->META->name; if( $self->isa( $list_c ) ) { $whence and croak "Botched $name: extraneous 'whence' argument"; $node } ... = $sentinel_node->();
35. ADJUST for a Cursor Cursors start with an object, extract its node. Anything applying NodeMgr will work. elsif( $self->isa( $curs_c ) ) { blessed $whence or croak "Botched $name: '$whence' is not an object"; $whence->DOES( $nmgr_r ) or croak "Botched $name: '$whence' does not $nmgr_r"; $node } = $whence->node;
36. ADJUST for a Fragment Might have a node, or be empty. Source node spliced, leaving them an empty sentinel. A Fragment has no head, just an iterate-able body. Notice that nodes are not objects, they are array refs.
37. if( $whence ) { state $sent_type = reftype $sentinel_node->(); blessed $whence and croak "Botched $name: '$whence' is not a bare node."; reftype $whence eq $sent_type or croak "Botched $name: '$whence' is not '$sent_type'"; $node = [ splice $whence->@* ]; } else { $node } = $sentinel_node->();
38. ADJUST for a Fragment Why $node is a single value. Not separated into $next and @data: $node = [ splice $whence->@* ]; Splice performs a Perly atomic copy. The source is empty, the destination is a new node. This transfers a sub-list in one operation.
39. Using a fragment Splice the middle of one list into a new one: my $sublist = $list1->first( $sublist_start )->fragment; my $after = $prior ->first( $sublist_end ->fragment ->push_onto( $list ); ) # $sublist can be inserted into a new list or consumed. $sublist->insert_at( $list2->first( $foo ) ); # or $sublist->consume( $dispatch_jobs );
40. lazy_generate populates lists Example from ‘big_lists’ benchmarks. my $i = 1_000_001; my $biglist = LinkedList::Single::List ->new ->lazy_generate ( sub { --$i or die “\n” } );
41. lazy_generate populates Read from disk files or load DBI results. my $data_list = LinkedList::Single::List ->new ->lazy_generate ( sub { $sth->selectrow_arrayef or die “\n”} );
42. Q: What’s with the die?? This is also used in map, grep, & friends. The exception signals end-of-processing. try { frobnicate->( $self->data ); $self->advance or last; } catch( $err ) { $err eq "\n" or die $err; last }
43. Q: What’s with the die?? Nodes can contain anything. Or nothing. Or undef. There is no return value to signal ‘end of list’. die “…\n” doesn’t append any line information. die “\n” returns an empty exception.
44. Q: What’s with the die?? Useful to leave a cursor within the list. Allows daisy-chaining lookups.
45. Q: What’s with the die?? The exception can percolate from anywhere. The closure can have a 40-level call stack. The empty exception simplifies all of it.
46. Using die “\n” Searching a sorted list for the first item < X. You find an item on the list > X. There will be nothing to find: It’s all > X. die “\n” avoids a wasted scan.
47. Summary The new linked list handler is cleaner, faster. It works with Object::Pad to be more maintainable. O::P makes adding features simpler.
48. Dealing with multiple Perl versions I don’t want to use Perl v5.8 forever. Fix: Release version-dependent modules. version/v5.24: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:16 lib drwxrwxr-x 2 lembark lembark 238 Mar 21 17:16 t version/v5.40: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:14 lib drwxrwxr-x 13 lembark lembark 196 Apr 11 20:33 t
49. Dealing with multiple Perl versions Compare $^V to the directory. Copy that make lib & t for installation. version/v5.24: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:16 lib drwxrwxr-x 2 lembark lembark 238 Mar 21 17:16 t version/v5.40: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:14 lib drwxrwxr-x 13 lembark lembark 196 Apr 11 20:33 t
50. Dealing with multiple Perl versions Compare $^V to the directory. Copy that make lib & t for installation. version/v5.24: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:16 lib drwxrwxr-x 2 lembark lembark 238 Mar 21 17:16 t version/v5.40: total 0 drwxrwxr-x 3 lembark lembark 24 Mar 21 17:14 lib drwxrwxr-x 13 lembark lembark 196 Apr 11 20:33 t
51. Class definitions are lightweight Mostly just applying roles. Few utility subs.
52. Boolean context is helpful if( $list ) { it has nodes } else { it lacks nodes } if( $cursor ) { can advance } else { at sentinel } if( $fragment ) { has nodes } else { at sentinel } use overload q/bool/ => sub { $_[0]->is_sentinel } ;
53. Roles are straightforward List apply LinkedList::Single::NodeMgr; apply LinkedList::Single::Header; Cursor apply LinkedList::Single::NodeMgr; apply LinkedList::Single::Iterator; Fragment apply LinkedList::Single::NodeMgr; apply LinkedList::Single::Iterator; apply LinkedList::Single::Transient;
54. Roles are straightforward The NodeMgr manages the internals. $thingy->DOES( ‘LinkedList::Single::NodeMgr’ ); Point is being able to modify the strucutre. e.g., replace $node with @node. Only your hairdresser knows for sure!