The $path to Knowlege: What little it takes to test perl.

Testing code has been a core feature of Perl since the beginning. Perl's flexibility in testing makes bulk testing large amounts of code straightforward using the UNIX filesystem and some simple metadata-driven testing techniques. This talk describes a framework I have used for testing in a variety of situations, including validating the module, @EXPORT contents, saving myself from typing "perl -wc" 70,000 times, and letting Jenkins do the work for me.

Transcript

1. The $path to Knowlege: What little it takes to test perl. Steven Lembark Workhorse Computing lembark@wrkhors.com
2. Everyone wants to leave a legacy. Makes sense: Leave your mark on history.
3. Everyone wants to leave a legacy. Makes sense: Leave your mark on history. What if your mark is history? It never changes. It never adapts. Forever frozen in the wastelands of 5.8.
4. Growing beyond legacy. Any number of companies have legacy Perl code. Any number of reasons it isn’t upgraded. Then you decide to upgrade, get current. Now what?
5. First step: Triage Obvious questions: What’s working? What’s broken? What’s next?
6. First step: Triage Obvious questions: What’s working? What’s broken? What’s next? A: Let’s find out.
7. Example I’ve worked for multiple clients with v5.8 code. Then they wanted to upgrade. Say to 5.2X or later. Undo a few decades of technical debt. Only takes a few weeks, right?
8. So what? You test the code? Next step is testing lots of files. Repeating the same tests, over and over. Rule 1: Avoid Red Flags. Don’t copy, cut, or paste. Use Lazyness.
9. Perl Somalier <https://www.perl.com/pub/2000/04/raceinfo.html/> The Repair Shop has aged well: Nice bouquet, pleasant aftertaste. All the things that make cleaning up code pleasing.
10. Step 1: use perl. #!/usr/bin/env perl not /usr/bin/perl. Then check your path.
11. Step 2: Find the perly files find lib -type f -name ‘*pm’; Skip shell programs: find bin scripts utils -type f | xargs file | grep ‘Perl’ | cut -d’:’ -f1 ;
12. OK: unit test the files OK, so you copy a red flag... er... template for each... Define a datafile? Write the YAML file from hell? Create JSON from worse? Nope.
13. Start with a path. Q: What do we need in order to unit test one module? A: Its path. /my/sandbox/XYX/lib/Foo/Bar.pm
14. Start with a path. Q: What do we need in order to unit test one module? A: Its path. /my/sandbox/XYX/lib/Foo/Bar.pm Encoded in a symlink: ./t/01-pm/~my~sandbox~XYZ~lib~Foo~Bar.pm.t
15. Start with a path. Q: What do we need in order to unit test one module? A: Its path. /my/sandbox/XYX/lib/Foo/Bar.pm Along with its package: ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t
16. Start with a path. Q: What do we need in order to unit test one module? A: Its path. /my/sandbox/XYX/lib/Foo/Bar.pm Along with its package: ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t ~Foo~Bar
17. Start with a path. Q: What do we need in order to unit test one module? A: Its path. /my/sandbox/XYX/lib/Foo/Bar.pm Along with its package: ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t Foo::Bar
18. Start with a path. use Test::More; use File::Basename; my $base0 = basename $0; my $sep = substr $base0, 0, 1; my $path = join '/' => split m{[$sep]}, $base0; my ($pkg) = $path =~ m{// (.+?) [.]pm$}x; my $pkg =~ s{\W}{::}g; require_ok $path or skip "Failed require", 1; can_ok $pkg, 'VERSION' or skip "Missing packge: '$pkg' ($path)";
19. Given a path and a packge... What else would you want to test? How about exports?
20. Validating @EXPORT & @EXPORT_OK Basic problem: Exporting what isn’t. Undefined values in @EXPORT or @EXPORT_OK. Botched names.
21. Validating @EXPORT & @EXPORT_OK Both can easily be checked. Symbol is your friend. qualify_to_ref is your buddy.
22. Validating @EXPORT & @EXPORT_OK Starting with $path and $pkg: Require the path. Check for @EXPORT, @EXPORT_OK. Walk down whichever is defined. Check that the contents are defined.
23. Validating @EXPORT & @EXPORT_OK # basic sanity checks: configured for exporting. require_ok $path or skip “oops...”, 1; isa_ok $pkg, ‘Exporter’ or skip “$pkg is not Exporter”; can_ok $path -> 'import' or do { diag “Botched $pkg: Exporter lacks ‘import’”; skip “$pkg cannot ‘import’” }; # hold off calling import until we have some values. # maybe a diag for can_ok
24. Validating @EXPORT & @EXPORT_OK # second step: check the contents of EXPORT & _OK # require_ok doesn’t call import: need it for @EXPORT. for my $exp ( qw( EXPORT EXPORT_OK ) ) { my $ref = qualify_to_ref $exp => $pkg; my $found = *{$ref}{ARRAY} or next; note "Validate: $pkg $exp\n", explain $found; my @namz = @$found or skip "$pkg has empty '$exp'"; $pkg->import( @namz );
25. Validating @EXPORT & @EXPORT_OK for my $name ( @namz ) { my $sigil = $name =~ m{^\w} ? '&' : substr $name, 0, 1, '' ; if( ‘&’ eq $sigil ) { # anything exported should exist in both places. can_ok $pkg, $name; can_ok __PACKAGE__, $name; } else ...
26. Validating @EXPORT & @EXPORT_OK ... { state $sig2type = [qw( @ ARRAY % HASH $ SCLALAR ...) ]; my $type = $sigil2type{ $sigil }; my $src = qualify_to_ref $name, $pkg; my $dst = qualify_to_ref $name, __PACKGE__; my $src_v = *{ $ref }{ $type } or do { diag “$pkg lacks ‘$name’”; next }; my $dst_v = *{ $dst }{ $type }; is_deeply $src_v, $dst_v, “$name exported from $pkg”; } }
27. Ever get sick of typing “perl -wc”? Lazyness is a virtue: Let perl type it for you. All you need is the path, perl, and a version.
28. Ever get sick of typing “perl -wc”? chomp ( my $perl = qx{ which perl } ); my $run_d = dirname $0; my $path = ( basename $0, '.t' ) =~ tr{~}{/}r; my $base = basename $path; SKIP: { -e $perl or skip "Non-existant: 'perl'"; -x $perl or skip "Non-executable: '$perl'"; -e $path or skip "Non-existant: '$path'"; -r $path or skip "Non-readable: '$path'"; -s $path or skip "Zero-sized: '$path'"; # at this point the test is run-able
29. Ever get sick of typing “perl -wk”? chomp ( my $perl = qx{ which perl } ); # $^V isn’t perfect, but it’s a start. my $input = "(echo ‘use $^V’; cat $path)"; my $cmd = "$input | perl -wc -"; my $out = qx{ $cmd 2>&1 }; my $exit = $?; ok 0 == $exit, "Compile: '$base'"; $out eq "- Syntax OK\n" or diag "\nPerl diagnostic: '$path'\n$out\n"; }
30. Ever get sick of typing “perl -wk”? chomp ( my $perl = qx{ which perl } ); # $^V isn’t perfect, but it’s a start. my $input = "(echo ‘use $^V’; cat $path)"; my $cmd = "$input | perl -wc -"; my $out = qx{ $cmd 2>&1 }; my $exit = $?; ok 0 == $exit, "Compile: '$base'"; $out eq "- Syntax OK\n" or diag "\nPerl diagnostic: '$path'\n$out\n"; }
31. Prove is lazier than perl -wc for each file. Ouptut is quite paste-able: GitLab issue: ~~~perl <paste test output here> ~~~
32. Cleaning up Exports Life begings with Globals.pm @EXPORT qw( … ); ~1500 entries. Can’t delete any: Nobody knows what’s used. Gotta maintain them all.
33. Cleaning up Exports Life begings with Globals.pm @EXPORT qw( … ); ~1500 entries. Can’t delete any: Nobody knows what’s used. Gotta maintain them all. Not...
34. Cleaning up Exports Tests give us @EXPORT* & diagnostics. Step 1: @EXPORT_OK Yes, this breaks all of the code. Result: We know what is missing.
35. Cleaning up Exports Step 2: Diagnostics list undefined variables. Grep them out. Generate “use Global qw( … )” lines. Updates via perl -i -p. Result: We know what is used.
36. Cleaning up Exports Step 3: Start removing unused exports. Look at what we added. Comment the rest. Test diag’s tell us when we go too far. And when to stop.
37. Testing with Jenkins As always: There’s more than one way. One simple fix: ./devops/Jenkinsfile ./devops/run-tests
38. Testing with Jenkins stage("Runtime Env") { steps { sh "uname -a" sh "hostname" sh "pwd -L" sh "pwd -P" sh "set" } }
39. Testing with Jenkins stage("Run Tests") { environment { path = “${env.WORKSPACE_TMP + ‘/prove’}” } steps { sh "git submodule init" sh "git submodule update" sh "./devops/run-tests" } }
40. Testing with Jenkins stage("Run Tests") { environment { path = “${env.WORKSPACE_TMP + ‘/prove’}” } steps { sh "git submodule init" sh "git submodule update" sh "./devops/run-tests" } }
41. Testing with Jenkins #!/bin/bash -x perl -V; prove -V; echo “Ouput: ‘$path’”; cmd=”prove -r --jobs=4 --state=save --statefile=$path t”; cd $(dirname $0)/../..; ./t/bin/install-test-symlinks; $cmd 2>&1 | tee "$path.out";
42. How do we know which files to test? Simple: Ask *.pm files are easy. Find #!perl files with find lib scripts utils progs apps -type f | xargs file | grep 'Perl' | cut -d':' -f1 | xargs ./t/bin/install-symlinks $dir ;
43. Bits of Jenins Put site_perl into a git submodule. Advanced Options for Jenkins: Submodules. Allows shallow copy & recurse submodules. Maintain as a seprate repo: cpanm --local-lib=. --self-contained ... ; git add .; git commit -m’update CPAN’;
44. Bits of Jenins environment { PERL5LIB = "$WORKSPACE/site_perl/lib/perl5" TEMPDIR = "$WORKSPACE_TMP" } Test & Package Perl with Jenkins Distribute your own cpan via ./site_perl. Nice place for a submodule. Catch: How do you find it?
45. Bits of Jenins environment { PERL5LIB = "$WORKSPACE/site_perl/lib/perl5" TEMPDIR = "$WORKSPACE_TMP" } Test & Package Perl with Jenkins Distribute your own cpan via ./site_perl. Nice place for a submodule. Catch: How do you find it?
46. Bits of Jenins environment { PERL5LIB = "$WORKSPACE/site_perl/lib/perl5" TEMPDIR = "$WORKSPACE_TMP" } Test & Package Perl with Jenkins Tee results or store ‘artifacts’.
47. Net result: Paths can tell you a lot. Easy to acquire. Easy to use: basenames & require_ok. Explore with Symbol.
48. Net result: Paths can tell you a lot. Jenkins’ plays nicely with Perl. Distribute tests from git. Including CPAN modules.