The $path to Knowlege: What little it takes to test perl.

Testing code has been a core feature of Perl since the beginning. Perl's flexibility in testing makes bulk testing large amounts of code straightforward using the UNIX filesystem and some simple metadata-driven testing techniques. This talk describes a framework I have used for testing in a variety of situations, including validating the module, @EXPORT contents, saving myself from typing "perl -wc" 70,000 times, and letting Jenkins do the work for me.

Transcript

1. The $path to Knowlege:
   What little it takes to test perl.
   Steven Lembark
   Workhorse Computing
   lembark@wrkhors.com
   

   View Slide

2. Everyone wants to leave a legacy.
   Makes sense: Leave your mark on history.
   

   View Slide

3. Everyone wants to leave a legacy.
   Makes sense: Leave your mark on history.
   What if your mark is history?
   It never changes.
   It never adapts.
   Forever frozen in the wastelands of 5.8.
   

   View Slide

4. Growing beyond legacy.
   Any number of companies have legacy Perl code.
   Any number of reasons it isn’t upgraded.
   Then you decide to upgrade, get current.
   Now what?
   

   View Slide

5. First step: Triage
   Obvious questions:
   What’s working?
   What’s broken?
   What’s next?
   

   View Slide

6. First step: Triage
   Obvious questions:
   What’s working?
   What’s broken?
   What’s next?
   A: Let’s find out.
   

   View Slide

7. Example
   I’ve worked for multiple clients with v5.8 code.
   Then they wanted to upgrade.
   Say to 5.2X or later.
   Undo a few decades of technical debt.
   Only takes a few weeks, right?
   

   View Slide

8. So what? You test the code?
   Next step is testing lots of files.
   Repeating the same tests, over and over.
   Rule 1: Avoid Red Flags.
   Don’t copy, cut, or paste.
   Use Lazyness.
   

   View Slide

9. Perl Somalier
   
   The Repair Shop has aged well:
   Nice bouquet, pleasant aftertaste.
   All the things that make cleaning up code pleasing.
   

   View Slide

10. Step 1: use perl.
    #!/usr/bin/env perl
    not /usr/bin/perl.
    Then check your path.
    

    View Slide

11. Step 2: Find the perly files
    find lib -type f -name ‘*pm’;
    Skip shell programs:
    find bin scripts utils -type f |
    xargs file |
    grep ‘Perl’ |
    cut -d’:’ -f1 ;
    

    View Slide

12. OK: unit test the files
    OK, so you copy a red flag... er... template for each...
    Define a datafile?
    Write the YAML file from hell?
    Create JSON from worse?
    Nope.
    

    View Slide

13. Start with a path.
    Q: What do we need in order to unit test one module?
    A: Its path.
    /my/sandbox/XYX/lib/Foo/Bar.pm
    

    View Slide

14. Start with a path.
    Q: What do we need in order to unit test one module?
    A: Its path.
    /my/sandbox/XYX/lib/Foo/Bar.pm
    Encoded in a symlink:
    ./t/01-pm/~my~sandbox~XYZ~lib~Foo~Bar.pm.t
    

    View Slide

15. Start with a path.
    Q: What do we need in order to unit test one module?
    A: Its path.
    /my/sandbox/XYX/lib/Foo/Bar.pm
    Along with its package:
    ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t
    

    View Slide

16. Start with a path.
    Q: What do we need in order to unit test one module?
    A: Its path.
    /my/sandbox/XYX/lib/Foo/Bar.pm
    Along with its package:
    ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t
    ~Foo~Bar
    

    View Slide

17. Start with a path.
    Q: What do we need in order to unit test one module?
    A: Its path.
    /my/sandbox/XYX/lib/Foo/Bar.pm
    Along with its package:
    ./t/01-pm/~my~sandbox~XYZ~lib~~Foo~Bar.pm.t
    Foo::Bar
    

    View Slide

18. Start with a path.
    use Test::More;
    use File::Basename;
    my $base0 = basename $0;
    my $sep = substr $base0, 0, 1;
    my $path = join '/' => split m{[$sep]}, $base0;
    my ($pkg) = $path =~ m{// (.+?) [.]pm$}x;
    my $pkg =~ s{\W}{::}g;
    require_ok $path
    or skip "Failed require", 1;
    can_ok $pkg, 'VERSION'
    or skip "Missing packge: '$pkg' ($path)";
    

    View Slide

19. Given a path and a packge...
    What else would you want to test?
    How about exports?
    

    View Slide

20. Validating @EXPORT & @EXPORT_OK
    Basic problem: Exporting what isn’t.
    Undefined values in @EXPORT or @EXPORT_OK.
    Botched names.
    

    View Slide

21. Validating @EXPORT & @EXPORT_OK
    Both can easily be checked.
    Symbol is your friend.
    qualify_to_ref is your buddy.
    

    View Slide

22. Validating @EXPORT & @EXPORT_OK
    Starting with $path and $pkg:
    Require the path.
    Check for @EXPORT, @EXPORT_OK.
    Walk down whichever is defined.
    Check that the contents are defined.
    

    View Slide

23. Validating @EXPORT & @EXPORT_OK
    # basic sanity checks: configured for exporting.
    require_ok $path or skip “oops...”, 1;
    isa_ok $pkg, ‘Exporter’ or skip “$pkg is not Exporter”;
    can_ok $path -> 'import'
    or do
    {
    diag “Botched $pkg: Exporter lacks ‘import’”;
    skip “$pkg cannot ‘import’”
    };
    # hold off calling import until we have some values.
    # maybe a diag for can_ok
    

    View Slide

24. Validating @EXPORT & @EXPORT_OK
    # second step: check the contents of EXPORT & _OK
    # require_ok doesn’t call import: need it for @EXPORT.
    for my $exp ( qw( EXPORT EXPORT_OK ) )
    {
    my $ref = qualify_to_ref $exp => $pkg;
    my $found = *{$ref}{ARRAY} or next;
    note "Validate: $pkg $exp\n", explain $found;
    my @namz = @$found
    or skip "$pkg has empty '$exp'";
    $pkg->import( @namz );
    

    View Slide

25. Validating @EXPORT & @EXPORT_OK
    for my $name ( @namz )
    {
    my $sigil
    = $name =~ m{^\w} ? '&' : substr $name, 0, 1, '' ;
    if( ‘&’ eq $sigil )
    {
    # anything exported should exist in both places.
    can_ok $pkg, $name;
    can_ok __PACKAGE__, $name;
    }
    else
    ...
    

    View Slide

26. Validating @EXPORT & @EXPORT_OK
    ...
    {
    state $sig2type = [qw( @ ARRAY % HASH $ SCLALAR ...) ];
    my $type = $sigil2type{ $sigil };
    my $src = qualify_to_ref $name, $pkg;
    my $dst = qualify_to_ref $name, __PACKGE__;
    my $src_v = *{ $ref }{ $type }
    or do { diag “$pkg lacks ‘$name’”; next };
    my $dst_v = *{ $dst }{ $type };
    is_deeply $src_v, $dst_v, “$name exported from $pkg”;
    }
    }
    

    View Slide

27. Ever get sick of typing “perl -wc”?
    Lazyness is a virtue: Let perl type it for you.
    All you need is the path, perl, and a version.
    

    View Slide

28. Ever get sick of typing “perl -wc”?
    chomp ( my $perl = qx{ which perl } );
    my $run_d = dirname $0;
    my $path = ( basename $0, '.t' ) =~ tr{~}{/}r;
    my $base = basename $path;
    SKIP:
    {
    -e $perl or skip "Non-existant: 'perl'";
    -x $perl or skip "Non-executable: '$perl'";
    -e $path or skip "Non-existant: '$path'";
    -r $path or skip "Non-readable: '$path'";
    -s $path or skip "Zero-sized: '$path'";
    # at this point the test is run-able
    

    View Slide

29. Ever get sick of typing “perl -wk”?
    chomp ( my $perl = qx{ which perl } );
    # $^V isn’t perfect, but it’s a start.
    my $input = "(echo ‘use $^V’; cat $path)";
    my $cmd = "$input | perl -wc -";
    my $out = qx{ $cmd 2>&1 };
    my $exit = $?;
    ok 0 == $exit, "Compile: '$base'";
    $out eq "- Syntax OK\n"
    or
    diag "\nPerl diagnostic: '$path'\n$out\n";
    }
    

    View Slide

30. Ever get sick of typing “perl -wk”?
    chomp ( my $perl = qx{ which perl } );
    # $^V isn’t perfect, but it’s a start.
    my $input = "(echo ‘use $^V’; cat $path)";
    my $cmd = "$input | perl -wc -";
    my $out = qx{ $cmd 2>&1 };
    my $exit = $?;
    ok 0 == $exit, "Compile: '$base'";
    $out eq "- Syntax OK\n"
    or
    diag "\nPerl diagnostic: '$path'\n$out\n";
    }
    

    View Slide

31. Prove is lazier than perl -wc for each file.
    Ouptut is quite paste-able:
    GitLab issue:
    ~~~perl
    
    ~~~
    

    View Slide

32. Cleaning up Exports
    Life begings with Globals.pm
    @EXPORT qw( … );
    ~1500 entries.
    Can’t delete any: Nobody knows what’s used.
    Gotta maintain them all.
    

    View Slide

33. Cleaning up Exports
    Life begings with Globals.pm
    @EXPORT qw( … );
    ~1500 entries.
    Can’t delete any: Nobody knows what’s used.
    Gotta maintain them all.
    Not...
    

    View Slide

34. Cleaning up Exports
    Tests give us @EXPORT* & diagnostics.
    Step 1: @EXPORT_OK
    Yes, this breaks all of the code.
    Result: We know what is missing.
    

    View Slide

35. Cleaning up Exports
    Step 2: Diagnostics list undefined variables.
    Grep them out.
    Generate “use Global qw( … )” lines.
    Updates via perl -i -p.
    Result: We know what is used.
    

    View Slide

36. Cleaning up Exports
    Step 3: Start removing unused exports.
    Look at what we added.
    Comment the rest.
    Test diag’s tell us when we go too far.
    And when to stop.
    

    View Slide

37. Testing with Jenkins
    As always: There’s more than one way.
    One simple fix:
    ./devops/Jenkinsfile
    ./devops/run-tests
    

    View Slide

38. Testing with Jenkins
    stage("Runtime Env")
    {
    steps
    {
    sh "uname -a"
    sh "hostname"
    sh "pwd -L"
    sh "pwd -P"
    sh "set"
    }
    }
    

    View Slide

39. Testing with Jenkins
    stage("Run Tests")
    {
    environment
    {
    path = “${env.WORKSPACE_TMP + ‘/prove’}”
    }
    steps
    {
    sh "git submodule init"
    sh "git submodule update"
    sh "./devops/run-tests"
    }
    }
    

    View Slide

40. Testing with Jenkins
    stage("Run Tests")
    {
    environment
    {
    path = “${env.WORKSPACE_TMP + ‘/prove’}”
    }
    steps
    {
    sh "git submodule init"
    sh "git submodule update"
    sh "./devops/run-tests"
    }
    }
    

    View Slide

41. Testing with Jenkins
    #!/bin/bash -x
    perl -V;
    prove -V;
    echo “Ouput: ‘$path’”;
    cmd=”prove -r --jobs=4 --state=save --statefile=$path t”;
    cd $(dirname $0)/../..;
    ./t/bin/install-test-symlinks;
    $cmd 2>&1 | tee "$path.out";
    

    View Slide

42. How do we know which files to test?
    Simple: Ask
    *.pm files are easy.
    Find #!perl files with
    find lib scripts utils progs apps -type f |
    xargs file |
    grep 'Perl' |
    cut -d':' -f1 |
    xargs ./t/bin/install-symlinks $dir ;
    

    View Slide

43. Bits of Jenins
    Put site_perl into a git submodule.
    Advanced Options for Jenkins: Submodules.
    Allows shallow copy & recurse submodules.
    Maintain as a seprate repo:
    cpanm --local-lib=. --self-contained ... ;
    git add .;
    git commit -m’update CPAN’;
    

    View Slide

44. Bits of Jenins
    environment
    {
    PERL5LIB = "$WORKSPACE/site_perl/lib/perl5"
    TEMPDIR = "$WORKSPACE_TMP"
    }
    Test & Package Perl with Jenkins
    Distribute your own cpan via ./site_perl.
    Nice place for a submodule.
    Catch: How do you find it?
    

    View Slide

45. Bits of Jenins
    environment
    {
    PERL5LIB = "$WORKSPACE/site_perl/lib/perl5"
    TEMPDIR = "$WORKSPACE_TMP"
    }
    Test & Package Perl with Jenkins
    Distribute your own cpan via ./site_perl.
    Nice place for a submodule.
    Catch: How do you find it?
    

    View Slide

46. Bits of Jenins
    environment
    {
    PERL5LIB = "$WORKSPACE/site_perl/lib/perl5"
    TEMPDIR = "$WORKSPACE_TMP"
    }
    Test & Package Perl with Jenkins
    Tee results or store ‘artifacts’.
    

    View Slide

47. Net result: Paths can tell you a lot.
    Easy to acquire.
    Easy to use: basenames & require_ok.
    Explore with Symbol.
    

    View Slide

48. Net result: Paths can tell you a lot.
    Jenkins’ plays nicely with Perl.
    Distribute tests from git.
    Including CPAN modules.
    

    View Slide