Cool CPAN Modules

Cool CPAN Modules

The importance of CPAN and some of my favorite modules, including: MS Excel integration, terminal, files, command line arguments, and many others

82fe3950bdbb0b1048e870fa04c89cfd?s=128

ynonperek

July 10, 2012
Tweet

Transcript

  1. Perl and CPAN Ynon Perek Tuesday, July 10, 12

  2. Agenda • Perl 5 is Alive • Review: References &

    Modules • Our First CPAN Installation • Automatic CPAN tools • Office Integration • Other Cool CPAN Modules Tuesday, July 10, 12
  3. The Future • Perl5 is very alive • perl 5.16

    was out in May (2012) • Release cycle: New version every year • Where do they get new ideas ? Tuesday, July 10, 12
  4. CPAN • Over 25,000 free perl software bundles • Solving

    problems so you don’t have to Tuesday, July 10, 12
  5. The Future • Future of perl is in CPAN Tuesday,

    July 10, 12
  6. The Future • Future of perl is in CPAN •

    CPAN easier to change (than CORE) Tuesday, July 10, 12
  7. The Future • Future of perl is in CPAN •

    It’s easier to change (than CORE) • CPAN is Easier to maintain (than CORE) Tuesday, July 10, 12
  8. CPAN • Not just new methods, some modules change the

    syntax of perl itself Tuesday, July 10, 12
  9. ACME::Pythonic for my $n in 1..100: while $n != 1:

    if $n % 2: $n = 3*$n + 1 else: $n /= 2 Tuesday, July 10, 12
  10. ACME::Morse use Acme::Morse; .--.-..--..---.-.--..--.-..--..---.-.--. .-.-........---..-..---.-..-.--..---.--. ..-.---......-...-...-..--..-.-.-.--.-.. ----..-.-.--.-..--..-.-...---.-..---.--. .-...-..--.---...-.-.... Tuesday, July

    10, 12
  11. MooseX::Declare class BankAccount { has 'balance' => ( isa =>

    'Num', is => 'rw', default => 0 ); method deposit (Num $amount) { $self->balance( $self->balance + $amount ); } method withdraw (Num $amount) { my $current_balance = $self->balance(); ( $current_balance >= $amount ) || confess "Account overdrawn"; $self->balance( $current_balance - $amount ); } } Tuesday, July 10, 12
  12. Importance of CPAN • CORE is getting smaller • World

    is getting more complicated • Thus, New perl is born in CPAN Tuesday, July 10, 12
  13. mapofcpan.org Tuesday, July 10, 12

  14. CPAN Resources • metacpan.org • cpan-explorer.org Tuesday, July 10, 12

  15. Review References & Modules Tuesday, July 10, 12

  16. References • Perl has 3 main data types: $, @,

    % • Sometimes that’s not enough • What if you wanted to pass 2 lists to a subroutine ? Tuesday, July 10, 12
  17. Will This Work ? #!/usr/bin/perl use strict; use warnings; sub

    longer { my (@l1, @l2) = @_; return @l1 > @l2 ? @l1 : @l2; } Tuesday, July 10, 12
  18. The Problem 1 2 9 8 1 2 9 8

    @l1 @l2 @_ • Perl is linear • All data passed to a sub goes through @_ • No way to know when @l1 ended Tuesday, July 10, 12
  19. Refs To The Rescue • A ref is a scalar

    that can be used to access other data • Scalars are much easier to handle 1 2 @l $l_ref Tuesday, July 10, 12
  20. Creating Refs • Use \ operator to create a reference

    to any perl data type • Note & means subroutine # create a list ref with \@ my $l_ref = \@l; # create a hash ref with \% my $h_ref = \%h; # create a scalar ref with \$ my $x_ref = \$x; # create a subroutine ref with \& my $f_ref = \&foo; Tuesday, July 10, 12
  21. Dereferencing • Use the original data type before a ref

    to get the original value • This is called “dereference” # note that assignment operator # generates a copy operation my @copy = @$l_ref; my %copy = %$h_ref; my $copy = $$x_ref; my $res = &$f_ref; Tuesday, July 10, 12
  22. Direct Element Deref • Direct elements are accessed via ->

    operator • Note the different types of parens # Direct element access using # references my $first = $l_ref->[0]; my $val = $h_ref->{key}; my $result = $f_ref->('x', 'y'); Tuesday, July 10, 12
  23. Review • What is the type of “x” in each

    line ? • $x->[4] • $x->{4} • $x[4] • $$x[4] Tuesday, July 10, 12
  24. Review • What is printed ? use strict; use warnings;

    my @l = (2, 3, 5); my $l_ref = \@l; push @l, 10; print "@$l_ref\n"; Tuesday, July 10, 12
  25. Lab • Write a function that takes two lists and

    returns sum(@first) - sum(@second). Use references to pass the data • Write a function that takes a hash ref, key and value and inserts the new (key, value) to the hash Tuesday, July 10, 12
  26. Q & A Tuesday, July 10, 12

  27. Perl Modules Tuesday, July 10, 12

  28. Writing Modules Photo Editing Program Tuesday, July 10, 12

  29. Writing Modules Photo Editing Program XML Parser http://www.everystockphoto.com/photo.php?imageId=4319409 http://www.flickr.com/photos/psd/1005276581/ http://morguefile.com/archive/display/611435

    Tuesday, July 10, 12
  30. Writing Modules Photo Editing Program XML Parser Pet Encyclopedia http://www.everystockphoto.com/photo.php?imageId=4319409

    http://www.flickr.com/photos/psd/1005276581/ http://morguefile.com/archive/display/611435 Tuesday, July 10, 12
  31. Common Functionality Photo Editing Program XML Parser Pet Encyclopedia http://www.everystockphoto.com/photo.php?imageId=4319409

    http://www.flickr.com/photos/psd/1005276581/ http://morguefile.com/archive/display/611435 Tuesday, July 10, 12
  32. Modules Benefits • Write Common functionality once, use for all

    applications • Even better - skip the writing part and get it from CPAN Tuesday, July 10, 12
  33. Module Rules • Name the file CamelCased.pm • Last statement

    evaluated true • Put it somewhere perl can find it use strict; use warnings; print "Hello Modules\n"; 1; Hello.pm Tuesday, July 10, 12
  34. Using Modules #!/usr/bin/perl use strict; use warnings; use Hello; Tuesday,

    July 10, 12
  35. Playing Friendly • Modules usually define new subroutines • Consider:

    How will your module play in a team ? Tuesday, July 10, 12
  36. Plants Vs. Aliens sub secret_weapon { print("Aliens win\n") } sub

    hit_plants { secret_weapon(); } 1; sub secret_weapon { print("Plants Win\n") } sub hit_aliens { secret_weapon(); } 1; use Plants; use Aliens; print "Hitting the aliens\n"; hit_aliens(); Aliens.pm Plants.pm main.pl Tuesday, July 10, 12
  37. Namespaces • Keeping all names of all subroutines inside modules

    unique is very hard • Solution: Namespaces Tuesday, July 10, 12
  38. Namespaces Example package Aliens; use strict; sub secret_weapon { print("Aliens

    win\n") } sub hit_plants { secret_weapon(); } 1; use strict; use warnings; use Aliens; Aliens::hit_plants(); Tuesday, July 10, 12
  39. Namespaces Example package Aliens; use strict; sub secret_weapon { print("Aliens

    win\n") } sub hit_plants { secret_weapon(); } 1; use strict; use warnings; use Aliens; Aliens::hit_plants(); Tuesday, July 10, 12
  40. Q & A Tuesday, July 10, 12

  41. Lab • Write a module providing two subroutines: • read_file(

    $filename ) - returns a single scalar with all contents of the file • write_file( $filename, $content ) - takes a scalar and write it to a file Tuesday, July 10, 12
  42. Module Location • Perl searches for modules in the following

    locations: • Prefix paths decided at compilation time • PERL5LIB environment variable • Current directory Tuesday, July 10, 12
  43. Add New Modules • Create your own modules path in

    a friendly location • Add that location to PERL5LIB • Bash: export PERL5LIB=/my/location • Csh: setenv PERL5LIB /my/location Tuesday, July 10, 12
  44. Demo: Modules • Download File::Slurp from CPAN • Extract with:

    tar xvfz filename.tar.gz • Copy lib/* to your modules path • Use the module ! Tuesday, July 10, 12
  45. CPAN Toolchain • Automated tools to download and install modules

    for you • Handle dependencies • Handle compilation of C code Tuesday, July 10, 12
  46. Using The Toolchain • install local::lib • install App::cpanminus •

    Enjoy Tuesday, July 10, 12
  47. Install local::lib •wget http://search.cpan.org/CPAN/ authors/id/A/AP/APEIRON/local- lib-1.008004.tar.gz •tar xvfz local-lib-1.008004.tar.gz •cd

    local-lib-1.008004 •perl Makefile.pl --bootstrap •make •make install Tuesday, July 10, 12
  48. Install local::lib • Add local::lib to your .cshrc file: perl

    -I$HOME/perl5/lib/perl5 -Mlocal::lib >> ~/.cshrc • Start a new shell Tuesday, July 10, 12
  49. Install App::cpanminus • wget http://search.cpan.org/CPAN/ authors/id/M/MI/MIYAGAWA/App- cpanminus-1.5015.tar.gz • tar xvfz

    App-cpanminus-1.5015.tar.gz • cd App-cpanminus-1.5015 • perl Makefile.pl • make • make install Tuesday, July 10, 12
  50. Install New Modules • cpanm File::Slurp • cpanm Acme::Morse •

    cpanm Acme::Pythonic Tuesday, July 10, 12
  51. Q & A Tuesday, July 10, 12

  52. CPAN Modules #1 • Data::Dumper, Data::Printer • Spreadsheet::ParseExcel, Spreadsheet::WriteExcel •

    Try::Tiny, autodie, Carp::REPL, English Tuesday, July 10, 12
  53. Printing Data • Use (core) Data::Dumper to get an ugly

    printout of the data • Use Data::Printer to get a pretty printout of the data Tuesday, July 10, 12
  54. Data::Dumper • Dumper( $ref ) returns a string representation of

    a complex data structure • Don’t forget to print it use v5.14; use Data::Dumper; my @letters = ('a'..'f'); my @numbers = (0..7); my %contacts = ( Mike => 7, Jim => 5 ); my @data = (\@letters, \@numbers, \%contacts); print Dumper(\@data); Tuesday, July 10, 12
  55. Data::Printer • New function: p • Don’t have to pass

    a reference • Print in colors use v5.14; use Data::Printer; my @letters = ('a'..'f'); my @numbers = (0..7); my %contacts = ( Mike => 7, Jim => 5 ); my @data = (\@letters, \@numbers, \%contacts); p @data; Tuesday, July 10, 12
  56. Excel Modules Tuesday, July 10, 12

  57. Spreadsheet::ParseExcel • Reads an excel file (95-2003) • Access all

    information in that file • Help: https://groups.google.com/forum/? fromgroups#!forum/spreadsheet-parseexcel Tuesday, July 10, 12
  58. Spreadsheet::ParseExcel • Get a parser • Parse excel file •

    Each workbook “has” an array of work sheets use Spreadsheet::ParseExcel; use strict; use warnings; use Data::Printer; my $parser = Spreadsheet::ParseExcel->new; my $workbook = $parser->parse('demo1.xls') or die $parser->error; p $workbook; Tuesday, July 10, 12
  59. Spreadsheet::ParseExcel • Print out contents of the first work sheet

    my $ws = $workbook->worksheet(0); my ( $row_min, $row_max ) = $ws->row_range; my ( $col_min, $col_max ) = $ws->col_range; for my $row ( $row_min .. $row_max ) { for my $col ( $col_min .. $col_max ) { my $cell = $ws->get_cell( $row, $col ); say "($row, $col) = ", $cell->value; } } Tuesday, July 10, 12
  60. Spreadsheet::ParseExcel • Spreadsheet::ParseExcel::Workbook is our entry point to the document

    • Use ->worksheet( $idx ) to get a specific worksheet • Use ->worksheets() to get them all, i.e. for looping Tuesday, July 10, 12
  61. Spreadsheet::ParseExcel • Worksheet objects represents individual sheets • Main methods:

    • ->get_cell($i, $j) • ->row_range(), col_range() • ->get_name Tuesday, July 10, 12
  62. Spreadsheet::ParseExcel • Cell represents a cell • Methods: • ->value(),

    unformatted(): Same for non- numeric fields • ->get_format Tuesday, July 10, 12
  63. Lab • Take an excel sheet with 5 columns, each

    containing a different row of numbers • Print out the column number whose sum is largest Tuesday, July 10, 12
  64. Spreadsheet::WriteExcel • Same classes, opposite direction • Create workbook •

    Add worksheets • Write to cells Tuesday, July 10, 12
  65. Spreadsheet::WriteExcel • Same classes, opposite direction • Create workbook •

    Add worksheets • Write to cells Tuesday, July 10, 12
  66. Spreadsheet::WriteExcel use v5.14; use Spreadsheet::WriteExcel; # Create the work book

    my $workbook = Spreadsheet::WriteExcel->new('writer1.xls'); my $ws = $workbook->add_worksheet(); $ws->write('A1', 'Hello World'); Tuesday, July 10, 12
  67. Spreadsheet::WriteExcel • Use ->write( $row, $col, $data ) for writing

    in a loop • Use ->write( $row, $col, $data, $format ) for writing formatted data • Note $row and $col start from 0 • Examples Tuesday, July 10, 12
  68. Spreadsheet::WriteExcel use v5.14; use Spreadsheet::WriteExcel; # Create the work book

    my $workbook = Spreadsheet::WriteExcel->new('writer1.xls'); my $ws = $workbook->add_worksheet(); for my $row ( 1..10 ) { for my $col ( 1..10 ) { $ws->write( $row - 1, $col - 1, $row * $col ); } } Tuesday, July 10, 12
  69. Spreadsheet::WriteExcel use v5.14; use Spreadsheet::WriteExcel; # Create the work book

    my $workbook = Spreadsheet::WriteExcel->new('writer1.xls'); my $ws = $workbook->add_worksheet(); my @data = qw/red blue green white cyan magenta black/; while ( my ($idx, $val) = each @data ) { my $fmt = $workbook->add_format; $fmt->set_color( $val ); $ws->write( $idx, 0, $val, $fmt ); } Tuesday, July 10, 12
  70. Spreadsheet::WriteExcel • Use ->write_date_time( $row, $col, $date_string, $format ) for

    writing dates • $format specifies output format (such as ‘dd/ mm/yyyy’) • get $date_string by stringifying a DateTime object Tuesday, July 10, 12
  71. Spreadsheet::WriteExcel use v5.14; use Spreadsheet::WriteExcel; use DateTime; my $wb =

    Spreadsheet::WriteExcel->new('book.xls'); my $ws = $wb->add_worksheet(); my $date = DateTime->now; my $fmt1 = $wb->add_format( num_format => 'dd/mm/yyyy' ); my $fmt2 = $wb->add_format( num_format => 'dd mmm yy'); $ws->write_date_time(0, 0, $date, $fmt1); $ws->write_date_time(1, 0, $date, $fmt2); Tuesday, July 10, 12
  72. Spreadsheet::WriteExcel • We can use a $workbook to set properties

    for the file • Available properties: title, subject, author, manager, company, category, keywords, comments Tuesday, July 10, 12
  73. Spreadsheet::WriteExcel use strict; use Spreadsheet::WriteExcel; my $workbook = Spreadsheet::WriteExcel->new('properties.xls'); $workbook->set_properties(

    title => 'This is an example spreadsheet', subject => 'With document properties', author => 'John McNamara', comments => 'Created with Perl and Spreadsheet::WriteExcel', ); Tuesday, July 10, 12
  74. Q & A Tuesday, July 10, 12

  75. Less Than Excel Tuesday, July 10, 12

  76. Data::Report • Sometimes you just need a simple textual report

    at the end of your script • Excel is an overkill • Use Data::Report Tuesday, July 10, 12
  77. Data::Report • Report Example: Acct Description Debet Credit -------------------------------------------------- 1234

    Received 242.33 5678 Paid 699.45 1259 Taxes 12.00 244.00 TOTAL 254.33 943.45 Tuesday, July 10, 12
  78. use Data::Report; # Create a new reporter. my $rep =

    Data::Report::->create(type => "text"); # or "html", or "csv", ... # Define the layout. $rep->set_layout ([ { name => "acct", title => "Acct", width => 6 }, { name => "desc", title => "Description", width => 40, align => "<" }, { name => "deb", title => "Debet", width => 10, align => ">" }, { name => "crd", title => "Credit", width => 10, align => ">" }, ]); # Start the reporter. $rep->start; # Add data, row by row. $rep->add({ acct => 1234, desc => "Received", deb => "242.33" }); $rep->add({ acct => 5678, desc => "Paid", crd => "699.45" }); $rep->add({ acct => 1259, desc => "Taxes", deb => "12.00", crd => "244.00" }); $rep->add({ desc => "TOTAL", deb => "254.33", crd => "943.45" }); # Finish the reporter. $rep->finish; Tuesday, July 10, 12
  79. Lab • Write a perl script that reads all files

    in current directory and creates an excel sheet including: file name, file size, modification date • Use red color for files larger than 2K • Create a textual report of the data Tuesday, July 10, 12
  80. English • Easier global variable names •perldoc perlvar to get

    the full list use English qw/-no_match_vars/; my $filename = $PROGRAM_NAME; open my $fh, "<", $filename or die $OS_ERROR; while (<$fh>) { print; } close $fh; Tuesday, July 10, 12
  81. Try::Tiny • Catch exceptions (thrown with ‘die’) • Exception text

    stored in $_ • Don’t forget to end with semicolon use strict; use warnings; use Try::Tiny; try { die "good bye cruel world"; } # catch is optional catch { warn "Exception: $_"; }; Tuesday, July 10, 12
  82. autodie • Does what it says on the tin •

    No more open or die use v5.14; use autodie; # DIE open my $f, '<', 'invalid.txt'; Tuesday, July 10, 12
  83. Carp::REPL • Start a REPL loop instead of dying •

    access all your variables and run code to understand what went wrong •perl -MCarp::REPL script.pl • Demo Tuesday, July 10, 12
  84. Lab • Find out what does File::ReadBackwards do • Write

    a program that reads a file and prints out the last line which is longer than 15 characters Tuesday, July 10, 12
  85. Shell And File System Tuesday, July 10, 12

  86. Modules #2 • Getopt::Long, Term::ANSIColor • File::Find::Declare, File::Which, File::Copy •

    Capture::Tiny • Term::ProgressBar::Simple, IO::Interactive Tuesday, July 10, 12
  87. Getopt::Long • Parse command line options GNU style • Options

    are passed prefixed with -- • Examples: grep --count cut --delimeter=: --fields=1,2,3 ls --all --color Tuesday, July 10, 12
  88. Getopt::Long • Load the module with use Getopt::Long • Call

    GetOptions(...) to parse Tuesday, July 10, 12
  89. Getopt::Long • Three types of command line options: • Do

    not take parameters (ls --all) • Take parameter (ls --ignore=PATTERN) • Could take parameter (ls --color) Tuesday, July 10, 12
  90. Bare Options • Pass in ‘name’ => \$variable to get

    a boolean value • Pass in ‘name+’ => \$variable to get a numeric (count) value Tuesday, July 10, 12
  91. Bare Options use v5.14; use Getopt::Long; my ( $verbose, $debug,

    $x ); GetOptions( 'verbose' => \$verbose, 'debug' => \$debug, 'x+' => \$x); say "v=$verbose, d=$debug, x=$x"; perl go2.pl --verbose --debug -x -x -x Tuesday, July 10, 12
  92. Bare Options • $verbose is true • $debug is undef

    • $x is 3 use v5.14; use Getopt::Long; my ( $verbose, $debug, $x ); GetOptions( 'verbose' => \$verbose, 'debug' => \$debug, 'x+' => \$x); say "v=$verbose, d=$debug, x=$x"; perl go2.pl --verbose --debug -x -x -x Tuesday, July 10, 12
  93. Parameterized Opts • Pass in ‘delimeter=s’ => \$delim to get

    a string value • Pass in ‘library=s’ => \@libs to get multiple values • Pass in ‘define=s’ => \%defines to get key/value pairs Tuesday, July 10, 12
  94. Parameterized Opts use v5.14; use Getopt::Long; use Data::Printer; my (@libs,

    $name, %defines); GetOptions('lib=s' => \@libs, 'name=s' => \$name, 'def=s' => \%defines); perl go3.pl --lib foo --lib bar --def x=7 --def y=9 --name=me Tuesday, July 10, 12
  95. Parameterized Opts • @libs = (‘foo’, ‘bar’) • $name =

    ‘me’ • %defines = ( x => 7, y => 9 ) use v5.14; use Getopt::Long; use Data::Printer; my (@libs, $name, %defines); GetOptions('lib=s' => \@libs, 'name=s' => \$name, 'def=s' => \%defines); perl go3.pl --lib foo --lib bar --def x=7 --def y=9 --name=me Tuesday, July 10, 12
  96. Getopt::Long • Can use | for aliases: ‘all|a’ => \$all

    • Can use a single hash for options • Lots of other configuration options perldoc Getopt::Long Tuesday, July 10, 12
  97. Q & A Tuesday, July 10, 12

  98. Term::ANSIColor • Colors are fun ! (and easy) • use

    print color to change terminal text color • Don’t forget to print color ‘reset’ when you’re done Tuesday, July 10, 12
  99. Term::ANSIColor use strict; use warnings; use Term::ANSIColor; my @colors =

    qw/red blue yellow magenta cyan/; foreach my $clr (@colors) { print color $clr; print "Im $clr\n"; print color 'reset'; } Tuesday, July 10, 12
  100. Lab • Write a perl that prints a file’s contents

    in colors • Take a color as an optional input switch • Take file name as argument • Implement an Easter Egg for a full rainbow Tuesday, July 10, 12
  101. Capture::Tiny • Run code and get its output (yeah, like

    backticks but better) • Don’t care if code is perl or external • Can also tee output Tuesday, July 10, 12
  102. Capture::Tiny • Use capture to run code and capture its

    output/ errors • Use tee to run code normally AND get its output/errors use v5.14; use Capture::Tiny qw/:all/; sub do_stuff { say "Hello World"; warn "3... 2.... 1...."; return 7; } # Put "Hello WOrld" in $stdout # "3... 2... 1..." in $stderr # 7 in @result my ($stdout, $stderr, @result) = capture { do_stuff; }; # Same, but also print output my ($t_stdout, $t_stderr, @t_result) = tee { do_stuff; }; Tuesday, July 10, 12
  103. File::Which • Find an executable file in one of the

    known exe paths • Works on all platforms use v5.14; use File::Which; my $perldoc = which('perldoc'); say "perldoc is at: $perldoc"; Tuesday, July 10, 12
  104. File::Copy • No surprises here • Way cleaner than rolling

    your own • Provides: copy, move • Return 1 on success, 0 on failure use v5.14; use File::Copy; my $program_name = $0; copy($program_name, "copy1.pl"); copy($program_name, "copy2.pl"); move("copy2.pl", "copy3.pl"); Tuesday, July 10, 12
  105. File::Find::Declare • Easy interface to File::Find • Specify attributes of

    files you want, and get a list of files matching the attributes use v5.14; use Data::Printer; use File::Find::Declare; my $finder = File::Find::Declare->new({ dirs => '/home', ext => '.mp3', recurse => 1, }); my @music = $finder->find; p @music; Tuesday, July 10, 12
  106. File::Find::Declare • Some Available Options: like, unlike, size, modified, owner,

    perms Tuesday, July 10, 12
  107. Q & A Tuesday, July 10, 12

  108. Lab • Write a program that asks the user for

    a file size and search for all files larger than that size • Use IO::Interactive to exit immediately if called as non-interactive • Use Term::ProgressBar::Simple to draw a progress bar Tuesday, July 10, 12
  109. Lab • Given an excel file with three columns: source_name,

    dest_name, reverse • Write a perl that reads the file, if reverse is 0 simply copy from source_name to dest_name • If reverse is 1 copy in reverse order of lines Tuesday, July 10, 12