How to execute external programs in Perl

2018.06.11 Gotanda.pm Perl Technology Conference #18

Transcript

1. How to execute
   external programs
   in Perl
   Shoichi Kaji
   

   View Slide

2. About Me
   • Shoichi Kaji
   • One of the maintainers of

   Minilla, Perl::Build,
   Text::Xslate, Mouse,
   Test::PAUSE::Permissions
   

   View Slide

3. ͸͡Ίʹ
   • Perl͸betterγΣϧͱݺ͹ΕΔ
   • ΑͬͯPerlͰ֎෦ϓϩάϥϜΛ࣮ߦ͢Δػձ͸ଟ͍
   • ҰํͰ֎෦ϓϩάϥϜ࣮ߦ͸ߟྀ͢΂͖͜ͱ͕

   ͨ͘͞Μ͋Δ
   • ࢥ͍ͷ··ʹ֎෦ϓϩάϥϜΛ࣮ߦͰ͖ΔΑ͏ʹ
   ͳΓ͍ͨ
   

   View Slide

4. Perlͷ֎෦ϓϩάϥϜ࣮ߦؔ࿈ͷ

   τϐοΫε
   • shellΛհ͢΂͔Βͣ ←ࠓ೔͸͜͜ͱ
   • redirect, pipe ←͜͜Λѻ͏
   • γάφϧ
   • Windows
   

   View Slide

5. ໨࣍
   • shellΛհ͢΂͔Βͣ
   • ֎෦ϓϩάϥϜΛ࣮ߦ͢ΔPerlͷؔ਺
   • fork & exec
   • redirect
   • pipe
   • ·ͱΊ
   

   View Slide

6. shellΛհ͢΂͔Βͣ
   • system "find $dir -type f -delete >/dev/null
   2>&1";
   • Ͳ͕͜໰୊ʁ
   • system΍execΛ1Ҿ਺Ͱ࣮ߦ͢ΔͱshellΛհ͠
   ࣮ͯߦ͢ΔՄೳੑ͕͋Δɻ
   

   View Slide

7. shellΛհ͢΂͔Βͣ
   • ͍ΘΏΔshell injectionͷ͓ͦΕ͕͋Δɻ$dirʹ
   มͳจࣈ͍ΕΒΕͨΒʁ
   • ֎෦ϓϩάϥϜʹγάφϧૹͬͨͭ΋Γ͕shell
   ʹૹΔ͜ͱʹͳΔ
   • ۭനจࣈΛؚΉϑΝΠϧͰద੾ʹΫΦʔτ͠ͳ
   ͚Ε͹͍͚ͳ͍
   

   View Slide

8. shellΛհ͢΂͔Βͣ
   • shellΛىಈ͠ͳ͍ͷ͕Ұ൪
   • ͍ΘΏΔinderectه๏Λ࢖͑͹Ͱ͖Δ
   

   View Slide

9. #!/usr/bin/env perl
   use strict;
   use warnings;
   system {"perl"} "perl", "-E", 'say "hello!"';
   

   View Slide

10. ֎෦ϓϩάϥϜΛ࣮ߦ͢Δ

    Perlͷؔ਺
    • ҎԼͷ4ͭ
    • system
    • ` ` (όοΫΫΦʔτ, qx)
    • fork & exec ← ࠓ೔͸͜Εʹ஫໨
    • open (͜Ε͕Ұ൪PerlͬΆ͍!)
    

    View Slide

11. fork & exec
    • ࠓ೔͸fork & execͰ֎෦ϓϩάϥϜΛىಈͯ͠
    ΈΔ
    • fork͸ࣗ෼ͷ෼਎Λ࡞ΔγεςϜίʔϧ
    • exec͸ࣗ෼ΛมԽͤ͞ΔγεςϜίʔϧ
    

    View Slide

12. #!/usr/bin/env perl
    use strict;
    use warnings;
    my $pid = fork // die;
    if ($pid == 0) {
    exec {"ls"} "ls", "-al";
    exit 127;
    }
    wait;
    my $status = $?;
    

    View Slide

13. redirect
    • system "gzip —stdout out.txt";
    • ίϚϯυͷग़ྗΛout.txt.gzʹ޲͚͍ͨɻ
    • system "gzip —stdout out.txt > out.txt.gz"; ͸
    ΍Γͨ͘ͳ͍ɻ
    • open STDOUT, ">", "out.txt.gz" ͰͰ͖Δ
    

    View Slide

14. #!/usr/bin/env perl
    use strict;
    use warnings;
    my $pid = fork // die;
    if ($pid == 0) {
    open STDOUT, ">", "out.txt.gz";
    exec {"gzip"} "gzip", "--stdout", "out.txt";
    exit 127;
    }
    wait;
    my $status = $?;
    

    View Slide

15. pipe
    • ௕͘ଓ͘ϓϩάϥϜͷग़ྗʹ࣌ࠁΛ͚͍ͭͨɻ
    • pipeͰ֎෦ϓϩάϥϜͷग़ྗΛϦΞϧλΠϜʹ
    ड͚औΕ͹͍͍
    

    View Slide

16. #!/usr/bin/env perl
    use strict;
    use warnings;
    use POSIX 'strftime';
    pipe my $stdout_read, my $stdout_write;
    my $pid = fork // die;
    if ($pid == 0) {
    close $stdout_read;
    open STDOUT, ">&=", $stdout_write;
    exec {"perl"} "perl", "-le",
    '$|++; for (1..10) { print $_; sleep 1 }';
    }
    close $stdout_write;
    while (<$stdout_read>) {
    print strftime("%FT%T", localtime) . " $_";
    }
    wait;
    

    View Slide

17. stderr΋औΓ͍ͨ
    

    View Slide

18. #!/usr/bin/env perl
    use strict;
    use warnings;
    use POSIX 'strftime';
    pipe my $stdout_read, my $stdout_write;
    my $pid = fork // die;
    if ($pid == 0) {
    close $stdout_read;
    open STDOUT, ">&=", $stdout_write;
    open STDERR, ">&=", \*STDOUT;
    exec {"perl"} "perl", "-le",
    '$|++; for (1..10) { print $_; warn $_; sleep 1 }';
    }
    close $stdout_write;
    while (<$stdout_read>) {
    print strftime("%FT%T", localtime) . " $_";
    }
    wait;
    ϦμΠϨΫτ!
    

    View Slide

19. stdout/errผʑʹऔΓ
    ͍ͨ
    

    View Slide

20. #!/usr/bin/env perl
    use strict;
    use warnings;
    use POSIX 'strftime';
    use IO::Select;
    pipe my $stdout_read, my $stdout_write;
    pipe my $stderr_read, my $stderr_write;
    my $pid = fork // die;
    if ($pid == 0) {
    close $stdout_read;
    close $stderr_read;
    open STDOUT, ">&=", $stdout_write;
    open STDERR, ">&=", $stderr_write;
    exec {"perl"} "perl", "-le",
    '$|++; for (1..10) { print $_; warn $_; sleep 1 }';
    }
    close $stdout_write;
    close $stderr_write;
    my $select = IO::Select->new($stdout_read, $stderr_read);
    while ($select->count) {
    for my $fh ($select->can_read) {
    my $len = sysread $fh, my $str, 1024;
    if ($len) {
    my $type = $fh == $stdout_read ? "out": "err";
    print strftime("%FT%T", localtime) . " $type $str";
    } else {
    $select->remove($fh);
    close $fh;
    }
    }
    }
    wait;
    select!
    

    View Slide

21. ·ͱΊ
    • ֎෦ϓϩάϥϜ࣮ߦͰ͸indirectه๏Λ࢖ͬͯ
    shellΛհ͞ͳ͍ͷ͕͍͍΍Γํ
    • open STDOUT, ">", "file.txt" ͰSTDOUTΛϑΝ
    ΠϧʹϦμΠϨΫτͰ͖Δ
    • pipeΛۦ࢖ͯ͠֎෦ϓϩάϥϜͱ௨৴Ͱ͖Δ
    

    View Slide

22. Further Reading
    • CPAN moduleΛಡΉͷ͕͓͢͢Ί
    • IPC::Run3, System::Command, Capture::Tiny
    • Proclet, Server::Starter
    

    View Slide