$30 off During Our Annual Pro Sale. View Details »

Perl5の静的解析入門 / The static analysis of Perl5

mackee
January 26, 2019

Perl5の静的解析入門 / The static analysis of Perl5

Perl5の静的解析入門
機械と人間双方の歩み寄りによる平和編

YAPC::Tokyo 2019 Room1 14:00 -

mackee

January 26, 2019
Tweet

More Decks by mackee

Other Decks in Programming

Transcript

  1. Perl5ͷ੩తղੳೖ໳
    ػցͱਓؒ૒ํͷาΈدΓʹΑΔฏ࿨ฤ
    YAPC::Tokyo 2019
    ໘ന๏ਓΧϠοΫ macopy a.k.a @mackee_w
    #yapcjapan #yapcjapanRoom1
    1

    View Slide

  2. ͜ͷεϥΠυ
    2

    View Slide

  3. ͜ͷτʔΫ
    ͷ಺༰
    3

    View Slide

  4. ಘΒΕΔ஌ࣝɾମݧ (1)
    •ਖ਼نදݱͰPerlίʔυΛύʔε͢ΔPPR.pm
    ͷ࢖͍ํ
    • ۀ຿Ͱͷ۩ମతͳ࢖༻ྫ
    •ਓؒͱػցʹ෼͔Γʹ͍͘ίʔυͱ௚͠ํ
    ͷఏҊ
    4

    View Slide

  5. ಘΒΕΔ஌ࣝɾମݧ (2)
    •Perlίʔυͷؔ਺ʹܕγάωνϟΛ෇͚Δ
    •ؔ਺ͷܕγάωνϟΛPPRͰൈ͖ग़͢
    5

    View Slide

  6. ࿩ͷྲྀΕ
    1. ੩తղੳͱ͸
    2. PPIΛ࢖͏ྫ
    3. PPRͷઆ໌ͱྫ
    4. ػցͱฏ࿨ͷ࿩
    5. ੩తܕ෇͚ͷೖΓޱ
    6

    View Slide

  7. ࢲͱ͸ɾɾɾʁ
    • ໘ന๏ਓΧϠοΫ ιʔγϟ
    ϧήʔϜࣄۀ෦
    • ࢓ࣄݴޠ͸Perl5ͱGo
    • झຯ͸3DϓϦϯλ࡞Γ
    • աڈͷτʔΫ: ήʔϜӡ༻ͷ
    ࿩ͱ͔Ϛελσʔλͷ࿩ͱ͔
    WebSocket
    7

    View Slide

  8. ΠϯλʔωοτͰ͸͜͏͍͏ΞΠ
    ίϯͰ͢
    8

    View Slide

  9. [Ad]͜ͷτʔΫ͸ۀ຿࣌ؒʹॻ͖·ͨ͠
    9

    View Slide

  10. ੩తղੳͱ͸Ұ
    ମɾɾɾʁ
    10

    View Slide

  11. static analysis
    ੩తղੳ
    11

    View Slide

  12. ίʔυΛͨͩͷ
    จࣈྻͱͯ͠ѻ͏
    12

    View Slide

  13. ΍Γํ 1
    ਖ਼نදݱͰҾֻ͚ͬͨΓී௨ͷςΩετͱͯ͠ѻ͏
    use Path::Tiny;
    my $target_package = "Example";
    my $script = path("$target_package.pm")->slurp;
    my $is_valid_package = $script =~ /\Apackage $target_package;/;
    13

    View Slide

  14. ͜͏͍͏ͷ͕ग़ͯ͘Δͱ؆୯ʹ٧Ή
    use
    Example;
    14

    View Slide

  15. ΑΓ࣮֬ͳ΍Γํ͸ͳ͍ͷ͔ʂ
    15

    View Slide

  16. ΍Γํ 2
    perl͕΍Δ͜ͱΛ్த·Ͱ΍ͬͯɺந৅ߏจ໦Λར༻͢Δ
    16

    View Slide

  17. ίϥϜ: ࠷ۙͷݴޠͩͱݴޠຊମͷASTΛར༻Ͱ͖Δ͜ͱ͕͋Δ
    •Goͷ go/parser go/token go/types go/ast
    • GoίϯύΠϥ΋͜ΕΒΛ࢖͍ͬͯΔ
    • ͦ͏Ͱ͸ͳ͍Β͍͠
    •Rubyͷ RubyVM::AST
    • 2.6.0͔Βར༻Մೳ
    17

    View Slide

  18. ੩తղੳͷϝϦοτ (1)
    •ίʔυͷݟͨ··Λѻ͑Δ
    • ந৅ߏจ໦ҎલͰ͋Ε͹ίϝϯτͷதΛ
    ѻ͑Δ͜ͱ΋
    •ݴޠػೳΛ௒͑ͨදݱྗ
    • ޙ͔Β੩తܕ෇͚ͱ͔
    18

    View Slide

  19. ࣮ߦग़དྷΔঢ়ଶʹͳΔʹͭΕͯιʔείʔυ্ͷ৘ใ͸ൈ͚མ͍ͪͯ͘
    19

    View Slide

  20. ࣮ߦ࣌ղੳͰ͸ར༻Ͱ͖ͳ͍৘ใ
    •ۭന, վߦ, ίϝϯτ
    • ಛघͳॻ͖ํͰར༻Ͱ͖Δ৔߹΋͋Δ
    (ྫ: Pythonͷdocstring)
    20

    View Slide

  21. ࣮ߦ࣌ղੳͰ͸ར༻Ͱ͖ͳ͍৘ใ
    •τʔΫϯͷҐஔ΍લޙͷτʔΫϯ
    • ར༻͞Ε͍ͯΔม਺͕࣮ࡍʹ͸Ͳ͜Ͱએ
    ݴ͞Ε͍ͯΔ͔ͱ͔
    • ؔ਺΍จͷҐஔͳͲ͸ελοΫτϨʔε
    Ͱར༻͢ΔͨΊʹ෇͍͍ͯΔ
    21

    View Slide

  22. ੩తղੳͷϝϦοτ (2)
    •࣮ߦͤͣʹίʔυΛѻ͑Δ
    • BEGINͰϠόΠ͜ͱ͍ͯͯ͠΋େৎ෉
    • ٯʹݴ͏ͱBEGINͰେࣄͳ͜ͱ͍ͯͯ͠
    ΋ݕ஌͸೉͍͠(ޙड़)
    22

    View Slide

  23. ੩తղੳͷϝϦοτ (3)
    •͢΂ͯͷ࣮ߦύεʹ͍ͭͯௐ΂্͛ΒΕΔ
    ৔߹͕͋Δ
    • ࣮ߦ࣌ʹܕόϦσʔγϣϯ͢Δ৔߹ɺ࣮
    ߦ͢Δ·Ͱ੒ޭ͢Δ͔Ͳ͏͔෼͔Βͳ͍
    23

    View Slide

  24. ੩తղੳͷσϝϦοτ
    •PerlͷύʔεΛࣗલ(Ϟδϡʔϧ࢖͏͕)Ͱ΍
    Βͳ͍ͱ͍͚ͳ͍͔Βେม
    •ͦ΋ͦ΋໦ߏ଄ΛಡΜͰ͍ͬͯͲ͏ʹ͔͢
    Δͷେม
    •perlͷؾ࣋ͪʹͳΒͳ͍ͱ͍͚ͳͯ͘େม
    24

    View Slide

  25. ࢲ͕੩తղੳ͍ͨ͠ಈػ
    ػցͷྗΛआΓͯ
    ؒҧ͍ͷͳ͍ίʔυΛ
    ॻ͖͍ͨʂ
    25

    View Slide

  26. Perl5Ͱ੩తղੳ
    ͢ΔͨΊͷπʔϧ
    26

    View Slide

  27. Perl5੩తղੳքͷୈҰਓऀ
    PPI
    27

    View Slide

  28. PPI
    Parse, Analyze and Manipulate Perl (without
    perl)
    •Pure Perl͚ͩͰPerlίʔυΛύʔεͯ͠੩
    తղੳͰར༻Ͱ͖Δܗʹ͢ΔϞδϡʔϧ
    28

    View Slide

  29. ࢖༻ྫ (1)
    use PPI;
    use PPI::Dumper;
    my $document = PPI::Document->new(\'my $v1 = $v2;');
    PPI::Dumper->new($document)->print;
    29

    View Slide

  30. ࢖༻ྫ (2)
    PPI::Document
    PPI::Statement::Variable
    PPI::Token::Word 'my'
    PPI::Token::Whitespace ' '
    PPI::Token::Symbol '$v1'
    PPI::Token::Whitespace ' '
    PPI::Token::Operator '='
    PPI::Token::Whitespace ' '
    PPI::Token::Symbol '$v2'
    PPI::Token::Structure ';'
    30

    View Slide

  31. ࢖༻ྫ (3)
    my ($stmt) = $document->children;
    my $op = $stmt->find_first("PPI::Token::Operator");
    $op->content; # => "="
    31

    View Slide

  32. PPIͷಛ௃ (1)
    •PDOM(ύʔεޙͷ໦ߏ଄ͷ͜ͱ)͔Βίʔυ
    ʹॻ͖໭͢͜ͱ͕ग़དྷΔ
    $document->prune("PPI::Token::Whitespace");
    $document->serialize; # => "my$v1=$v2;"
    32

    View Slide

  33. PPIͷಛ௃ (2)
    •ॻ͖໭ͨ͢ΊʹτʔΫϯ΍εϖʔεɺ;ͳͲ
    ͸อ࣋ͨ͠··
    •͋͘·Ͱղੳ͢ΔͨΊͷߏ଄Ͱ͋ͬͯɺ࣮
    ߦ͢ΔͨΊͷߏ଄Ͱ͸ͳ͍
    33

    View Slide

  34. PPIͷσϝϦοτ
    •݁ߏ஗͍
    • 1175ߦͷ࣮ࡍͷۀ຿ͷίʔυΛύʔε͠
    ͨ݁Ռ
    • 6.80/s1
    1 Macbook Pro 15inch mid 2015 Ͱܭଌ
    34

    View Slide

  35. ͦͷଞͷϞδϡʔϧ
    •Compiler::Lexer ಠࣗͷCݴޠͰॻ͔Εͨ
    τʔΧφΠβΛ࢖༻
    •Perl::Lexer perlຊମͷ಺෦APIΛୟ͍ͯτʔ
    ΫϯྻΛऔΓग़͢
    35

    View Slide

  36. PPRͷ঺հͱ࢖
    ͍ํ
    36

    View Slide

  37. PPR
    Pattern-based Perl Recognizer
    •PerlίʔυͷύλʔϯΛఆٛͯ͠ݕ஌Ͱ͖
    Δਖ਼نදݱΛ࡞ΕΔϞδϡʔϧ
    37

    View Slide

  38. ࢖༻ྫ
    # my $hoge = "fuga"; ͷม਺໊ΛҾֻ͚ͬΔʹ͸
    my $matcher = qr{
    my (?&PerlOWS)
    ((?&PerlVariableScalar)) (?&PerlOWS)
    (?&PerlAssignmentOperator) (?&PerlOWS)
    (?&PerlString) (?&PerlOWS) ;
    $PPR::GRAMMAR
    }x
    38

    View Slide

  39. ࢖༻ྫ
    my ($var) =
    grep { defined } $script =~ $matcher;
    ͜ΕͰ $var ʹ "$hoge" ͕ೖΔ
    39

    View Slide

  40. ศརʂʂʂ
    40

    View Slide

  41. ʮΘʔʂ͍͖ͳΓ
    ϠόΠਖ਼نදݱ͕
    ग़͖ͯͨͧʂʂʯ
    41

    View Slide

  42. ଴͍ͬͯͩ͘͞
    ղઆ͠·͢
    42

    View Slide

  43. PPR͸PerlίʔυͷτʔΫϯʹϚον͢Δਖ਼نදݱͷύλʔϯू
    Perlίʔυྫ PPRͷύλʔϯ
    my (?&PerlBuiltinFunction)
    $var (?&PerlVariableScalar)
    = (?&PerlAssignmentOperator)
    "hogehoge" (?&PerlString)
    43

    View Slide

  44. සग़ύλʔϯ
    •(?&PerlOWS)
    • θϩจࣈҎ্ͷPerl্ͷۭനจࣈ
    • ۭന΍վߦΛؚΉ
    44

    View Slide

  45. ࣮༻ྫ
    45

    View Slide

  46. ฐࣾͩͱ͜͏͍͏όϦσʔγϣϯΛΑ͘΍͍ͬͯΔ͚ΕͲ
    sub do_something {
    my $self = shift;
    state $rule = Data::Validator->new(
    a1 => "Str",
    b1 => "UInt",
    );
    my $args = $rule->validate(@_);
    my ($a1, $b1) = $args->{qw/a1 b1/};
    # do something...
    }
    46

    View Slide

  47. ϓϥΠϕʔτϝιου
    ͡Όͳ͔ͬͨΒ
    ͪΌΜͱόϦσʔγϣϯ
    ͯ͠ΔΑͶνΣοΫ
    47

    View Slide

  48. εΫϦϓτ͔Βؔ਺ఆٛΛൈ͖ग़͢
    my $script = ...;
    my $sub_matcher = qr{
    ((?&PerlSubroutineDeclaration))
    $PPR::GRAMMAR
    }x;
    my @decls = grep { defined } $script =~ m{$sub_matcher}gx;
    48

    View Slide

  49. ؔ਺ఆ͔ٛΒؔ਺໊ͱϒϩοΫΛൈ͖ग़͢
    my $subname_matcher = qr{
    \Asub (?&PerlOWS)
    ((?&PerlQualifiedIdentifier)) (?&PerlOWS)
    ((?&PerlBlock))
    $PPR::GRAMMAR
    }x;
    for my $decl (@decls) {
    my ($subname, $block) =
    grep { defined } $decl =~ $subname_matcher;
    49

    View Slide

  50. ؔ਺໊͕ΞϯμʔείΞ࢝·ΓͳΒεΩοϓ͢Δ
    for my $decl (@decls) {
    my ($subname, $block) =
    grep { defined } $decl =~ $subname_matcher;
    next if $subname =~ /\A_/;
    50

    View Slide

  51. validatorͷఆٛ
    my $validator_matcher = qr{
    (
    (state) (?&PerlOWS)
    (?&PerlVariableScalar) (?&PerlOWS)
    (?&PerlAssignmentOperator) (?&PerlOWS)
    Data::Validator->new (?&PerlOWS)
    (?&PerlParenthesesList) (?&PerlOWS)
    )
    $PPR::GRAMMAR
    }x;
    51

    View Slide

  52. validatorͷఆ͕ٛ͋Δ͔Λௐ΂Δ
    for my $decl (@decls) {
    ...
    next if $subname =~ /\A_/;
    my ($validator) =
    grep { defined } $block =~ $validator_matcher;
    52

    View Slide

  53. ͋ͱ͸ࣽΔͳΓম͘ͳΓ
    # warningΛग़͢ͳΓ
    if (!defined $validator) {
    warn "$subname Ͱ Data::Validator ͕࢖ΘΕͯͳ͍Αʂ";
    }
    # ςετͰ͚ͤ͜͞ΔͳΓ
    use Test::More;
    fail "$subname Ͱ Data::Validator ͕࢖ΘΕͯͳ͍Αʂ";
    53

    View Slide

  54. ͜ͷνΣοΫͰൈ͚͍ͯΔͱ͜Ζ
    •Ҿ਺ΛऔΒͳ͍ؔ਺Λޡݕ஌
    •Attribute΍Subroutine signature
    •Ҿ਺όϦσʔγϣϯҎ֎ͷvalidator
    54

    View Slide

  55. ׬ᘳΛ໨ࢦͣ͞ʹࣗ෼ͷ࣋ͭίʔυ͚ͩͰ௨Ε͹ྑ͍
    ͋Δఔ౓͸αϘΔ
    55

    View Slide

  56. ͞ΒʹԠ༻Ҋ
    •Smart::Args Λ࢖͏Α͏ʹஔ׵͢Δ
    • όϦσʔλͷதͷݸʑͷఆٛ·Ͱόϥ͠
    ͯൈ͖ग़ͤΔ
    •LSPͳͲͰҾ਺ϦετΛग़͢
    56

    View Slide

  57. PPR͸ਖ਼نදݱͰίʔυ
    ͷҰ෦Λൈ͖ग़ͯ͠
    ݕࠪ΍ஔ׵ʹ࢖͑Δ
    57

    View Slide

  58. ͦ͏͍͏໨తͷطଘͷϞδϡʔϧ
    •Perl::Critic
    • PPIΛ࢖ͬͨνΣοΧʔ
    •Perl::Lint
    • Compiler::LexerΛ࢖ͬͨνΣοΧʔ
    58

    View Slide

  59. ੩తղੳͰ
    ίʔυͷ࣏҆Λ
    Α͍ͯ͘͘͠ʹ͸ʁ
    59

    View Slide

  60. ػցͱਓؒʹྑ͍
    ίʔυͱ
    ཆ੒Ϊϓεͱͯ͠
    ͷ੩తղੳ
    60

    View Slide

  61. perl͸ೲಘ͕Ͱ͖Δڍಈ΋͢Δ͠
    ڻ͖MAXͷڍಈΛى͜͢͜ͱ΋͋Δ
    61

    View Slide

  62. Only perl can parse Perl (1)
    •Only perl can parse Perl ͳ෦෼
    •PPIͷυΩϡϝϯτΑΓ
    @result = (dothis $foo, $bar);
    # Which of the following is it equivalent to?
    @result = (dothis($foo), $bar);
    @result = dothis($foo, $bar);
    62

    View Slide

  63. Only perl can parse Perl (2)
    $ perl -MO=Deparse -e '@result = (dothis $foo, $bar);'
    @result = ($foo->dothis, $bar);
    -e syntax OK
    $ perl -MO=Deparse -e 'sub dothis {} @result = (dothis $foo, $bar);'
    sub dothis {
    }
    @result = dothis($foo, $bar);
    -e syntax OK
    63

    View Slide

  64. ΍Γ͍ͨํʹ໌ࣔ͢Δʂʂʂ
    @result = dothis($foo, $bar);
    •͍ΖΜͳॻ͖ํ͕Ͱ͖Δݴޠ͔ͩΒͦ͜ɺ
    ΍Γ͍ͨ͜ͱΛओு͢Δ
    64

    View Slide

  65. Only perl can parse Perl (3)
    sub hoge { "hoge" }
    sub f {
    {
    hoge() => "fuga"
    }
    }
    # hashref or list ?
    my @result = f();
    65

    View Slide

  66. Only perl can parse Perl (4)
    use DDP; p @result;
    # [
    # [0] "hoge",
    # [1] "fuga"
    # ]
    ϦετʹͳΔͷ͕ਖ਼ղ
    66

    View Slide

  67. Only perl can parse Perl (5)
    hashrefʹ͢Δʹ͸ʁ
    •return Λ෇͚Δ
    •ϑΝοτΧϯϚͷࠨล஋Λจࣈྻʹ͢Δ
    •+{} ʹ͢Δ
    67

    View Slide

  68. ͭ·ΓͪΌΜͱreturnॻ͚͹ղܾ͢Δ
    sub hoge { "hoge" }
    sub f {
    return {
    hoge() => "fuga"
    };
    }
    # hashref desu!!!
    my @result = f();
    68

    View Slide

  69. Only perl can parse Perl (6)
    •1ͭͷه߸͕ෳ਺ͷҙຯʹ࢖ΘΕ͍ͯͯ೉
    ͍͠
    • είʔϓͱϋογϡϦϑΝϨϯεͷ {}
    • আࢉͱਖ਼نදݱͷ /
    69

    View Slide

  70. ʮ͜Ε͔ͩΒPerl͸ʂʯ
    70

    View Slide

  71. !
    ͚ΕͲ
    ίʔυͷ໎͍͸ࢥߟͷ໎͍
    Ұݺٵ͓͍ͯɺ΋͏গ͠ߟ͑Α͏
    71

    View Slide

  72. άϨούϏϦςΟ
    greppability
    72

    View Slide

  73. greppability
    •ૂͬͨίʔυΛgrepͰݕ஌Ͱ͖Δ͔Ͳ͏͔
    • ΄͔ͷఆ͕ٛ͋ͬͨΒڭ͍͑ͯͩ͘͞
    $ hw 'Greeter->say_hello' lib | \
    xargs perl -pie 's/Greeter->say_hello/Greeter->send_hello'
    73

    View Slide

  74. greppability͕௿͍ίʔυ
    •ϝιου໊Λಈతʹ૊Έཱ͍ͯͯΔ
    my @params = qr/str vit dex int/;
    my $total = sum(
    map {
    my $method = $_ . "_factor";
    $person->$method;
    } @params
    );
    74

    View Slide

  75. ϝιου໊Λಈతʹ૊Έཱͯͨͱ͖ͷฐ֐
    •ʮ͜ͷϝιουͲ͜Ͱ࢖͍ͬͯΔʁʯ͕೉
    ͘͠ͳΔ
    •ϝιουݺͼग़͠Λ੩తղੳͰநग़͢Δͱ
    ͖ʹ൑ఆͰ͖ͳ͘ͳΔ
    ͱʹ͔͘ػցʹ༏͘͠ͳ͍ʂʂʂ
    75

    View Slide

  76. ͤΊͯจࣈྻ࿈݁Λ΍ΊΔ
    my @methods = qr/
    str_factor
    vit_factor
    dex_factor
    int_factor
    /;
    my $total = sum(map { $person->$_ } @methods);
    •ϝιου໊͕ϢχʔΫͰ͋Ε͹grepʹҾͬ
    ͔͔ͬͯ͘Δ
    76

    View Slide

  77. ಈతܕ෇͚ݴޠͰ΋৺ʹ੩తܕ෇͚ίϯύΠϥΛ࣋ͭ
    •ίʔυΛॻ͍ͯܕ͕੩తʹܾఆ͢Δ͔Λߟ
    ͑Δ
    •Πϛϡʔλϒϧม਺ͳͲϞμϯݴޠʹ͋Δ
    ػೳ΍੍ݶΛಋೖ͢Δͱྑ͍
    77

    View Slide

  78. ͦͷଞ੩తղੳͰࠔΔ΋ͷ
    •࣮ߦ࣌ʹuse͢Δ΋ͷ͕ܾ·Δͱ͔ BEGIN
    ಈ͔͞ͳ͍ͱ֬ఆ͠ͳ͍ͱ͔
    •Mo+([uo]se)? ͷ has ΍ method ͸ͦΕΛղऍ
    ͢Δ΋ͷΛ࡞Ε͹·ͩܕͷ੩తղܾ͕Ͱ͖
    Δ৔߹͕͋Δ
    78

    View Slide

  79. ػցʹ༏͚͠Ε͹ಡΉਓؒʹ΋͍͍ͩͨ༏͍͠
    •ॻ͘ਓؒʹ͸ݫ͘͠ͳΔ͜ͱ͕͋Δ
    • ॻ͘ͷ͸Ұճ͔ͭҰਓɺಡΉͷ͸n(n>=1)
    ճ͔ͭmਓ
    •ػցʹ༏͚͠Ε͹ػցͷࢧԉΛड͚Δ͜ͱ
    ͕ग़དྷΔ
    79

    View Slide

  80. ػցʹ΋ਓؒʹ΋༏͍͠Perlίʔυ͸PerlͷαϒηοτʹͳΔ
    => ੩తղੳͰྑ͘ͳ͍ίʔυΛݕ஌Ͱ͖Δ
    80

    View Slide

  81. ਓؒʹ͸ཉ΍ࡶ೦͕͋ΔͨΊɺࣗݾΛ཯͢Δͷ͸೉͍͠
    •ʮλΠϓ਺ଟ͘ͳΔ͔ΒखΛൈ͜͏ʯͬͯ
    ͳΔ͜ͱ͕͋Δ
    •ػցʹ஫ҙͯ͠΋Β͏
    • returnෆ଍ݕ஌ͷϙϦγʔ͸ Perl::Critic
    ʹ͋Δ
    81

    View Slide

  82. ϋ΢εϧʔϧ
    •Rubyͷࣄྫ Querly
    •ʮ͜͏͍͏ॻ͖ํ͏ͪͩͱڻ͖͕͍ͬͺ͍
    ͔ͩΒνΣοΫ͍ͨ͠ΑͶʯ
    •PerlͰ΋PPRͰݕ஌Մೳ
    82

    View Slide

  83. ͦͷଞϋ΢εϧʔϧͷྫ
    •Plack::RequestͰ $c->req->param(...)͸࢖Θ
    ͳ͍
    •DBIx::Class Ͱ ResultSet->search Λ arrayί
    ϯςΩετͰड͚ͳ͍
    ͳͲͳͲ
    83

    View Slide

  84. Ұ୴࣭໰λΠϜ
    84

    View Slide

  85. ͔͜͜Β͸͍࣌ؒͬͺ͍·Ͱ
    ૸Γൈ͚·͢
    85

    View Slide

  86. PPRΛ࢖͍ͬͯͨ͘Ίʹ
    PPRΛ΋ͬͱਂ͘஌Ζ
    ͏ʂ
    86

    View Slide

  87. PPRͷυΩϡϝϯτʹ͸࢖͑Δਖ਼نදݱͷϧʔϧ͕ྻڍ͞Ε͍ͯΔ͕...
    87

    View Slide

  88. υΩϡϝϯτಡΜͰ΋
    ۩ମతʹԿʹϚον
    ͢Δ͔Θ͔ΒΜͱ͜ΖΛ
    ίʔυͰಡΜͰΈΔ
    88

    View Slide

  89. $PPR::GRAMMAR ΛݟͯΈΔ
    my $stmt_matcher = qr{
    \G (?&PerlOWS)
    ((?&PerlStatement))
    $PPR::GRAMMAR # <= ίϨ
    }x;
    my @stmts = grep { defined } $script =~ m{$stmt_matcher}gcx;
    89

    View Slide

  90. 90

    View Slide

  91. !
    1761ߦ
    ໨ͱ೴ʹ௚઀μϝʔδ͕དྷΔߦͷਖ਼نදݱ
    91

    View Slide

  92. ਖ਼نදݱ͓͹͚
    !
    ʹཱͪ޲͔͏
    92

    View Slide

  93. ಡΈํ (1)
    •(? ... ) ϧʔϧఆٛ
    •(?&PerlStatementSequence) ϧʔϧݺͼग़͠
    93

    View Slide

  94. ಡΈํ (2)
    (?
    \x{FEFF}?+ # Optional BOM marker
    (?&PerlStatementSequence)
    ) # End of rule
    Ͷɺ؆୯Ͱ͠ΐʁ
    94

    View Slide

  95. ಡΈํ (3)
    ͱ͖Ͳ͖͜͏͍͏পʹग़͘Θ͚͢ΕͲ
    (?
    # Optimized to match any Perl builtin name, without backtracking...
    (?=[^\W\d]) # Skip if possible
    (?>
    s(?>e(?>t(?>(?>(?>(?>hos|ne)t|gr)en|s(?>erven|ockop))t|p(?>r(?>iority|otoent)|went|grp))
    | g(?>et(?>p(?>r(?>oto(?>byn(?>umber|ame)|ent)|iority)|w(?>ent|nam|uid)|eername|grp|pid)|
    | r(?>e(?>ad(?>lin[ek]|pipe|dir)?|(?>quir|vers|nam)e|winddir|turn|set|cv|do|f)|index|mdir
    | c(?>h(?>o(?>m?p|wn)|r(?>oot)?|dir|mod)|o(?>n(?>tinue|nect)|s)|lose(?>dir)?|aller|rypt)
    | e(?>nd(?>(?>hos|ne)t|p(?>roto|w)|serv|gr)ent|x(?>i(?>sts|t)|ec|p)|ach|val(?>bytes)?+|of
    | l(?>o(?>c(?>al(?>time)?|k)|g)|i(?>sten|nk)|(?>sta|as)t|c(?>first)?|ength)
    | u(?>n(?>(?>lin|pac)k|shift|def|tie)|c(?>first)?|mask|time)
    | p(?>r(?>ototype|intf?)|ack(?>age)?|o[ps]|ipe|ush)
    | d(?>bm(?>close|open)|e(?>fined|lete)|ump|ie|o)
    | f(?>or(?>m(?>line|at)|k)|ileno|cntl|c|lock)
    | t(?>i(?>mes?|ed?)|ell(?>dir)?|runcate)
    95

    View Slide

  96. Regexp::OptimizerͰੜ੒͞ΕͯΔΈ͍͔ͨͩ
    Βେৎ෉ʂ
    96

    View Slide

  97. ͍͍ײ͡ͷࢹ֮ԽΛ͢Δ
    •ఆ͕ٛผͷͲΕΛࢀর͍ͯ͠Δ͔
    • (?&PerlVariable) ͸
    • (?&PerlScalarAccess) ͱ
    • (?&PerlHashAccess) ͱ
    • (?&PerlArrayAccess) ͷ OR ͩΑ Έ͍ͨͳ
    97

    View Slide

  98. ͍͍ײ͡ͷՄࢹԽ (1)
    •PPRͷਖ਼نදݱͷจࣈྻΛൈ͖ग़͢
    • ਖ਼نදݱϦϑΝϨϯεͩͱ͍Ζ͍Ζͬ͘
    ͍ͭͯ͘ΔͷͰίʔυ্ͷςΩετͦͷ
    ··͕΄͍͠
    • PPRͰൈ͖ग़͢
    98

    View Slide

  99. our $GRAMMAR = qr{ ... };
    # ίί ^^^^^^^^^
    99

    View Slide

  100. ͍͍ײ͡ͷՄࢹԽ (2)
    my $stmt_regexp = qr{
    our (?&PerlOWS) \$GRAMMAR (?&PerlOWS) = (?&PerlOWS) ((?&PerlRegex))
    $PPR::GRAMMAR
    }x;
    my ($regexp) = grep { defined } $ppr_script =~ /$stmt_regexp/gcx;
    # => "qr{ ... }"
    100

    View Slide

  101. ͍͍ײ͡ͷՄࢹԽ (3)
    •PPIx::Regexp Ͱ ਖ਼نදݱͷந৅ߏจ໦ʹ͢
    Δ
    use PPIx::Regexp;
    my $tree = PPIx::Regexp->new($regexp);
    101

    View Slide

  102. ͍͍ײ͡ͷՄࢹԽ (4)
    •͋ͱ͸࠶ؼͰτʔΫϯΛऩू͍ͯ͘͠
    sub traverse {
    my ($tree, $bucket, $name) = @_;
    my $literal = "";
    for my $child ($tree->children) {
    if (ref $child eq "PPIx::Regexp::Token::Literal") {
    $literal .= $child->content;
    }
    elsif (ref $child eq "PPIx::Regexp::Token::Whitespace") {
    # skip
    }
    else {
    if (length($literal) > 1 && $literal ne '\n') {
    $bucket->{$name}->{'"' . $literal . '"'} = 1;
    }
    $literal = "";
    }
    102

    View Slide

  103. ͍͍ײ͡ͷՄࢹԽ (5)
    if (ref $child eq "PPIx::Regexp::Token::Recursion") {
    next if $child->name eq "PerlOWS";
    $bucket->{$name}->{$child->content} = 1;
    }
    if ($child->can("children")) {
    my $next_name = $name;
    if (ref $child eq "PPIx::Regexp::Structure::NamedCapture") {
    next if $child->name =~ /^PPR_/;
    $next_name = $name ? $name . " > (?&" . $child->name . ")" : "(?&" . $child->name . ")";
    #PPIx::Regexp::Dumper->new($child)->print;
    }
    traverse($child, $bucket, $next_name);
    }
    }
    }
    103

    View Slide

  104. ͍͍ײ͡ͷՄࢹԽ (6)
    (?&PerlPackageDeclaration)
    |-- "\}"
    |-- "package"
    |-- (?&PerlBlock)
    |-- (?&PerlNWS)
    |-- (?&PerlQualifiedIdentifier)
    `-- (?&PerlVersionNumber)
    (?&PerlParenthesesList)
    |-- "\("
    `-- (?&PerlExpression)
    (?&PerlPod)
    `-- "=cut"
    104

    View Slide

  105. ࣗ༝ࣗࡏʹτʔΫϯΛൈ͖ग़ͤΔʂ
    PPRͷத਎΋Θ͔ͬͨʂ
    105

    View Slide

  106. ͜͜·ͰདྷΔͱܕΛॻ͍ͯ
    ࢖͍ͨ͘ͳΓ·ͤΜ͔ʁ
    106

    View Slide

  107. ઴ਐత੩తܕ෇͚
    •ಈతܕ෇͚ݴޠʹ෦෼తʹ੩తܕ෇͚Λಋ
    ೖ͢Δ
    • TypeScript(JavaScript) ΍ Hack(PHP) ͳ
    Ͳ
    107

    View Slide

  108. •ޙ෇͢Δʹ͸ҎԼͷཁૉ͕ඞཁ
    • ܕ͕ॻ͚ΔΑ͏ʹͳΔ (PythonͷPEP
    484 Type Hints)
    • ੩తղੳͰܕΛௐ΂Δ͜ͱ͕ग़དྷΔ
    • ܕγεςϜΛ࣮૷͢Δ
    108

    View Slide

  109. ܕ͸͢Ͱʹॻ͚Δ
    package Greeter {
    use Function::Parameters;
    use Function::Return;
    use Types::Standard qw/Str Int Bool/;
    method say_hello(Str :$name, Int :$times) :Return(Bool) {
    say $self . ": Hello " . $name for 1..$times;
    return 1;
    }
    }
    Greeter->say_hello(name => "shinpei0213", times => 10);
    109

    View Slide

  110. ͜ͷؔ਺ͷ
    γάωνϟΛ
    PPRͰղੳ͢Δ
    110

    View Slide

  111. ίʔυͱͯ͠ಡΜͰ͍Ε͹͜͏ग़དྷΔ͚ΕͲ
    my $pinfo = Function::Parameters::info \&Greeter::say_hello;
    my $rinfo = Function::Return::info \&Greeter::say_hello;
    111

    View Slide

  112. ޙʑͷ͜ͱΛߟ͑Δͱ੩తղੳͰऔΓ͍ͨ
    112

    View Slide

  113. ΍ͬͯΈΑ͏ͱ͢Δ͕ͦ΋ͦ΋Perlͷίʔυͱͯ͠͏·͘
    ೝࣝͯ͘͠Εͳ͍
    my $doc_matcher = qr{
    \A (?&PerlDocument) \Z
    $PPR::GRAMMAR
    }x;
    say !!($script =~ $doc_matcher); # => ""(false)
    113

    View Slide

  114. ໰୊͸ keyword plugin
    method say_hello(Str :$name, Int :$times) :Return(Bool) {
    say $self . ": Hello " . $name for 1..$times;
    return 1;
    }
    keyword plugin ͸ perl 5.12.0 ͔Β࢖͑Δจ
    ๏֦ுػೳ
    114

    View Slide

  115. PPR͸ͦ͏͍͏ͷ΋ߟྀग़དྷΔ࡞Γʹͳ͍ͬͯΔ
    •ͦ΋ͦ΋keyword pluginΛ؆୯ʹॻ͚Δ
    Keyword::Declare ͷͨΊʹ࡞ΒΕͨϞ
    δϡʔϧ
    •keywordͷϧʔϧΛ֦ுͰ͖Δ
    115

    View Slide

  116. Function::ParametersʹରԠ͢Δ (1)
    my $FP_GRAMMAR = qr{
    (?(DEFINE)
    (?
    method (?&PerlOWS)
    (?&PerlIdentifier) (?&PerlOWS) # => say_hello
    (?&kw_balanced_parens) (?&PerlOWS) # => (Str :$name, Int :$times)
    (?:
    (?&PerlAttributes) (?&PerlOWS) # => :Return(Bool)
    )?+
    (?&PerlBlock) # => { ... }
    )
    (?
    (?&PerlFunctionParametersMethod)
    )
    (?
    \( (?: [^()]++ | (?&kw_balanced_parens) )*+ \)
    )
    )
    $PPR::GRAMMAR
    }x;
    116

    View Slide

  117. Function::ParametersʹରԠ͢Δ (2)
    qr{
    (?
    method (?&PerlOWS)
    (?&PerlIdentifier) (?&PerlOWS) # => say_hello
    (?&kw_balanced_parens) (?&PerlOWS) # => (Str :$name, Int :$times)
    (?:
    (?&PerlAttributes) (?&PerlOWS) # => :Return(Bool)
    )?+
    (?&PerlBlock) # => { ... }
    )
    }
    117

    View Slide

  118. Function::ParametersʹରԠ͢Δ (3)
    qr{
    (? # <= keyword֦ுͱͯ͠ొ࿥
    (?&PerlFunctionParametersMethod)
    )
    (?
    \( (?: [^()]++ | (?&kw_balanced_parens) )*+ \)
    )
    }
    118

    View Slide

  119. Function::ParametersʹରԠ͢Δ (4)
    my $doc_matcher = qr{
    \A (?&PerlDocument) \Z
    $FP_GRAMMAR
    }x;
    say !!($script =~ $doc_matcher); # => 1
    Perlίʔυͱͯ͠ೝࣝͯ͘͠Εͨʂ ͜ΕͰউ
    ͭΔʂ
    119

    View Slide

  120. DEMO
    120

    View Slide

  121. ܕ͕ݟΕͨ͋ͱͷ࢖͍Έͪ
    •ܕγεςϜΛ࣮૷ͯ͠઴ਐత੩తܕ෇͚
    •Language Server Protocolʹར༻͢Δ
    • ΤσΟλ/IDEͰิ׬΍ϝιου৘ใ͕ར
    ༻Ͱ͖ΔΑ͏ʹͳΔ
    121

    View Slide

  122. PPI౳ͱPPRͷ࢖͍෼͚
    •ίʔυͷ͢΂ͯΛར༻͢ΔͳΒPPI
    •ίʔυͷҰ෦෼Λൈ͖ग़ͨ͠Γ༗Δແ͠Λ
    ݟΔͳΒPPR
    • ਖ਼نදݱͳͷͰϦϑΝΫλϦϯάʹ΋ར
    ༻Մೳ
    122

    View Slide

  123. ·ͱΊ
    •Perl5͸े෼ʹ੩తղੳ͕Ͱ͖Δ͜ͱΛࣔ͠
    ·ͨ͠
    •۩ମతͳ΍Γํ΍࢖͍ಓʹ͍ͭͯ࿩͠·͠
    ͨ
    •ػցͱਓؒͷڞଘʹ͍ͭͯ࿩͠·ͨ͠
    123

    View Slide

  124. Any Questions?
    124

    View Slide

  125. Q: ࣏҆Λྑ͘͢ΔͨΊͷίʔυ͕࣏͕҆ѱ͍
    •A1: ਖ਼نදݱ͸ίʔυͷ࣏͕҆ѱ͘ͳΔҰ
    ͭͷཁૉͰ͕͢ɺίϝϯτ΍ϧʔϧΛ׆༻
    ͠·͠ΐ͏
    125

    View Slide

  126. Q: ࣏҆Λྑ͘͢ΔͨΊͷίʔυ͕࣏͕҆ѱ͍
    my $matcher = qr{
    # ͜͜ʹίϝϯτ͕ॻ͚·͢
    (?(DEFINE)
    (?
    # ͜͜ʹ޷͖ͳύλʔϯΛ࡞ͬͯύλʔϯʹ໊લΛ͚ͭΑ͏!
    )
    )
    }x;
    126

    View Slide

  127. Q: ͳΜͰͦ͜·Ͱͯ͠PerlΛ࢖͏ΜͰ͔͢ (1)
    •A1: 10ສߦҎ্͋ΔPerlίʔυ͕͋Γɺࠓ
    ͪΌΜͱಈ͍͍ͯͯɺͪΌΜͱอक΋͞Ε
    ͍ͯΔ৔߹ʹଞͷݴޠʹஔ͖׵͑Δ͔Ͳ͏
    ͔
    • ଞͷݴޠͷ઴ਐతܕ෇͚ͷ࣮૷͢Δಈػ
    127

    View Slide

  128. Q: ͳΜͰͦ͜·Ͱͯ͠PerlΛ࢖͏ΜͰ͔͢ (2)
    •A2: Perl5͸͔ͳΓջͷਂ͍ݴޠ
    • hackable
    • จ๏͕͔ͳΓࣗ༝ͰɺϞμϯͬΆ͍ॻ͖
    ํ͕Ͱ͖Δ
    • keyword plugin
    128

    View Slide