Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Perlを中心としたワンライナーあれこれ

 Perlを中心としたワンライナーあれこれ

YAPC::Okinawa 2018 ONNASON B会場
(Keynote版に修正しました)

Shigeki SUGAI

March 03, 2018
Tweet

More Decks by Shigeki SUGAI

Other Decks in Technology

Transcript

  1. ࣗݾ঺հ w 4IJHFLJ46("* !UFDLM  w γʔαʔגࣜձࣾ   w

    αʔόαΠυΤϯδχΞ %FW0QT w ࠷ۙ͸ͳ͔ͳ͔ίʔυΛॻ͕࣌ؒ͘ʜ w ΒʔΊΜ໺࿠ w *OHSFTT 3FTJTUBODF "  w ϙέϞϯ(0 5-
  2. ձࣾ঺հ w 4FFTBBϒϩά w 4FFTBBXJLJ w ͨ͠Β͹ܝࣔ൘ w #PPN"QQ(BNFT w

    ଞʹ΋ϒϩάγεςϜͷ0&.ɾεϚϗΞϓϦ։ൃ ͳͲ΍ͬͨΓ͍ͯ͠·͢
  3. &ॲཧ w ɹ w lFzͱಉ༷ʹৼΔ෣͍·͕͢ɺ w VTFGFBUVSFbBMM`΍ɺ w VTFW w

    ͱಉ౳ͷޮՌ͕͋Γ·͢ɻ # say ͕࢖͑ΔΑ͏ʹͳΔʂ $ perl -E 'say "Hello World!"'
  4. M w ೖྗߦΛDIPNQͯ͘͠Εͯɺ·ͨQSJOU࣌ʹվ ߦΛ௥Ճͯ͘͠Ε·͢ɻ # say Ͱ͸ͳ͍ͨΊɺվߦίʔυؚ͕·Εͳ͍ $ perl -e

    'print "Hello World!"' Hello World!$ # -l ͍ͯ͠ΔͷͰɺվߦͯ͘͠ΕΔ $ perl -le 'print "Hello World!"' Hello World! $
  5. B $ head -n 3 KEN_ALL.CSV | nkf -w 01101,"060

    ","0600000","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΠΧχέΠαΠΨφΠόΞΠ","๺ւಓ","ࡳຈࢢதԝ۠","ҎԼʹ ߹",0,0,0,0,0,0 01101,"064 ","0640941","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΞαώΨΦΧ","๺ւಓ","ࡳຈࢢதԝ۠","Ѵέٰ",0,0,1,0,0 01101,"060 ","0600041","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΦΦυΦϦώΨγ","๺ւಓ","ࡳຈࢢதԝ۠","େ௨౦",0,0,1, # ۠੾ΓจࣈΛ "," ʹมߋ͢Δ͜ͱͰCSVʹରԠ $ head -n 3 KEN_ALL.CSV | nkf -w | perl -F, -lane 'print $F[3]' "ϗοΧΠυ΢" "ϗοΧΠυ΢" "ϗοΧΠυ΢" w BXLͬΆ͍࢖͍৺஍ʹͳΓ·͢ɻ w O΍QͱҰॹʹ༻͍ΔͱɺࣗಈͰTQMJUϞʔυʹͳΓɺ഑ྻ!'ʹ஋͕୅ೖ͞Ε·͢ɻ w 'Λ࢖͏ͱ۠੾ΓจࣈΛมߋͰ͖·͢ɻ
  6. .Ϟδϡʔϧ໊ w ίϚϯυϥΠϯ͔Β1FSMϞδϡʔϧΛ࢖༻͢Δ͜ ͱ͕Ͱ͖·͢ɻ w ͭ·ΓͳΜͰ΋ग़དྷ·͢ɻ # LWPͰHTMLΛऔಘͯ͠grep $ perl

    -MLWP::Simple -e 'getprint "http://www.perl.org/"' | grep title <title>The Perl Programming Language - www.perl.org</title> # strftime ͷ %Y ͱ %G Ͱҧ͏ڍಈΛ֬ೝ $ perl -MTime::Piece -E \ 'say Time::Piece->strptime("2014-12-30", "%Y-%m-%d")->strftime("%Y")' 2014 $ perl -MTime::Piece -E \ 'say Time::Piece->strptime("2014-12-30", "%Y-%m-%d")->strftime("%G")' 2015
  7. .0%FQBSTF w #%FQBSTFίΞϞδϡʔϧ w ϫϯϥΠφʔͩͱҰݟ෼͔Γʹ͍͘Α͏ͳίʔυ΋ɺ w .0%FQBSTFΛ࢖͏͜ͱʹΑͬͯίϚϯυϥΠϯͰ͸ͳ͘ɺεΫϦϓτͩ ͬͨΒͲ͏͍͏ίʔυΛ࣮ߦ͍ͯ͠Δ͔Λ֬ೝ͢Δࣄ͕Ͱ͖·͢ɻ # ϫϯϥΠφʔͷίϚϯυϥΠϯΛ΋͠εΫϦϓτͰॻ͍ͨΒͲ͏ͳΔ͔Λ

    -MO=Deparse Ͱදࣔ $ head -n 3 KEN_ALL.CSV | nkf -w | perl -MO=Deparse -F, -lane 'print $F[3]' BEGIN { $/ = "\n"; $\ = "\n"; } LINE: while (defined($_ = readline ARGV)) { chomp $_; our @F = split(/,/, $_, 0); print $F[3]; } -e syntax OK
  8. ·ͣ͸࠷΋Α͋͘Δ΍͔ͭΒ # tsvΛcsvʹஔ׵ $ perl -p -e 's/\t/,/' /tmp/hoge.tsv >

    /tmp/hoge.csv # ର৅σΟϨΫτϦ಺ͷ htmlϑΝΠϧதͷ 2017- Λ 2018- ʹஔ׵ $ find target_dir -type f -name '*.html' | \ xargs perl -pi -e 's/2017-/2018-/g'
  9. ͪΐͬͱͨ͠$47ͷ46.Λू ܭ͍ͨ͠ # ͜ΜͳCSV͕͋Δͱͯ͠ɺ֤ߦͷֹۚͷ߹ܭΛٻΊ͍ͨ $ echo -e "ϥʔϝϯ,690\nຯۄ,100\nᰤࢠ,300" ϥʔϝϯ,690 ຯۄ,100

    ᰤࢠ,300 # ֹۚͷྻΛՃࢉ͠ɺ END {} Λ࢖ͬͯ࠷ޙʹֹۚͷ߹ܭ஋͚ͩग़ྗ $ echo -e "ϥʔϝϯ,690\nຯۄ,100\nᰤࢠ,300" | \ perl -F, -lane '$sum += $F[1]; END {print $sum}' 1090
  10. खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ͜͏͍͏$47͕͋Δͱ͠·͢ # ͜ΕΛٯॱʹ(߱ॱͰ)ग़͍ͨ͠ $ cat /tmp/yapc.csv 2016.12.10,YAPC::Hokkaido

    2016 SAPPORO 2017.3.4,YAPC::Kansai 2017 OSAKA 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2018.3.3,YAPC::Okinawa 2018 ONNASON
  11. खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ΋͠AUBDAίϚϯυ ADBUAͷٯ൛ Λ஌͍ͬͯΕ͹ # Linux $ tac

    /tmp/yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO
  12. खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ͔͠͠ɺUBDίϚϯυ͸-JOVYʹ͔͠ೖ͍ͬͯ·ͤΜɻ w .BD #4%ܥ Ͱ͸ɺ୅ΘΓʹAUBJMSA SFWFSTF ͕͋Γ·͢ɻ

    # Mac(BSDܥ) $ tail -r yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO
  13. खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w -JOVYͰ΋.BDͰ΋ಉ͡ͷΛ࢖͍͍ͨͰ͢ΑͶʂ w ϙʔλϏϦςΟΛߟ͑Δͱɺ΍ͬͺΓ1FSM # ϑΝΠϧͷߦΛٯॱʹग़ྗ $ perl

    -e 'print reverse <>' /tmp/yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO
  14. ΞΫηεݩͷ*1্ҐΛ஌Γ͍ͨ # ௚ۙͷIP্ҐΛ߱ॱʹදࣔ $ tail -500000 access_log | perl -lane

    'print $F[0]' | sort | \ uniq -c | sort -nr | head -3 72802 11.22.33.44 527 222.33.44.55 521 3.4.5.6 # `perl -lane 'print $F[0]'` ͸ɺ # `awk '{print $1}'` Ͱ΋ɺ # `cut -f 1 -d ' '` Ͱ΋OK
  15. ΞΫηεݩͷϢʔβʔΤʔδΣ ϯτ্ҐΛ஌Γ͍ͨ # UserAgent্ҐΛ߱ॱʹιʔτ $ tail -500000 access_log | perl

    -F'"' -lane 'print $F[5]' | \ sort | uniq -c | sort -nr | head -4 756981 Mozilla/5.0 (compatible; EvilBot/3.0; +http://www.example.com/evilbot.html) 7192 Mozilla/5.0 (compatible; Normalbot/2.1; +http://www.example.com/bot.html) 1277 Mozilla/5.0 (compatible; smallbot/2.0; +http://www.example.com/smallbot.htm) 253 Mozilla/5.0 (compatible; minbot/2.0; +http://www.example.com/min.htm)
  16. σϓϩΠޙ͸Τϥʔܥͷϩά ͚ͩʹ஫ࢹ͍ͨ͠ # ΤϥʔܥͷϩάΛࡶʹtail $ tail -F access_log | awk

    '$9 !~ /200|301|302|304/ {print}' # perlͷ৔߹ $ tail -F access_log | perl -lane 'print if $F[8] !~ /200|301|302|304/'
  17. ෼୯ҐʹΞΫηε਺Λूܭ͍ͨ͠ w ಛఆͷ࣌ؒଳ͚ͩେྔʹΫϩʔϧ͕དྷ͍ͯΔ͔Ͳ͏͔͕Θ͔Δ # ෼୯ҐͷϦΫΤετ਺ $ cat access_log | perl

    -lne 'm{2017:(\d+):(\d+)} and print "$1:$2"' | \ sort | uniq -c | head 6559 05:06 5642 05:07 89774 05:08 5592 05:09 4010 05:10
  18. *&ͷ֤8JOEPXTຖͷׂ߹Λ ஌Γ͍ͨ w 6OEFSTUBOEJOHVTFSBHFOUTUSJOHT *OUFSOFU &YQMPSFS c.JDSPTPGU%PDT # IE11ͷUAจࣈྻͰɺWindows NTͷޙΖͷ਺ࣈΛൈ͖ग़͢

    $ head -500000 access_log | egrep 'rv:11\.0' | \ perl -lane '/Windows NT ([\d.]+)/ && print $1' | \ sort | uniq -c | sort -nr | head 11160 6.1 # Windows 7 8006 10.0 # Windows 10 5769 6.3 # Windows 8.1 12 6.2 # Windows 8 1 5.1 # Windows XP
  19. ಛఆͷ6"͕શମʹ઎ΊΔൺ཰ ʢϒϥ΢βγΣΞʣΛ஌Γ͍ͨ # Firefoxͷશମͷൺ཰ $ cat access_log | perl -lane

    '/Firefox/i && $cnt++ ; {print ($cnt / $. ) } ' 0.0734 # IE11ͷൺ཰ $ cat access_log | perl -lane '/rv:11\.0/ && $cnt++ ; {print ($cnt / $. ) } ' 0.065 $. ͷಛघม਺ʢݱࡏͷߦ਺Λද͢ʣΛ࢖͏ͱָͰ͢
  20. େྔ63-͔ΒϦϯΫ੾Ε΍ε ςʔλείʔυΛௐ΂͍ͨ # େྔͷURLͷςΩετ͔ΒɺεςʔλείʔυΛࡶʹνΣοΫ͢Δ $ cat /tmp/url.txt | xargs -t

    -L1 -P1 curl -X GET -I -s | grep HTTP curl -X GET -I -s mojolicious.org HTTP/1.1 200 OK curl -X GET -I -s mojolicious.io HTTP/1.1 301 Moved Permanently curl -X GET -I -s mojocasts.com HTTP/1.1 302 Found curl -X GET -I -s http://example.com/not_found.txt HTTP/1.1 404 Not Found
  21. +40/ΛίϚϯυϥΠϯ͔Β࢖͏ $ echo -e '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' \ | jq { "items":

    [ { "name": "ϥʔϝϯ", "price": 690 }, { "name": "ຯۄ", "price": 100 }, { "name": "ᰤࢠ", "price": 300 } ] }
  22. +40/ΛίϚϯυϥΠϯ͔Β࢖͏ w KRίϚϯυͰඞཁͳཁૉͷΈΛநग़ $ echo -e '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' | \ jq

    -r '.items[].price' 690 100 300 $ echo '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' | \ jq -r '.items[].price' | perl -lane '$sum += $F[0]; END {print $sum}' 1090
  23. αʔό؅ཧ w .BDLFSFMΛ࢖͍ͬͯΔέʔε w $-*πʔϧNLS w IUUQTNBDLFSFMJPKBEPDTFOUSZBEWBODFEDMJ # ϗετҰཡΛϦετ $

    mkr hosts --service yapc --role app | jq -r '.[].name' yapc-app1 yapc-app2 yapc-app3 # ҰׅͰεςʔλε͕poweroff͍ͯ͠Δ΋ͷΛநग़ $ for x in $(seq 1 3) ; do mkr hosts -n yapc-app$x --status poweroff ; done # 1୆ͣͭWEB͔Βୀ໾ͤ͞Δͷ͸େมͳͷͰɺϗετΛҰׅͰୀ໾ͤ͞Δ $ for x in $(seq 1 3) ; do mkr hosts -n yapc-app$x | jq -r '.[].id' | xargs mkr retire ; done
  24. (SBQI2- # ϦϙδτϦ಺ͷOpenதͷissueҰཡΛऔಘ $ curl -s -H "Authorization: bearer $TOKEN"

    -X POST -d \ '{"query":"{ organization(login: \"plack\") { \ repository(name: \"Plack\") { issues(last:3, states:OPEN) { \ edges{ node{ url title } } } } } }"' https://api.github.com/graphql | \ jq -r '.data.organization.repository.issues.edges[].node | .url' https://github.com/plack/Plack/issues/544 https://github.com/plack/Plack/issues/562 https://github.com/plack/Plack/issues/570 # GraphQL Ͱ github organizationʹॴଐ͍ͯ͠ΔϝϯόʔҰཡΛग़ྗ͢Δ $ curl -s -H "Authorization: bearer $TOKEN" -X POST -d \ '{"query":"{ organization(login: \"org_name_hoge\") \ { members(first: 100) { edges { node { login } } } } }","variables":"{}","operationName":null}' https://api.github.com/graphql | \ jq -r '.data.organization.members.edges[].node.login' | sort | head fukuoka hokkaido kansai okinawa
  25. EPDLFS # ఀࢭ͍ͯ͠ΔίϯςφΛ࡟আ docker ps -a | grep [E]xited |

    awk '{print $1}' | xargs -n 1 docker rm # <none> ͷΠϝʔδΛҰׅ࡟আ͍ͨ͠ $ docker images -a | perl -lane '/<none>/ && print $F[2]' | xargs docker rmi
  26. .Z42-ͰεϩʔΫΤϦʔϩάΛ ग़ͨͯ͘͠΋ઃఆมߋग़དྷͳ͍࣌ # ਂԕͳΔཧ༝ʹΑΓslow_query_logΛઃఆग़དྷͳ͍৔߹ͷ࢑ఆાஔ… # whileΛ࢖ͬͯແݶʹॏ͍ΫΤϦΛϑΝΠϧʹग़ྗ $ while ``;do mkdir

    -p /tmp/slow_log_dbxxx/`date +%Y%m%d` ; \ mysql -urepl -hxxx -e 'show full processlist' | grep -v Sleep | \ egrep -v 'Binlog|Connect' | awk '$6 > 2' | grep -v '^Id' | \ sort -nr -k 6 | head -5 2>&1 | \ tee -a /tmp/slow_log_dbxxx/`date +%Y%m%d/%H%M_slow.txt` ;sleep 3;done;
  27. ૉ਺Λ਺͑ͯམ͖͍ͪͭͨ࣌ w ͍ΘΏΔγΣϧܳ w ۀ຿Ͱ໾ཱ͔ͭͲ͏͔͸ஔ͍͓͍ͯͯɺɺ $ perl -e 'print "མͪͭ͘Μͩ…ૉ਺Λ਺͑ͯམͪͭ͘Μͩ…

    "; \ for(1 .. 100){print "$_…" if $_==2 || 2**($_-1) %$_==1}' མͪͭ͘Μͩ…ૉ਺Λ਺͑ͯམͪͭ͘Μͩ… 2…3…5…7…11…13…17…19…23…29…31…37…41…43…47…53…59…61…67…71…73…79…
  28. ʮѨ෦׮ͷϗʔϜϖʔδʯ͔ ΒཁૉΛऔಘ # Ѩ෦׮͞ΜͷϗʔϜϖʔδ͔ΒtitleλάΛऔಘ $ perl -Mojo -E \ 'say

    g("http://abehiroshi.la.coocan.jp/")->dom->at("title")' | nkf -w <title>Ѩ෦׮ͷϗʔϜϖʔδ</title> # Ѩ෦׮͞ΜͷϗʔϜϖʔδ͔Β஀ੜ೔Λऔಘ $ perl -Mojo -E \ 'say g("http://abehiroshi.la.coocan.jp/top.htm")->dom("td")->[4]->text' | nkf -w ੜ೥݄೔ 1964೥6݄22೔
  29. .BD#PPLͷ੔උ඼৘ใΛνΣοΫ͍ͨ͠ # MacBook੔උࡁ੡඼ͷҰཡΛऔಘ $ perl -MData::Dumper -Mojo -C -E \

    '$url = "https://www.apple.com/jp/shop/browse/home/specialdeals/mac/macbook"; \ map { push @$hash, $_->text } g("$url")->dom->find("span[itemprop=\"price\"]")->each; \ g("$url")->dom->find(".specs")->each( sub { my ($e, $num) = @_; \ say join(", ", $hash->[$num-1], $e->text) }) ' ˇ120,800 (੫ผ), 2017೥6݄ൃച 12Πϯνʢର֯ʣLEDόοΫϥΠτσΟεϓϨΠɺ2,304 x 1,440ϐΫηϧղ૾౓ɺ226ppi 8GB 1,866 ΦϯϘʔυϝϞϦ 256GB PCIeϕʔεΦϯϘʔυϑϥογϡετϨʔδ 480p FaceTimeΧϝϥ Intel HD Graphics 615 ˇ120,800 (੫ผ), 2017೥6݄ൃച 12Πϯνʢର֯ʣLEDόοΫϥΠτσΟεϓϨΠɺ2,304 x 1,440ϐΫηϧղ૾౓ɺ226ppi 8GB 1,866 ΦϯϘʔυϝϞϦ 256GB PCIeϕʔεΦϯϘʔυϑϥογϡετϨʔδ 480p FaceTimeΧϝϥ Intel HD Graphics 615
  30. ͪΐͬͱͨ͠όάใࠂ࣌ʹ w 63*ϞδϡʔϧΛ࢖͍ͬͯΔՕॴͰɺ༧ظͤ͵Τϥʔ ͕ग़ͨ৔߹ w ϫϯϥΠφʔͳΒɺͨͬͨߦ͚ͩͰϙΠϯτΛઆ໌Մೳ $ perl -MURI -E

    'say URI->new("http://yapcjapan.org")->host ' yapcjapan.org # ϢʔβଆͰೖΕͨURLͷεΩʔϜ͕ෆ׬શͳͨΊɺdieͯ͠͠·͏έʔε͕ൃੜ $ perl -MURI -E 'say URI->new("ttp://yapcjapan.org")->host ' Can't locate object method "host" via package "URI::_foreign" at -e line 1. # ͜͏͢Ε͹OK $ perl -MURI -E 'my $uri = URI->new("ttp://yapcjapan.org/"); \ $uri->scheme('http') unless $uri->scheme =~ /^https?/; say $uri->host ' yapcjapan.org
  31. ϫϯϥΠφʔΛ࢖͍΍͘͢͢ ΔͨΊͷUJQT w جຊ͸ώετϦΛ$USMSͰྑ͍͕ɺ·ͩ׳Εͯͳ͔ͬͨΓɺ·ͨ࢖͍ͦ͏Ͱ ͋Ε͹εχϖοτπʔϧʹొ࿥͓ͯ͘͠ w ࣗ෼͸Ҏલ͸4IFFUͱ͍͏εχϖοτπʔϧΛ࢖͍ͬͯ·ͨ͠ɻ w IUUQPTDBSEFMCFODPNTIFFU w

    ίϚϯυϥΠϯͷΈͰࡁΉͷͰ͔ͳΓศར w .BDͷํͰ͸ w ͱ͍͏ΫϦοϓϘʔυ֦ுεχϖοτొ࿥͕Ͱ͖ΔπʔϧΛ࢖ͬͨΓ͠ ͍ͯ·͢ɻ w IUUQXXXDMJQNFOVDPNKB