Slide 1

Slide 1 text

1FSMΛத৺ͱͨ͠ w !UFDLM w w :"1$0LJOBXB0//"40/ ϫϯϥΠφʔ͋Ε͜Ε

Slide 2

Slide 2 text

:"1$0LJOBXB 0//"40/

Slide 3

Slide 3 text

ԭೄ࠷ߴͰ͢ʂ

Slide 4

Slide 4 text

:"1$0LJOBXB࠷ߴͰ͢ʂʂ

Slide 5

Slide 5 text

͢Έ·ͤΜɼۓுͯ͠·͆͢

Slide 6

Slide 6 text

ࠓ೔ͷ૝ఆର৅ w ීஈϫϯϥΠφʔΛ͋·Γ࢖͍ͬͯͳ͍ํ w $6*$-* ίϚϯυϥΠϯ ʹෆ׳Εͳํ w 1FSMΛ͋·Γ࢖ͬͯͳ͍ɺ͜Ε͔Β࢝Ί͍ͨํ

Slide 7

Slide 7 text

໨࣍ w ࣗݾ঺հɾձࣾ঺հ w ϫϯϥΠφʔͷྑ͞ w 6/*9ͱ͍͏ߟ͑ํ w 1FSMͷओͳίϚϯυϥΠϯ w ࣮ࡍͷ࣮ྫू w ʜͳͲΛബ͘঺հ͠·͢

Slide 8

Slide 8 text

ࣗݾ঺հ w 4IJHFLJ46("* !UFDLM w γʔαʔגࣜձࣾ w αʔόαΠυΤϯδχΞ %FW0QT w ࠷ۙ͸ͳ͔ͳ͔ίʔυΛॻ͕࣌ؒ͘ʜ w ΒʔΊΜ໺࿠ w *OHSFTT 3FTJTUBODF " w ϙέϞϯ(0 5-

Slide 9

Slide 9 text

ձࣾ঺հ γʔαʔגࣜձࣾ

Slide 10

Slide 10 text

ձࣾ঺հ w 4FFTBBϒϩά w 4FFTBBXJLJ w ͨ͠Β͹ܝࣔ൘ w #PPN"QQ(BNFT w ଞʹ΋ϒϩάγεςϜͷ0&.ɾεϚϗΞϓϦ։ൃ ͳͲ΍ͬͨΓ͍ͯ͠·͢

Slide 11

Slide 11 text

΄ͱΜͲ1FSMͰಈ͍ͯ· ͢ʂʂʂʂʂ̍

Slide 12

Slide 12 text

ձࣾ঺հ w ࡢ೥͸!YUFUTVKJ͞Μʹ1FSMೖֶࣜاۀग़ு ൛ JOTFFTBB Λճ΋։࠵͍͖ͯͨͩ͠·͠ ͨʂ w IUUQQPTUUFUTVKJKQQFSM FOUSBODFTFSWJDFGPSDPNQBOZ w ͜ͷ#ձ৔ͷ࣍ͷηογϣϯͳͷͰָ͓͠Έ ʹʂʂ

Slide 13

Slide 13 text

ʮγʔαʔʯ͍ͬͯ͏͔Βʹ ͸ɺ౰વԭೄʹؔ܎͕ʁ

Slide 14

Slide 14 text

ޙͰݸਓతʹฉ͍͍ͯͩ͘͞ʜ

Slide 15

Slide 15 text

ຊ୊

Slide 16

Slide 16 text

ϫϯϥΠφʔͱ͸ w ʮϫϯϥΠφʔʯͱ͸ɺͨͬͨߦ͚ͩͷϓϩά ϥϜͰ͢ɻ1FSM΍3VCZΛ࢖͏ͱɺߦ͚ͩͰ΋͔ͳ Γͷ࢓ࣄ͕Ͱ͖·͢ɻ w ϓϩάϥϜΛॻ͖͍͕ͨɺΤσΟλΛىಈ͢Δ΄ Ͳͷखؒ͸͔͚ͨ͘ͳ͍ɺͱ͍͏ͱ͖ʹҖྗΛൃش ͠·͢ɻ w IUUQMDMJOVYPSKQMDGBMMTT EFUBJMIUNMQFSMSVCZ

Slide 17

Slide 17 text

೔ຊޠͰݴ͏ͱ͜Ζͷɺ Ұߦ໺࿠

Slide 18

Slide 18 text

ϫϯϥΠφʔͷϝϦοτ

Slide 19

Slide 19 text

ϫϯϥΠφʔͷϝϦοτ w ͱʹ͔͘࢓ࣄ͕ૣ͍ʂ w ͍͍ͪͪΤσΟλΛ্ཱͪ͛Δඞཁ͢Βͳ͍ w ಛఆͷϓϩάϥϜݴޠ஌ࣝʹؔΘΒͳ͍ w ίϚϯυϥΠϯ͔Β࢖͑ΔύλʔϯΛ֮͑Ε͹֮͑Δ΄ ͲɺԠ༻ൣғ͕޿͕Δ w ΤϯδχΞͱͯ͠΍͍ͬͯ͘ͳΒɺࠓޙ΋ͣͬͱ࢖͑Δ ஌ࣝʹͳΔ

Slide 20

Slide 20 text

ϫϯϥΠφʔΛ ͲΜͳ࣌ʹ࢖͏ͷʁ

Slide 21

Slide 21 text

ϫϯϥΠφʔΛͲΜͳ࣌ʹ࢖ ͏ͷʁ w αʔόӡ༻ɾো֐ൃੜ࣌ͷϩάղੳ w ୯७ͳόάͷใࠂ࣌ w ଞਓʹઆ໌͢Δͱ͖ʹ΋ߦͰઆ໌͕෇͘ w ϓϧϦΫͳͲͷόάใࠂʹ͸ɺ࠷΋୹͘࠷ খݶͰ࠶ݱͰ͖ΔίʔυΛݟͤΔͷ͕࠷΋ ૣ͍

Slide 22

Slide 22 text

ϫϯϥΠφʔΛͲΜͳ࣌ʹ࢖ ͏ͷʁ w εΫϦϓτΛॻ͘·Ͱ΋ͳ͍ௐࠪґཔܥͷ࡞ۀ w େྔͷσʔλ͔Βಛఆͷσʔλ͚ͩΛநग़͠ ͍ͨ w ߦ͚ͩͳͷͰɺ࢖͑ΔཁૉΛվมͯ͠खݩͰ खܰʹࢼͤΔ w γΣϧܳ

Slide 23

Slide 23 text

ʜͯ͞ɺϫϯϥΠφʔͷ࿩ͷ ్தͰ͕͢

Slide 24

Slide 24 text

6/*9ͱ͍͏ߟ͑ํͦͷઃܭ ࢥ૝ͱ఩ֶ w ͱ͍͏ຊΛ͝ଘ஌Ͱ͠ΐ͏͔ʁ

Slide 25

Slide 25 text

6/*9ͱ͍͏ߟ͑ํ w ൃץ͔Β೥Ҏ্ܦա͍ͯͯ͠ݹయతͳײ͡ʹͳ ͍ͬͯ·͕͢ɺࠓ͔ΒͰ΋े෼ಡΉՁ஋͕͋Δ໊ ஶͰ͢ w ٕज़ॻͱ͍͏ΑΓ఩ֶॻ w ϓϩάϥϛϯάΛֶͼ࢝ΊͨࠒʹಡΜͰ͓͚ ͹ɻɻ w 6/*9ͱ͍͏ߟ͑ํʕͦͷઃܭࢥ૝ͱ఩ֶ

Slide 26

Slide 26 text

6/*9ͱ͍͏ߟ͑ํ w ೥͔Βɺશ͘಺༰͕มΘΒͣࠓ΋ചΕଓ͚ ͍ͯ·͢ ೔ຊޠ൛ɺݪஶ͸೥ࠒ

Slide 27

Slide 27 text

ͭͷఆཧ w εϞʔϧŋΠζŋϏϡʔςΟϑϧ খ͍͞΋ͷ͸ඒ͠ ͍ w ҰͭͷϓϩάϥϜʹ͸Ұͭͷ͜ͱΛ͏·͘΍ΒͤΔ w Ͱ͖Δ͚ͩૣ͘ࢼ࡞͢Δ w ޮ཰ΑΓҠ২ੑΛ༏ઌ͢Δ w ਺஋σʔλ͸"4$**ϑϥοτϑΝΠϧʹอଘ͢Δ

Slide 28

Slide 28 text

ͭͷఆཧ w ιϑτ΢ΣΞΛᑏࢠʢςίʣͱͯ͠࢖͏ w γΣϧεΫϦϓτʹΑͬͯᑏࢠͷޮՌͱҠ২ੑ ΛߴΊΔ w ա౓ͷର࿩తΠϯλϑΣʔεΛආ͚Δ w ͢΂ͯͷϓϩάϥϜΛϑΟϧλͱͯ͠ઃܭ͢Δ

Slide 29

Slide 29 text

6/*9ͱ͍͏ߟ͑ํ w ҰͭͷϓϩάϥϜͰͳΜͰ΋ग़དྷΔΑ͏ʹػೳΛ௥Ճ͍ͯ͠ ͘ͷͰ͸ͳ͘ɺ w ͨͬͨҰͭͷࣄ͔͠Ͱ͖ͳ͍খ͞ͳϓϩάϥϜɾίϚϯυΛ ͏·͘૊Έ߹ΘͤͯςίͷޮՌΛग़ͨ͠΄͏͕ɺང͔ʹޮ཰ ྑ͘໨తΛୡ੒Ͱ͖ΔΑ͏ʹͳΔ w ύΠϓΛհ͢Δ͜ͱʹΑͬͯɺখ͞ͳϓϩάϥϜಉ͕࢜͢΂ ͯ૬ޓʹ࿈ܞͰ͖Δ w ૄ݁߹Λྑ͠ͱ͢ΔϚΠΫϩαʔϏεతͳߟ͑ํͱ΋௨͡Δ

Slide 30

Slide 30 text

6/*9ͱ͍͏ߟ͑ํ w ௒ͬ͘͟Γ͍͏ͱɺ w େن໛ͳϓϩάϥϜ΍ಛఆͷόΠφϦͷΑ͏ͳ σʔλΑΓ΋ɺϓϨʔϯςΩετͰɺখ͍͞ϓϩ άϥϜ܈Λ͏·͘૊Έ߹ΘͤΔ͜ͱʹΑͬͯΏ Δ͘࿈ܞͤ͞·͠ΐ͏ɺͱ͍͏ߟ͑ํ

Slide 31

Slide 31 text

6/*9ͷ఩ֶΛݟͨͱ͜ΖͰʜ w ϫϯϥΠφʔͷ࿩ʹ໭Γ·͠ΐ͏

Slide 32

Slide 32 text

ԿނϫϯϥΠφʔͰ 1FSMͳͷ͔ʁ

Slide 33

Slide 33 text

ԿނϫϯϥΠφʔͰ1FSMͳͷ͔ʁ w ߴ͍Ҡ২ੑ w 1FSM͸ޓ׵ੑ͕ߴ͍ w BXLͱ͔ͩͱ04ʹΑͬͯ͸ඍົͳҧ͍͕͋ͬͨΓɺ w γΣϧ΋#PVSOF4IFMMܥͱ$4IFMMܥʹΑͬͯ΋ඍົʹҧ ͍͕͋ͬͨΓ͢Δ w 1FSM͸େ఍ͷ04ʹೖ͍ͬͯΔ w ڧྗͳਖ਼نදݱ

Slide 34

Slide 34 text

ԿނϫϯϥΠφʔͰ1FSMͳͷ͔ʁ w 1FSMʹݶΒͣɺ"XLͰ΋TFEͰ΋3VCZͰ΋ɺ ࣗ෼ͷ஌͍ͬͯΔൣғͰࣗ༝ʹ࢖͑͹े෼ w ΍Γํ͸ͻͱͭ͡Όͳ͍ w Ͱ͕͢ɺ΍ͬͺΓ1FSMΛ஌͍ͬͯΔͱָʹͳΓ ·͢Αɺͱ͍͏ྫΛ঺հ͠·͢

Slide 35

Slide 35 text

1FSMͷओͳίϚϯυϥΠϯΦ ϓγϣϯʢεΠονʣ w ʜ͞Βͬͱྲྀ͠·͢

Slide 36

Slide 36 text

F w Fॲཧɹ w ͢΂ͯ͸͔͜͜Βɻ w ޙଓͷॲཧΛίϚϯυϥΠϯͷϓϩάϥϜͱ͠ ࣮ͯߦ͠·͢ɻ $ perl -e 'print "Hello World!"'

Slide 37

Slide 37 text

&ॲཧ w ɹ w lFzͱಉ༷ʹৼΔ෣͍·͕͢ɺ w VTFGFBUVSFbBMM`΍ɺ w VTFW w ͱಉ౳ͷޮՌ͕͋Γ·͢ɻ # say ͕࢖͑ΔΑ͏ʹͳΔʂ $ perl -E 'say "Hello World!"'

Slide 38

Slide 38 text

M w ೖྗߦΛDIPNQͯ͘͠Εͯɺ·ͨQSJOU࣌ʹվ ߦΛ௥Ճͯ͘͠Ε·͢ɻ # say Ͱ͸ͳ͍ͨΊɺվߦίʔυؚ͕·Εͳ͍ $ perl -e 'print "Hello World!"' Hello World!$ # -l ͍ͯ͠ΔͷͰɺվߦͯ͘͠ΕΔ $ perl -le 'print "Hello World!"' Hello World! $

Slide 39

Slide 39 text

O w ɹೖྗσʔλͷ֤ߦΛॲཧ͠·͢ɻ while (<>) { ॲཧ; }

Slide 40

Slide 40 text

B $ head -n 3 KEN_ALL.CSV | nkf -w 01101,"060 ","0600000","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΠΧχέΠαΠΨφΠόΞΠ","๺ւಓ","ࡳຈࢢதԝ۠","ҎԼʹ ߹",0,0,0,0,0,0 01101,"064 ","0640941","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΞαώΨΦΧ","๺ւಓ","ࡳຈࢢதԝ۠","Ѵέٰ",0,0,1,0,0 01101,"060 ","0600041","ϗοΧΠυ΢","αοϙϩγνϡ΢Φ΢Ϋ","ΦΦυΦϦώΨγ","๺ւಓ","ࡳຈࢢதԝ۠","େ௨౦",0,0,1, # ۠੾ΓจࣈΛ "," ʹมߋ͢Δ͜ͱͰCSVʹରԠ $ head -n 3 KEN_ALL.CSV | nkf -w | perl -F, -lane 'print $F[3]' "ϗοΧΠυ΢" "ϗοΧΠυ΢" "ϗοΧΠυ΢" w BXLͬΆ͍࢖͍৺஍ʹͳΓ·͢ɻ w O΍QͱҰॹʹ༻͍ΔͱɺࣗಈͰTQMJUϞʔυʹͳΓɺ഑ྻ!'ʹ஋͕୅ೖ͞Ε·͢ɻ w 'Λ࢖͏ͱ۠੾ΓจࣈΛมߋͰ͖·͢ɻ

Slide 41

Slide 41 text

.Ϟδϡʔϧ໊ w ίϚϯυϥΠϯ͔Β1FSMϞδϡʔϧΛ࢖༻͢Δ͜ ͱ͕Ͱ͖·͢ɻ w ͭ·ΓͳΜͰ΋ग़དྷ·͢ɻ # LWPͰHTMLΛऔಘͯ͠grep $ perl -MLWP::Simple -e 'getprint "http://www.perl.org/"' | grep title The Perl Programming Language - www.perl.org # strftime ͷ %Y ͱ %G Ͱҧ͏ڍಈΛ֬ೝ $ perl -MTime::Piece -E \ 'say Time::Piece->strptime("2014-12-30", "%Y-%m-%d")->strftime("%Y")' 2014 $ perl -MTime::Piece -E \ 'say Time::Piece->strptime("2014-12-30", "%Y-%m-%d")->strftime("%G")' 2015

Slide 42

Slide 42 text

.0%FQBSTF w #%FQBSTFίΞϞδϡʔϧ w ϫϯϥΠφʔͩͱҰݟ෼͔Γʹ͍͘Α͏ͳίʔυ΋ɺ w .0%FQBSTFΛ࢖͏͜ͱʹΑͬͯίϚϯυϥΠϯͰ͸ͳ͘ɺεΫϦϓτͩ ͬͨΒͲ͏͍͏ίʔυΛ࣮ߦ͍ͯ͠Δ͔Λ֬ೝ͢Δࣄ͕Ͱ͖·͢ɻ # ϫϯϥΠφʔͷίϚϯυϥΠϯΛ΋͠εΫϦϓτͰॻ͍ͨΒͲ͏ͳΔ͔Λ -MO=Deparse Ͱදࣔ $ head -n 3 KEN_ALL.CSV | nkf -w | perl -MO=Deparse -F, -lane 'print $F[3]' BEGIN { $/ = "\n"; $\ = "\n"; } LINE: while (defined($_ = readline ARGV)) { chomp $_; our @F = split(/,/, $_, 0); print $F[3]; } -e syntax OK

Slide 43

Slide 43 text

ϫϯϥΠφʔͰ໾ʹཱͭ1FSMͷ ಛघม਺ w ɿݱࡏͷߦ൪߸ w ɿQSJOUҾ਺ͷηύϨʔλ w ɿೖྗʢPQFOʣ࣌ͷߦ۠੾Γจࣈ w aɿग़ྗʢQSJOUʣ࣌ͷߦ۠੾Γจࣈ w ɿΤϥʔͷ಺༰

Slide 44

Slide 44 text

΋ͬͱৄ͘͠ w QFSMίϚϯυϥΠϯʹ͍ͭͯ΋ͬͱ஌Γ͍ͨਓ͸ w QFSMEPDQFSMSVO w ΋͘͠͸೔ຊޠ༁΋͋Γ·͢ w IUUQQFSMEPDKQEPDTQFSM QFSMSVOQPE

Slide 45

Slide 45 text

ϫϯϥΠφʔͷࣄྫ঺հ

Slide 46

Slide 46 text

ಛఆจࣈྻΛஔ׵͍ͨ͠

Slide 47

Slide 47 text

·ͣ͸࠷΋Α͋͘Δ΍͔ͭΒ # tsvΛcsvʹஔ׵ $ perl -p -e 's/\t/,/' /tmp/hoge.tsv > /tmp/hoge.csv # ର৅σΟϨΫτϦ಺ͷ htmlϑΝΠϧதͷ 2017- Λ 2018- ʹஔ׵ $ find target_dir -type f -name '*.html' | \ xargs perl -pi -e 's/2017-/2018-/g'

Slide 48

Slide 48 text

ͪΐͬͱͨ͠$47ͷ 46.Λूܭ͍ͨ͠

Slide 49

Slide 49 text

ͪΐͬͱͨ͠$47ͷ46.Λू ܭ͍ͨ͠ # ͜ΜͳCSV͕͋Δͱͯ͠ɺ֤ߦͷֹۚͷ߹ܭΛٻΊ͍ͨ $ echo -e "ϥʔϝϯ,690\nຯۄ,100\nᰤࢠ,300" ϥʔϝϯ,690 ຯۄ,100 ᰤࢠ,300 # ֹۚͷྻΛՃࢉ͠ɺ END {} Λ࢖ͬͯ࠷ޙʹֹۚͷ߹ܭ஋͚ͩग़ྗ $ echo -e "ϥʔϝϯ,690\nຯۄ,100\nᰤࢠ,300" | \ perl -F, -lane '$sum += $F[1]; END {print $sum}' 1090

Slide 50

Slide 50 text

खݩͷϑΝΠϧΛ ٯॱʹग़ྗ͍ͨ͠

Slide 51

Slide 51 text

खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ͜͏͍͏$47͕͋Δͱ͠·͢ # ͜ΕΛٯॱʹ(߱ॱͰ)ग़͍ͨ͠ $ cat /tmp/yapc.csv 2016.12.10,YAPC::Hokkaido 2016 SAPPORO 2017.3.4,YAPC::Kansai 2017 OSAKA 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2018.3.3,YAPC::Okinawa 2018 ONNASON

Slide 52

Slide 52 text

खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ΋͠AUBDAίϚϯυ ADBUAͷٯ൛ Λ஌͍ͬͯΕ͹ # Linux $ tac /tmp/yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO

Slide 53

Slide 53 text

खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w ͔͠͠ɺUBDίϚϯυ͸-JOVYʹ͔͠ೖ͍ͬͯ·ͤΜɻ w .BD #4%ܥ Ͱ͸ɺ୅ΘΓʹAUBJMSA SFWFSTF ͕͋Γ·͢ɻ # Mac(BSDܥ) $ tail -r yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO

Slide 54

Slide 54 text

खݩͷϑΝΠϧΛٯॱʹग़ྗ ͍ͨ͠ w -JOVYͰ΋.BDͰ΋ಉ͡ͷΛ࢖͍͍ͨͰ͢ΑͶʂ w ϙʔλϏϦςΟΛߟ͑Δͱɺ΍ͬͺΓ1FSM # ϑΝΠϧͷߦΛٯॱʹग़ྗ $ perl -e 'print reverse <>' /tmp/yapc.csv 2018.3.3,YAPC::Okinawa 2018 ONNASON 2017.7.1,YAPC::Fukuoka 2017 HAKATA 2017.3.4,YAPC::Kansai 2017 OSAKA 2016.12.10,YAPC::Hokkaido 2016 SAPPORO

Slide 55

Slide 55 text

ΞΫηεϩάௐࠪܥ

Slide 56

Slide 56 text

αʔϏε͕ٸʹॏ͘ͳͬͨ࣌ ʢো֐࣌ͷରԠʣ w NBDLFSFMͳͲͷ؂ࢹπʔϧͷάϥϑͳͲΛݟΔ ͱͱ΋ʹɺΞϥʔτ͕ಧ͍ͨॠؒʹ·ͣ͸ϩά ΛݟΔ w ͕͜͜ϫϯϥΠφʔͷ࠷΋ಘҙͳͱ͜Ζ w ࣍ϖʔδҎ߱͸$PNCJOFE-PHϑΥʔϚοτ ͷ૝ఆͱ͠·͢

Slide 57

Slide 57 text

ΞΫηεݩͷ*1্ҐΛ஌Γ͍ͨ # ௚ۙͷIP্ҐΛ߱ॱʹදࣔ $ tail -500000 access_log | perl -lane 'print $F[0]' | sort | \ uniq -c | sort -nr | head -3 72802 11.22.33.44 527 222.33.44.55 521 3.4.5.6 # `perl -lane 'print $F[0]'` ͸ɺ # `awk '{print $1}'` Ͱ΋ɺ # `cut -f 1 -d ' '` Ͱ΋OK

Slide 58

Slide 58 text

ΞΫηεݩͷϢʔβʔΤʔδΣ ϯτ্ҐΛ஌Γ͍ͨ

Slide 59

Slide 59 text

ΞΫηεݩͷϢʔβʔΤʔδΣ ϯτ্ҐΛ஌Γ͍ͨ # UserAgent্ҐΛ߱ॱʹιʔτ $ tail -500000 access_log | perl -F'"' -lane 'print $F[5]' | \ sort | uniq -c | sort -nr | head -4 756981 Mozilla/5.0 (compatible; EvilBot/3.0; +http://www.example.com/evilbot.html) 7192 Mozilla/5.0 (compatible; Normalbot/2.1; +http://www.example.com/bot.html) 1277 Mozilla/5.0 (compatible; smallbot/2.0; +http://www.example.com/smallbot.htm) 253 Mozilla/5.0 (compatible; minbot/2.0; +http://www.example.com/min.htm)

Slide 60

Slide 60 text

Τϥʔܥͷո͍͠εςʔλε ίʔυͷϩάΛ؂ࢹ͍ͨ͠

Slide 61

Slide 61 text

σϓϩΠޙ͸Τϥʔܥͷϩά ͚ͩʹ஫ࢹ͍ͨ͠ # ΤϥʔܥͷϩάΛࡶʹtail $ tail -F access_log | awk '$9 !~ /200|301|302|304/ {print}' # perlͷ৔߹ $ tail -F access_log | perl -lane 'print if $F[8] !~ /200|301|302|304/'

Slide 62

Slide 62 text

෼୯ҐʹΞΫηε਺ Λूܭ͍ͨ͠

Slide 63

Slide 63 text

෼୯ҐʹΞΫηε਺Λूܭ͍ͨ͠ w ಛఆͷ࣌ؒଳ͚ͩେྔʹΫϩʔϧ͕དྷ͍ͯΔ͔Ͳ͏͔͕Θ͔Δ # ෼୯ҐͷϦΫΤετ਺ $ cat access_log | perl -lne 'm{2017:(\d+):(\d+)} and print "$1:$2"' | \ sort | uniq -c | head 6559 05:06 5642 05:07 89774 05:08 5592 05:09 4010 05:10

Slide 64

Slide 64 text

*&ͷ֤8JOEPXTຖͷ ׂ߹Λ஌Γ͍ͨ

Slide 65

Slide 65 text

*&ͷ֤8JOEPXTຖͷׂ߹Λ ஌Γ͍ͨ w 6OEFSTUBOEJOHVTFSBHFOUTUSJOHT *OUFSOFU &YQMPSFS c.JDSPTPGU%PDT # IE11ͷUAจࣈྻͰɺWindows NTͷޙΖͷ਺ࣈΛൈ͖ग़͢ $ head -500000 access_log | egrep 'rv:11\.0' | \ perl -lane '/Windows NT ([\d.]+)/ && print $1' | \ sort | uniq -c | sort -nr | head 11160 6.1 # Windows 7 8006 10.0 # Windows 10 5769 6.3 # Windows 8.1 12 6.2 # Windows 8 1 5.1 # Windows XP

Slide 66

Slide 66 text

ಛఆͷ6"͕શମʹ઎ΊΔൺ཰ ʢϒϥ΢βγΣΞʣΛ஌Γ͍ͨ

Slide 67

Slide 67 text

ಛఆͷ6"͕શମʹ઎ΊΔൺ཰ ʢϒϥ΢βγΣΞʣΛ஌Γ͍ͨ # Firefoxͷશମͷൺ཰ $ cat access_log | perl -lane '/Firefox/i && $cnt++ ; {print ($cnt / $. ) } ' 0.0734 # IE11ͷൺ཰ $ cat access_log | perl -lane '/rv:11\.0/ && $cnt++ ; {print ($cnt / $. ) } ' 0.065 $. ͷಛघม਺ʢݱࡏͷߦ਺Λද͢ʣΛ࢖͏ͱָͰ͢

Slide 68

Slide 68 text

ো֐࣌ͷϩάௐࠪʹ͍ͭͯͷิ଍ w ো֐࣌ͷରԠʹ͍ͭͯɺͦ΋ͦ΋؂ࢹ߲໨Λࣄલʹઃఆ͓ͯ͠ ͘͜ͱ͕ॏཁ w ো֐൑໌ޙʹ·ͣͲ͔͜Βಈ͘ͷ͔ w CPUͳͲͷ֎తཁҼ͔ɺͦΕͱ΋಺తཁҼ͔ w ໰୊੾Γ෼͚࣌ͷॳಈௐࠪ࣌Ͱ͸ੜϩάௐࠪͷεϐʔυ ͕࠷΋ॏཁͱͳΔ w ϫϯϥΠφʔͰͷϩάௐࠪ͸ɺ͋͘·Ͱ΋؂ࢹπʔϧͷิ׬ͱ ͯ͠ɺ྆ํॏཁͰ͢

Slide 69

Slide 69 text

ͦͷଞࡶଟͳϫϯϥΠφʔ

Slide 70

Slide 70 text

େྔ63-͔ΒϦϯΫ੾Ε΍ε ςʔλείʔυΛௐ΂͍ͨ

Slide 71

Slide 71 text

େྔ63-͔ΒϦϯΫ੾Ε΍ε ςʔλείʔυΛௐ΂͍ͨ # େྔͷURLͷςΩετ͔ΒɺεςʔλείʔυΛࡶʹνΣοΫ͢Δ $ cat /tmp/url.txt | xargs -t -L1 -P1 curl -X GET -I -s | grep HTTP curl -X GET -I -s mojolicious.org HTTP/1.1 200 OK curl -X GET -I -s mojolicious.io HTTP/1.1 301 Moved Permanently curl -X GET -I -s mojocasts.com HTTP/1.1 302 Found curl -X GET -I -s http://example.com/not_found.txt HTTP/1.1 404 Not Found

Slide 72

Slide 72 text

େྔͷϑΝΠϧ͕ೖͬͨσΟϨ ΫτϦΛঃʑʹ࡟আ͢Δ

Slide 73

Slide 73 text

େྔͷϑΝΠϧ͕ೖͬͨσΟϨ ΫτϦΛঃʑʹ࡟আ͢Δ w Ұ౓ʹ௚Լ͔Β࡟আͯ͠͠·͏ͱ*0ෛՙ͕͔͔ΔͷͰʜ w ͳ͓ɺYBSHT͔Βͷ࡟আʹ͍ͭͯ͸ɺQFSMΛ࢖ͬͨํ͕ޮ཰͕ྑ͍͜ͱ͕ଟ͍ͱ஄͞Μͷϒ ϩάʹৄ͍͠આ໌͕͋Γ·͢ɻ w IUUQCMPHMJWFEPPSKQEBOLPHBJBSDIJWFTIUNM # σΟϨΫτϦΛ2֊૚෼ԼΓͯগͣͭ͠࡟আ͍ͯ͘͠ $ time find dir/ -type d -mtime +100 -mindepth 2 -maxdepth 2 | \ xargs -r -n 1 -t rm -rf

Slide 74

Slide 74 text

+40/ΛίϚϯυϥΠϯ͔Β࢖͏

Slide 75

Slide 75 text

+40/ΛίϚϯυϥΠϯ͔Β࢖͏ $ echo -e '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' \ | jq { "items": [ { "name": "ϥʔϝϯ", "price": 690 }, { "name": "ຯۄ", "price": 100 }, { "name": "ᰤࢠ", "price": 300 } ] }

Slide 76

Slide 76 text

+40/ΛίϚϯυϥΠϯ͔Β࢖͏ w KRίϚϯυͰඞཁͳཁૉͷΈΛநग़ $ echo -e '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' | \ jq -r '.items[].price' 690 100 300 $ echo '{"items":[{"name":"ϥʔϝϯ","price":690},{"name":"ຯۄ","price":100},{"name":"ᰤࢠ","price":300}]}' | \ jq -r '.items[].price' | perl -lane '$sum += $F[0]; END {print $sum}' 1090

Slide 77

Slide 77 text

αʔό؅ཧ

Slide 78

Slide 78 text

αʔό؅ཧ w .BDLFSFMΛ࢖͍ͬͯΔέʔε w $-*πʔϧNLS w IUUQTNBDLFSFMJPKBEPDTFOUSZBEWBODFEDMJ # ϗετҰཡΛϦετ $ mkr hosts --service yapc --role app | jq -r '.[].name' yapc-app1 yapc-app2 yapc-app3 # ҰׅͰεςʔλε͕poweroff͍ͯ͠Δ΋ͷΛநग़ $ for x in $(seq 1 3) ; do mkr hosts -n yapc-app$x --status poweroff ; done # 1୆ͣͭWEB͔Βୀ໾ͤ͞Δͷ͸େมͳͷͰɺϗετΛҰׅͰୀ໾ͤ͞Δ $ for x in $(seq 1 3) ; do mkr hosts -n yapc-app$x | jq -r '.[].id' | xargs mkr retire ; done

Slide 79

Slide 79 text

(SBQI2-

Slide 80

Slide 80 text

(SBQI2- # ϦϙδτϦ಺ͷOpenதͷissueҰཡΛऔಘ $ curl -s -H "Authorization: bearer $TOKEN" -X POST -d \ '{"query":"{ organization(login: \"plack\") { \ repository(name: \"Plack\") { issues(last:3, states:OPEN) { \ edges{ node{ url title } } } } } }"' https://api.github.com/graphql | \ jq -r '.data.organization.repository.issues.edges[].node | .url' https://github.com/plack/Plack/issues/544 https://github.com/plack/Plack/issues/562 https://github.com/plack/Plack/issues/570 # GraphQL Ͱ github organizationʹॴଐ͍ͯ͠ΔϝϯόʔҰཡΛग़ྗ͢Δ $ curl -s -H "Authorization: bearer $TOKEN" -X POST -d \ '{"query":"{ organization(login: \"org_name_hoge\") \ { members(first: 100) { edges { node { login } } } } }","variables":"{}","operationName":null}' https://api.github.com/graphql | \ jq -r '.data.organization.members.edges[].node.login' | sort | head fukuoka hokkaido kansai okinawa

Slide 81

Slide 81 text

EPDLFS

Slide 82

Slide 82 text

EPDLFS # ఀࢭ͍ͯ͠ΔίϯςφΛ࡟আ docker ps -a | grep [E]xited | awk '{print $1}' | xargs -n 1 docker rm # ͷΠϝʔδΛҰׅ࡟আ͍ͨ͠ $ docker images -a | perl -lane '// && print $F[2]' | xargs docker rmi

Slide 83

Slide 83 text

.Z42-ͰεϩʔΫΤϦʔϩάΛ ग़ͨͯ͘͠΋ઃఆมߋग़དྷͳ͍࣌

Slide 84

Slide 84 text

.Z42-ͰεϩʔΫΤϦʔϩάΛ ग़ͨͯ͘͠΋ઃఆมߋग़དྷͳ͍࣌ # ਂԕͳΔཧ༝ʹΑΓslow_query_logΛઃఆग़དྷͳ͍৔߹ͷ࢑ఆાஔ… # whileΛ࢖ͬͯແݶʹॏ͍ΫΤϦΛϑΝΠϧʹग़ྗ $ while ``;do mkdir -p /tmp/slow_log_dbxxx/`date +%Y%m%d` ; \ mysql -urepl -hxxx -e 'show full processlist' | grep -v Sleep | \ egrep -v 'Binlog|Connect' | awk '$6 > 2' | grep -v '^Id' | \ sort -nr -k 6 | head -5 2>&1 | \ tee -a /tmp/slow_log_dbxxx/`date +%Y%m%d/%H%M_slow.txt` ;sleep 3;done;

Slide 85

Slide 85 text

ૉ਺Λ਺͑ͯམ͖͍ͪͭͨ࣌

Slide 86

Slide 86 text

ૉ਺Λ਺͑ͯམ͖͍ͪͭͨ࣌ w ͍ΘΏΔγΣϧܳ w ۀ຿Ͱ໾ཱ͔ͭͲ͏͔͸ஔ͍͓͍ͯͯɺɺ $ perl -e 'print "མͪͭ͘Μͩ…ૉ਺Λ਺͑ͯམͪͭ͘Μͩ… "; \ for(1 .. 100){print "$_…" if $_==2 || 2**($_-1) %$_==1}' མͪͭ͘Μͩ…ૉ਺Λ਺͑ͯམͪͭ͘Μͩ… 2…3…5…7…11…13…17…19…23…29…31…37…41…43…47…53…59…61…67…71…73…79…

Slide 87

Slide 87 text

खܰʹεΫϨΠϐϯά

Slide 88

Slide 88 text

खܰʹεΫϨΠϐϯά w εΫϨΠϐϯάͱ͍͑͹8FC4DSBQFSͰ͕͢ɺ ϫϯϥΠφʔͰ΍Δʹ͸ͪΐͬͱ͖͍ͭʜ w ɹͦ͜Ͱɺ1FSMͰ༗໊ͳ8FCΞϓϦέʔγϣϯϑϨ ʔϜϫʔΫͷ.PKPMJDJPVT͕͋Γ·͕͢ɺͦ͜ʹؚ ·Ε͍ͯΔϞδϡʔϧʹPKPQNͱ͍͏ͷ͕͋Γ· ͢ w ϞδϡʔϧεΠονͷA.Aͱηοτʹ͢Δͱɺ .PKPʹͳΓ·͢ɻ

Slide 89

Slide 89 text

ʮѨ෦׮ͷϗʔϜϖʔδʯ͔ ΒཁૉΛऔಘ

Slide 90

Slide 90 text

ʮѨ෦׮ͷϗʔϜϖʔδʯ͔ ΒཁૉΛऔಘ # Ѩ෦׮͞ΜͷϗʔϜϖʔδ͔ΒtitleλάΛऔಘ $ perl -Mojo -E \ 'say g("http://abehiroshi.la.coocan.jp/")->dom->at("title")' | nkf -w Ѩ෦׮ͷϗʔϜϖʔδ # Ѩ෦׮͞ΜͷϗʔϜϖʔδ͔Β஀ੜ೔Λऔಘ $ perl -Mojo -E \ 'say g("http://abehiroshi.la.coocan.jp/top.htm")->dom("td")->[4]->text' | nkf -w ੜ೥݄೔ 1964೥6݄22೔

Slide 91

Slide 91 text

.BD#PPLͷ੔උ඼৘ใΛ νΣοΫ͍ͨ͠

Slide 92

Slide 92 text

.BD#PPLͷ੔උ඼৘ใΛνΣοΫ͍ͨ͠ # MacBook੔උࡁ੡඼ͷҰཡΛऔಘ $ perl -MData::Dumper -Mojo -C -E \ '$url = "https://www.apple.com/jp/shop/browse/home/specialdeals/mac/macbook"; \ map { push @$hash, $_->text } g("$url")->dom->find("span[itemprop=\"price\"]")->each; \ g("$url")->dom->find(".specs")->each( sub { my ($e, $num) = @_; \ say join(", ", $hash->[$num-1], $e->text) }) ' ˇ120,800 (੫ผ), 2017೥6݄ൃച 12Πϯνʢର֯ʣLEDόοΫϥΠτσΟεϓϨΠɺ2,304 x 1,440ϐΫηϧղ૾౓ɺ226ppi 8GB 1,866 ΦϯϘʔυϝϞϦ 256GB PCIeϕʔεΦϯϘʔυϑϥογϡετϨʔδ 480p FaceTimeΧϝϥ Intel HD Graphics 615 ˇ120,800 (੫ผ), 2017೥6݄ൃച 12Πϯνʢର֯ʣLEDόοΫϥΠτσΟεϓϨΠɺ2,304 x 1,440ϐΫηϧղ૾౓ɺ226ppi 8GB 1,866 ΦϯϘʔυϝϞϦ 256GB PCIeϕʔεΦϯϘʔυϑϥογϡετϨʔδ 480p FaceTimeΧϝϥ Intel HD Graphics 615

Slide 93

Slide 93 text

PKPQN w ଞʹ΋PKPQNʹ͸ૉఢͳϫϯϥΠφʔίϚϯυ͕ͨͪ͋Γ·͢ w IUUQNPKPMJDJPVTPSHQFSMEPDPKP w IUUQTNPKPMJDJPVTJPCMPHEBZPOFMJOFSTGPSGVO BOEQSPpU w KTPOͰग़ྗͨ͠ΓɺϫϯϥΠφʔͰ8ΞϓϦΛ্ཱͪ͛ͨΓɺϕϯνϚʔ ΫͳͲ΋ग़དྷΔΑ͏Ͱ͢

Slide 94

Slide 94 text

ͪΐͬͱͨ͠όάใࠂ࣌ʹ

Slide 95

Slide 95 text

ͪΐͬͱͨ͠όάใࠂ࣌ʹ w 63*ϞδϡʔϧΛ࢖͍ͬͯΔՕॴͰɺ༧ظͤ͵Τϥʔ ͕ग़ͨ৔߹ w ϫϯϥΠφʔͳΒɺͨͬͨߦ͚ͩͰϙΠϯτΛઆ໌Մೳ $ perl -MURI -E 'say URI->new("http://yapcjapan.org")->host ' yapcjapan.org # ϢʔβଆͰೖΕͨURLͷεΩʔϜ͕ෆ׬શͳͨΊɺdieͯ͠͠·͏έʔε͕ൃੜ $ perl -MURI -E 'say URI->new("ttp://yapcjapan.org")->host ' Can't locate object method "host" via package "URI::_foreign" at -e line 1. # ͜͏͢Ε͹OK $ perl -MURI -E 'my $uri = URI->new("ttp://yapcjapan.org/"); \ $uri->scheme('http') unless $uri->scheme =~ /^https?/; say $uri->host ' yapcjapan.org

Slide 96

Slide 96 text

ࣄྫฤ͓ΘΓ

Slide 97

Slide 97 text

ϫϯϥΠφʔΛ࢖͍΍͘͢͢ ΔͨΊͷUJQT

Slide 98

Slide 98 text

ϫϯϥΠφʔΛ࢖͍΍͘͢͢ ΔͨΊͷUJQT w ώετϦ͸ๅͷࢁ w γΣϧͷIJTUPSZ͸ͱʹ͔͘େྔʹ࢒͓ͯ͠ ͘ w QFDPΛΠϯετʔϧͯ͠ɺ$USMSͷΠϯΫϦ ϝϯλϧαʔνΛߋʹศརʹͯ͠࢖͍·Θ͢

Slide 99

Slide 99 text

ϫϯϥΠφʔΛ࢖͍΍͘͢͢ ΔͨΊͷUJQT w جຊ͸ώετϦΛ$USMSͰྑ͍͕ɺ·ͩ׳Εͯͳ͔ͬͨΓɺ·ͨ࢖͍ͦ͏Ͱ ͋Ε͹εχϖοτπʔϧʹొ࿥͓ͯ͘͠ w ࣗ෼͸Ҏલ͸4IFFUͱ͍͏εχϖοτπʔϧΛ࢖͍ͬͯ·ͨ͠ɻ w IUUQPTDBSEFMCFODPNTIFFU w ίϚϯυϥΠϯͷΈͰࡁΉͷͰ͔ͳΓศར w .BDͷํͰ͸ w ͱ͍͏ΫϦοϓϘʔυ֦ுεχϖοτొ࿥͕Ͱ͖ΔπʔϧΛ࢖ͬͨΓ͠ ͍ͯ·͢ɻ w IUUQXXXDMJQNFOVDPNKB

Slide 100

Slide 100 text

ϫϯϥΠφʔΛ࢖͍΍͘͢͢ ΔͨΊͷUJQT w ʜ͔͠͠࠷ۙͰ͸ɺ6/*9ͷ఩ֶͷݪ఺ճؼతʹɺ εχϖοτ؅ཧʹ͸%SPQCPYͰ֤छςΩετϑΝ Πϧʹશ෦ಥͬࠐΉΑ͏ʹ͍ͯ͠·͢ɻ w ଟগݕࡧ͸஗ͯ͘΋ɺ͢΂ͯΛςΩετϑΝΠ ϧʹ͓͍ͯͯ͠ɺ͍ͭͰ΋HSFQSग़དྷΔͷ ͸΍ͬͺΓ҆৺ w ݸਓͷײ૝Ͱ͢ʣ

Slide 101

Slide 101 text

ϫϯϥΠφʔͷ஫ҙ఺

Slide 102

Slide 102 text

ϫϯϥΠφʔͷ஫ҙ఺ w ϫϯϥΠφʔ͸͋͘·Ͱ΋ૣ͘࢓ࣄΛย෇͚Δ ͨΊͷ΋ͷͰ͋ͬͯɺ߃ٱతʹ࢖͏Α͏ͳ΋ͷ Ͱ͸ͳ͍ࣄ͕ଟ͍ɻ w ࠓޙ΋௕͘࢖͏ॏཁͳ࡞ۀͰ͋Ε͹ɺ͖ͪΜͱ εΫϦϓτΛ૊ΜͰɺHJU؅ཧ͢΂͖ɻ w ౰વͳ͕Β(6**%&ͷํ͕ޮ཰͕ྑ͍৔߹΋͋ Γ·͢ɻྟػԠมʹ࢖͍෼͚·͠ΐ͏

Slide 103

Slide 103 text

·ͱΊ

Slide 104

Slide 104 text

·ͱΊ w ϫϯϥΠφʔͰָΛͯ͠࡞ۀΛૣ͘ऴΘΒͤΑ͏☺ w 6/*9ͷ఩ֶॏཁ w εϞʔϧŋΠζŋϏϡʔςΟϑϧ w ෺ࣄ͸γϯϓϧʹɺԿࣄʹ΋ૄ݁߹Λҙࣝ͠Α͏ w ಛʹ1FSMΛ஌͍ͬͯΔͱ$-*ܥͰͷ࡞ۀશൠʹ΋࢖ ͍ճ͠Ͱ͖͓ͯಘʂʂʂ

Slide 105

Slide 105 text

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

Slide 106

Slide 106 text

͓ΘΓ