Slide 1
Slide 1 text
ຖୟ͚Δ
γΣϧܳΛ֮͑Α͏ʂ
ʙேͷγΣϧܳษڧձʙ
Yasuhiro Yamada (@grethlen)
2016-08-27 ୈ6ճ͏ॳ৺ऀ͚Ͱͳ͍ͳΜͯݴΘͳ͍Α
ઈରޕલͷγΣϧษڧձ/ୈ24ճ̋̋őཪ൪γΣϧܳษڧձ ޕલͷ෦
Slide 2
Slide 2 text
$ whoami
ͳ·͑: Yasuhiro Yamada
͓͠͝ͱ:
ΞϓϦέʔγϣϯΤϯδχΞ
ిࢠܾࡁ·ΘΓ͕ຊۀ
͠ΎΈ:
Ξχϝؑ
γΣϧܳ
Twitter:
@grethlen
Qiita:
greymd
ةݥ͡Όͳ͍Α
Slide 3
Slide 3 text
γΣϧܳྺ
ܳྺ ຊ֨తʹ։͖࢝Ίͨͷ͕͓͓Αͦ6લ͘Β͍
γΣϧܳؔ࿈Ͱ։ൃ
egzact — γΣϧͷऑΛิ͏ίϚϯυୡ
Github: greymd/egzact
cureutils — ࣍ੈITਓࡐҭπʔϧ
Github: greymd/cureutils
هࣄ(Qiita)
ʲͨͷ͍͠ͳʳ༷ʑͳίϚϯυୡΛԿߟ͑ͣʹͭͳ͍Ͱ༡Ϳ (1079 stock)
ςετͷΛݮΒͦ͏ʂϓϦΩϡΞͰֶͿPICT (305 stock)
ϓϦΩϡΞͰֶͿϫϯϥΠφʔWebεΫϨΠϐϯά (122 stock)
Slide 4
Slide 4 text
γΣϧܳͷֶͼํ
ϘτϜΞοϓ
→ࣝΛ͔ΒݻΊΔ
→ྫ: ମܥతʹ·ͱ·ͬͨ
ࢿྉΛಡΉ
τοϓμϯ
→Γ͍ͨ͜ͱΛઌʹܾΊΔ
→࠷ݶඞཁͳ͔ࣝΒಘΔ
→ྫ: άάͬͨ݁ՌΛ
ϦόʔεΤϯδχΞϦϯά
ඪ४ೖग़ྗɺ੍ޚߏ
ࠐΈίϚϯυɺ֎෦ίϚϯυɺ
ύΠϓϥΠϯɺऴྃεςʔλεɺ
ύϥϝʔλల։……etc
֦ுਖ਼نදݱɺ
AWKͷGolfɺevalࠇຐज़
ίϚϯυͷෳ߹ɺ
ศརͳΦϓγϣϯ……etc
Γ͍ͨ
͜ͱ
Slide 5
Slide 5 text
ϘτϜΞοϓͳΞϓϩʔν
ମܥతʹ·ͱ·ͬͨࢿྉΛಡΉ
• γΣϧεΫϦϓτجຊϦϑΝϨϯε
(ஶ: ࢁ ৎൣ)
• Software DesignͰϐϯͱ͖ͨΒങ͏
(ྫ: 20166݄߸ bash ࠶ೖ)
• γΣϧܳຊ
• ͳͲ…(࠷ޙͷํͷεϥΠυʹ·ͱΊͯ·͢)
Slide 6
Slide 6 text
τοϓμϯͳΞϓϩʔν
Γ͍ͨ͜ͱΛάάΔ
→ग़͖ͯͨίϚϯυΛϦόʔεΤϯδχΞϦϯά
ษڧձͰ՝Λղ͘
→ ͔Βͳ͍՝γΣϧܳਓʹҰ୴ͬͯΒ͏
→ ؾʹͳͬͨϑϨʔζΛ֮͑ͯؼͬͯͬͯΈΔ
ຖΛ։͍֮ͯ͑ͨϑϨʔζΛୟ͘ͷΦεεϝ
ΧϨϯμʔͷ֬ೝͱ͔ɺ
ੜ׆ʹඞཁͳ࡞ۀ͕GOODʢޙड़ʣ
Slide 7
Slide 7 text
ͲͬͪͷΞϓϩʔν͕͍͍ͷʁ
ࠓͷ্ୡϨϕϧʹԠͯ͡όϥϯεΛऔΔͷ͕େ
ϘτϜΞοϓ͚ͩͩͱख͕ಈ͔ͳ͍ɻ
τοϓμϯ͚ͩͩͱ͕ࣝ๘ͷʹɻ
Slide 8
Slide 8 text
ϓνγΣϧܳษڧձΛ
͜Ε͔Β։͖·͢
Slide 9
Slide 9 text
ਐΊํ(1/3)
ೖऀʢࠇ͍ը໘͕·ͩ͜Θ͍͓ͱͩͪʣ
ը໘ʹग़ΔίϚϯυࣸܦ͋Δ͍άάͬͯԼ͍͞ɻ
͖ͨΒมଶΛ؍ͯ͠Լ͍͞ɻ͜Θ͘ͳ͍Ͱ͢ɻ
ॳ৺ऀʢgrepͰϩάΛ֬ೝ͢Δͷ͕͍͍ͤͥͷ͓ͱͩͪʣ
͋ͨΒ͍͠ίϚϯυͱΦϓγϣϯΛ֮͑·͠ΐ͏ɻ
ޙͰࢿྉΛݟฦͯ͠Έ·͠ΐ͏ɻ
தڃऀʢxargsͰվߦΛআ͘ͷ͕श׳Խͯ͠Δ͓ͱͩͪʣ
Βͳ͍Tips͕͋Δͱࢥ͏ͷͰɺੋඇ࣋ͪؼͬͯԼ͍͞ɻ
มଶʢAWKͰGolf͢Δ͓ͱͩͪʣ
Ғͦ͏ͳ͜ͱݴͬͯͯ͢Έ·ͤΜɻมଶճظͯ͠·͢ɻ
Slide 10
Slide 10 text
ਐΊํ(2/3)
ಈ࡞ڥ
Ubuntu14.14ͷbashͰಈ࡞֬ೝࡁΈɻ
ࢿྉࣗମطʹWebͰݟΕ·͢ɻ
Ͱ͖Δ͚ͩ͑Λݟͳ͍ͰਐΊͯԼ͍͞ɻ
࣮ࢪதτοϓμϯͳֶͼΛɻ
ࢿྉࣗମޙ͔ΒಡΈฦͯ͠ϘτϜΞοϓͳֶͼͷ
ͨΊͷࡧҾͱͯ͠͝׆༻Լ͍͞ɻ
Slide 11
Slide 11 text
ਐΊํ(3/3)
ͷઆ໌
$ ίϚϯυ -Φϓγϣϯͦͷଞ 'จࣈྻϦςϥϧ' ϑΝΠϧ
# ίϝϯτ
- ઌ಄ͷ $ ϓϩϯϓτΛද͢ه߸Ͱೖྗෆཁɻ
- จͷٯεϥογϡ \ վߦʢ͋ͬͯͳͯ͘OKʣ
[࣋ͪؼΓϙΠϯτ]
આ໌ͩΒͩΒ
Slide 12
Slide 12 text
Լ४උ (1/2)
1. Λ։͍ͯԼ͍͞
2. ԼهͷίϚϯυΛ࣮ߦͯ͠ɺϑΝΠϧΛμϯϩʔ
υ͍ͯͩ͘͠͞ɻ
$ wget goo.gl/PGDjKb -O holidays
※ wget͕ͳ͍ਓͪ͜Β
$ curl -L goo.gl/PGDjKb > holidays
Slide 13
Slide 13 text
Լ४උ (2/2)
3. ls ΛଧͬͯϑΝΠϧ͕͋Δ͜ͱΛ֬ೝɻ
$ ls
holidays
4. ϑΝΠϧͷ༰Λ֬ೝ
$ head holidays
2015-01-01 ݩ 1
2015-01-02 ۚ ࡾ͕ 1
2015-01-03 ࡾ͕ 1
…
Slide 14
Slide 14 text
͜ͷϑΝΠϧʁ
2015ʙ2021·ͰͷຊͷҰཡ
ϑΥʔϚοτʢ [ ] εϖʔεʣ
YYYY-MM-DD[ ]༵[ ]Πϕϯτ໊[ ]ٳϑϥά
ٳϑϥάʢٳ=1, ฏ=0)
12/30ʙ01/03·Ͱٳѻ͍
ʢࣄೲΊɺେᷢɺࡾ͕ʣ
͓ຍ8/13ʙ8/16ʹઃఆ
దٓݸਓͷঢ়گʹԠͯ͡ॻ͖͑ͯԼ͍͞ɻ
Slide 15
Slide 15 text
Q1
2016ͷ͚ͩΛඪ४ग़ྗʹදࣔͯ͠Լ͍͞ɻ
※தڃऀҎ্ͷํ”g”ͷͭ͘ίϚϯυېࢭ
$ ? ? ?
2016-01-01 ۚ ݩ 1
2016-01-02 ࡾ͕ 1
2016-01-03 ࡾ͕ 1
2016-01-04 ݄ ฏ 0
…
Slide 16
Slide 16 text
ύλʔϯϚονΛ͠Α͏(1/2)
ղྫ
$ grep '2016' holidays
2016-01-01 ۚ ݩ 1
2016-01-02 ࡾ͕ 1
2016-01-03 ࡾ͕ 1
…
[grepίϚϯυͷ͍ํͷجຊ]
1. grep <நग़͍ͨ͠ύλʔϯ> <ϑΝΠϧ໊>
2. cat <ϑΝΠϧ໊> | grep <நग़͍ͨ͠ύλʔϯ>
Slide 17
Slide 17 text
ύλʔϯϚονΛ͠Α͏(2/2)
ผղ
$ cat holidays | grep 2016
grepͷ͍ํ͕Θ͔Βͳ͍ਓʹ͜ΜͳΓํɻ
$ cat holidays | sed -n '/2016/p'
$ cat holidays | awk '/2016/'
$ cat holidays | perl -ne '/2016/ and print'
: grep͍ͷͰɺେͷ߹grepΛͬͨ΄͏͕͍͍
Ͱ͢ɻ
Slide 18
Slide 18 text
Q2
20162݄Կ͋Δ͔
͑ͯΈ·͠ΐ͏ɻ
$ ? ? ?
29
Slide 19
Slide 19 text
ߦΛΧϯτ͠Α͏(1/2)
ղྫ
$ cat holidays | grep '2016-02' | wc -l
29
Slide 20
Slide 20 text
ߦΛΧϯτ͠Α͏(2/2)
Լه wc -l (ߦΧϯτ)ͱ
΄΅ಉ͡ڍಈΛ͢Δɻ
• awk 'END{print NR}'
• grep -c .
• nl -nln | cut -f 1 | tail -n 1
• perl -pE '}{ say $.'
Slide 21
Slide 21 text
Q3
20169݄19Կͷʁ
Πϕϯτ໊ͷΈΛදࣔͯ͠Έ·͠ΐ͏ɻ
தڃऀҎ্ͷਓ”a”ͷͭ͘ίϚϯυېࢭ
$ ? ? ?
ܟͷ
Slide 22
Slide 22 text
ಛఆͷϑΟʔϧυΛநग़(1/2)
ղྫ
$ cat holidays | grep '2016-09-19' | \
awk '{print $3}'
[ϑΟʔϧυ]
จࣈྻΛεϖʔεͰ۠ͬͨͱ͖ͷจࣈͷ·ͱ·Γͷ
͜ͱɻ্هྫࠨ͔Β̏൪ͷϑΟʔϧυΛදࣔɻ
2015-01-01[εϖʔε][εϖʔε]ݩ[εϖʔε]1
$1 $2 $3 $4
Slide 23
Slide 23 text
ಛఆͷϑΟʔϧυΛநग़(2/2)
Լهͷྫશͯɺ̏ϑΟʔϧυΛநग़͢Δྫ
• awk '$0=$3'
• cut -d ' ' -f 3
• perl -anle 'print $F[2]' # Perl0ΠϯσοΫε
• perl -aple '$_=$F[2]'
• self 3 # Open usp Tukubai ༻
Slide 24
Slide 24 text
Q4
ࠓͷใΛදࣔͯ͠Έ·͠ΐ͏ɻ
ͨͩ͠ɺίϚϯυʹҰࣈΛΘͣʹɻ
$ ? ? ?
2016-08-27 ٳ 1
Slide 25
Slide 25 text
ίϚϯυஔΛ͔͓ͭ͏(1/2)
ղྫ
$ cat holidays | grep $(date +%F)
[ίϚϯυஔ]
$(ʙ) ͋Δ͍ όοΫΫΦʔτʢ`ʣΛͬͨ
`ʙ` ͱ͍͏දݱʮίϚϯυஔʯɻ
ʙ ʹॻ͍ͨίϚϯυͷ݁ՌΛจࣈྻʹ
ͨ͠ͷͷΑ͏ʹ;Δ·͏ɻ
Slide 26
Slide 26 text
ίϚϯυஔΛ͔͓ͭ͏(2/2)
ίϚϯυஔͷྫ
$ date +%F
2016-08-27
$ echo $( date +%F ) #(1)
2016-08-27
$ echo '2016-08-27' #(2)
2016-08-27
(1)ͱ(2)ͷίϚϯυ݁Ռతʹಉ͡ɻ
Slide 27
Slide 27 text
Q5
2016ʹɺ֤༵͕͍ͭ͋͘Δ͔දࣔͯ͠Έ·͠ΐ͏ɻ
$ ? ? ?
53
52
52 ݄
52
52 ਫ
52 Ր
53 ۚ
Slide 28
Slide 28 text
্͑͛(1/2)
ղྫ
$ cat holidays | grep '2016' | awk '{print $2}' | \
sort | uniq -c
[Α͘͏͑ํͷϑϨʔζ]
• sort | uniq -c
• sort | uniq -c | sort -n
Slide 29
Slide 29 text
্͑͛(2/2)
sortͱuniqͷ͍ํ͕͔Βͳ͍ਓ͚ճ
$ cat holidays | \
grep '2016' | awk '{print $2}' | \
awk '{a[$1]++}END{for (k in a){print a[k],k}}'
$ cat holidays | \
grep '2016' | awk '{print $2}' | \
perl -0777 -anlE 'grep{!$h{$_}++} @F; say
"$h{$_} $_" for keys(%h)'
Slide 30
Slide 30 text
Q6
201608݄03 ͔Β
201609݄30 ·Ͱදࣔͯ͠Έ·͠ΐ͏ɻ
$ ? ? ?
2016-08-03 ฏ 0
ʙ
2016-09-30 ۚ ฏ 0
Slide 31
Slide 31 text
͋Δύλʔϯ͔Β
͋Δύλʔϯ·Ͱநग़(1/2)
ղྫ
$ cat holidays | \
awk '/2016-08-03/,/2016-09-30/'
[ಛఆͷ2ύλʔϯؒͷग़ྗ]
awk '/։࢝ύλʔϯ/,/ऴྃύλʔϯ/'
Slide 32
Slide 32 text
͋Δύλʔϯ͔Β
͋Δύλʔϯ·Ͱநग़(2/2)
ผղ
# sedΛ͏
$ cat holidays | \
sed -n '/2016-08-03/,/2016-09-30/p'
# perlΛ͏
$ cat holidays | \
perl -nle 'print if /2016-08-03/../2016-09-30/'
ڧҾʹ
$ cat holidays | \
grep '2016-08-03' -A 50 | grep '2016-09-30' -B 50
Slide 33
Slide 33 text
Q7
2015ͱ2016Λൺֱͯ͠ɺ
ยํʹ͔͠ଘࡏ͠ͳ͍ॕΛදࣔͯ͠Լ͍͞ɻ
̎ͭ͋Γ·͢ɻ
$ ? ? ?
# ࣮ࡍղ͍ͯ֬ೝͯ͠ΈͯͶ
Slide 34
Slide 34 text
ϓϩηεஔΛ͔͓ͭ͏(1/3)
ղྫ
$ diff \
<(grep 2015 holidays | awk '{print $3}' | sort | uniq)
<(grep 2016 holidays | awk '{print $3}' | sort | uniq)
[ϓϩηεஔ]
<(ʙ) ͱ͍͏දݱʮϓϩηεஔʯɻ
<(ʙ) ͷՕॴɺ ʙ ʹॻ͍ͨίϚϯυͷ݁ՌΛهड़ͨ͠
ϑΝΠϧͷΑ͏ʹৼΔ͏ɻ
Slide 35
Slide 35 text
ϓϩηεஔΛ͔͓ͭ͏(2/3)
ϓϩηεஔྫ
$ echo A B C > abc.txt
$ cat abc.txt | sed 's/A/B/' # (1)
B B C
$ cat <( echo A B C ) | sed 's/A/B/' # (2)
B B C
(1)ͱ(2)ͷίϚϯυ݁Ռతʹಉ͡ɻ
Slide 36
Slide 36 text
ϓϩηεஔΛ͔͓ͭ͏(3/3)
ϓϩηεஔΛΘͳ͍ผղ
$ cat holidays | grep -e '2015' -e '2016' | \
awk '{print $3}' | sort | uniq -u
Slide 37
Slide 37 text
Q8
※ ্ڃऀ͚
༗څΛͬͯܭըతʹ࿈ٳΛ࡞ΔͨΊʹ
2015 ʙ 2021ͷؒͰ
ٳ→ฏ→ٳ
ͱͳ͍ͬͯΔ3ؒΛશͯநग़ͯ͠Έ·͠ΐ͏ɻ
$ ? ? ?
2015-11-01 2015-11-02 2015-11-03
2016-02-11 2016-02-12 2016-02-13
2016-05-01 2016-05-02 2016-05-03
…
݄͕Θ͔ΕΑ͍ͷͰɺ༵ٳϑϥάΛग़ྗͯ͠OKɻ
ग़ྗํ๏͓·͔ͤ͠·͢ɻ
Slide 38
Slide 38 text
ҙ
ෳߦʹ·͕ͨͬͯ
ॏෳ͢Δ͕͋Δ͔ɻ
{08-09, 08-10, 08-11} ͷτϦΦ
{08-11, 08-12, 08-13} ͷτϦΦ
༵ ༰
2020-08-09 ٳ
2020-08-10 ݄ ฏ
2020-08-11 Ր ࢁͷ
2020-08-12 ਫ ฏ
2020-08-13 ͓ຍ
Slide 39
Slide 39 text
ҰߦͰϧʔϓΛճͦ͏
ղྫ
$ cat holidays | tr '\n' '\t'| awk -F'\t' \
'{for(i=3;i<=NF;i++){print $(i-2),$(i-1),$i}}' \
| grep ' 1.* 0.* 1$'
গ͠Θ͔Γ͘͢
cat holidays | tr '\n' '\t'| awk -F'\t' \ # λϒจࣈΛ۠Γจࣈʹࢦఆ
'{
for(i=3;i<=NF;i++){ # NF(Number of Field)…ϑΟʔϧυ
print $(i-2),$(i-1),$i # ྫ: i͕3ͷ࣌$(i-1)$2ͱͯ͠ධՁ͞ΕΔ
}
}' \
| grep ' 1.* 0.* 1$' # 1(ٳ) " 0(ฏ) " 1(ٳ) (εϖʔεҙ)
Slide 40
Slide 40 text
ศརͳಓ۩Λ׆༻͠Α͏
egzactʢ࠷ऴεϥΠυࢀরʣͷ
convίϚϯυ͕͋ΕγϯϓϧʹΊΔʂ
ʢखલຯḩࣦྱʣ
$ cat holidays | tr '\n' '\t' | \
conv fs='\t' 3 | \
grep ' 1.* 0.* 1$'
convίϚϯυ
ϑΟʔϧυ۠ΓͷೖྗΛཐટঢ়ʹ܁Γฦ͢ɻ
$ echo 'A B C D E' | conv 3
A B C
B C D
C D E
Slide 41
Slide 41 text
͜ͷϑΝΠϧΛͬͨศརٕ(1/3)
ʮ2ϲ݄ͰͰ͖ΔΑͶʁʯͱݴΘΕͨ࣌
→ ࣮ࡍͷӦۀʁ
→ େܕ࿈ٳΛڬΉͱҙ֎ʹগͳ͍
$ cat holidays | \
awk '/2016-08-01/,/2016-09-30/' |\
grep '0$' | nl
Ӧۀ͚ͩநग़
→ ΤΫηϧʹషΓ͚ͯਐḿཧʢനʣ
Slide 42
Slide 42 text
͜ͷϑΝΠϧΛͬͨศརٕ(2/3)
ࠓͷ༗څ͍ͭ͘Թଘ͠Α͏͔ͳ……ɻ
ٳˠฏˠٳ
$ cat holidays | grep '2016' | \
tr '\n' '\t' | conv fs='\t' 3 | \
grep ' 1.* 0.* 1$'
ٳˠฏˠฏˠٳ
$ cat holidays | grep '2016' | \
tr '\n' '\t' | conv fs='\t' 4 | \
grep ' 1.* 0.* 0.* 1$'
Slide 43
Slide 43 text
͜ͷϑΝΠϧΛͬͨศརٕ(3/3)
࣍ͷૉͷ͍ͭͩΖ͏ʁؾʹͳΔͳ͋
$ cat holidays | grep '2016' | \
tr -d '-' | awk '$0=$1' | \
factor | awk '$0*=!$3' | \
sed -r 's/(..)(..)$/-\1-\2/' | \
grep -f - holidays
Slide 44
Slide 44 text
·ͱΊ
ೖऀϘτϜΞοϓֶशॏࢹ͕٢ͩͱࢥ͍·͢
׳Ε͖ͯͨΒΓ͍ͨ͜ͱΛ࡞ͬͯτοϓμϯ
Γํ͕Θ͔Βͳ͍࣌……
→ άάͬͯϦόʔεΤϯδχΞϦϯά
→ ͭΑ͍γΣϧܳਓʹฉ͘
ຖୟ͘ͱϚΠφʔͳૢ࡞ɾΦϓγϣϯ֮͑ͯ͘Δ
ͳΜͱ͔श׳Խͤ͞ΔͱͭΑ͘ͳΕΔɻ
ࠓͷࢿྉͷؾʹͳͬͨ෦ΛҰ୴ؙ҉ه͢Δͷखɻ
Slide 45
Slide 45 text
ࢀߟจݙɾURL
ੴҪ͞ΜͷʮγΣϧܳೖʯ
http://slideck.io/github.com/hisaharu/shellgei/slides/shellgei23AM2.md#/
ࡈ౻͞ΜͷAWKҰߦඦݓ
http://gauc.no-ip.org/awk-users-jp/material/100_one_liners_20131223.pdf
γΣϧϓϩάϥϛϯά࣮༻ςΫχοΫ
http://gihyo.jp/book/2015/978-4-7741-7344-3
γΣϧεΫϦϓτجຊϦϑΝϨϯε
http://gihyo.jp/book/2011/978-4-7741-4643-0
Software Design 20166݄߸ bash ࠶ೖ
http://gihyo.jp/magazine/SD/archive/2016/201606
Perl One-Liners (※༸ॻ)
http://shop.oreilly.com/product/9781593275204.do
Slide 46
Slide 46 text
ࢧԉπʔϧURL
egzact — Έ߹Θͤྻڍ͕ܰʑͰ͖ΔΑ͏ʹͳΔ
https://github.com/greymd/egzact
Open usp Tukubai — DB͍ΒͣͰूܭ࡞ۀ͕ϥΫνϯʹͳΔ
https://github.com/usp-engineers-community/Open-usp-Tukubai
nixar — ຯʹΓʹ͍͘࡞ۀָ͕ʹͳΔ͔
https://github.com/askucher/nixar
MCMD — ԯ୯Ґͷσʔλʹෳࡶͳ౷ܭॲཧ͕Ͱ͖Δ
http://www.nysol.jp/
Cureutils — ϓϦςΟͰΩϡΞͳศརίϚϯυ
https://github.com/greymd/cureutils