Cookpad Tech Kitchen #9 https://cookpad.connpass.com/event/60831/
ΫοΫύουͷϩάΛ͍͍ײ͡ʹ͍ͯ͠ΔΞʔΩςΫνϟ2017/07/26 Cookpad Tech Kitchen #9Hokuto Hoshi[email protected]
View Slide
ే (΄͠ ΄͘ͱ) / @kani_b• ΫοΫύουגࣜձࣾ ΠϯϑϥετϥΫνϟʔ෦• SRE• AWS ͱηΩϡϦςΟ͕ϝΠϯྖҬ• ࣾ֎ͷใηΩϡϦςΟશൠ୲
ࠓͷςʔϚ: ϩά
https://speakerdeck.com/kanny/miao-jian-shu-mo-falseroguwoiigan-zinisuruakitekutiya
ͳͥɺਓϩάΛूΊΔͷ͔
“ى͖ͨ͜ͱ” ϩά͚͕ͩࣔ͢• αʔϏεվળͷͨΊʹඞཁෆՄܽ• Ծઆͷݕূ• ύϑΥʔϚϯεվળ• γεςϜτϥϒϧͷղফ• etc…
ऩू͍ͯ͠Δϩά• PV ϩά• ϖʔδӾཡ͝ͱʹه͞ΕΔ• ӾཡɺݕࡧಈͷੳͳͲ• ࠂܥϩά• ΠϯϓϨογϣϯɺΫϦοΫɺ ίϯόʔδϣϯ• ৴੍ޚޮՌଌఆʹ• ΞΫςΟϏςΟϩά• Ϣʔβߦಈ (ը໘ભҠͳͲ)• ੳɺࢪࡦݕূɺ ϞόΠϧΞϓϦͷಈ࡞ϩά• ࠪϩά• Ϣʔβσʔλͷ࡞ɾߋ৽ɾ আɺϩάΠϯϩάͳͲ• αϙʔτɺηΩϡϦςΟࠪ
ऩू͍ͯ͠Δϩά• KPI ༻ϩά• “ग़དྷࣄ” Λه͢Δϩά• UU, DU ͳͲࣗಈܭࢉɺ KPI ཧ༻• γεςϜϩά• OSɺϛυϧΣΞͷϩά• γεςϜཧ༻• etc…
ݱࡏͷن• Fluentd ʹྲྀΕ͍ͯΔϩά (֓ࢉ)• σʔλ૯ྔ: 400~600GB / • Ϩίʔυ: 8ԯϨίʔυҎ্ / • ඵؒ8,000 ~ 25,000 Ϩίʔυ͘Β͍※ࡢͷൃද͔࣌ΒϨίʔυͷݮΛ͍ͯ͠·͢
ཁ͍Ζ͍Ζͳϩά͕ ͍ͬͺ͍͋Δ
ϩάʹٻΊΔཁ݅• ࣮֬ʹૹ͞ΕΔ͜ͱ• Ͱ͖Δ͚ͩԆ͕গͳ͍͜ͱ• ྔΛݮΒ͢ (αϯϓϦϯά) ඞཁ͕ͳ͍͜ͱ• ੳ͍͢͠ܗͰอ࣋Ͱ͖Δ͜ͱ
ϩάͷҰੜ• ͋ͭΊͯ• ॲཧͯ͠• ͔ͭ͏͜ͷΜͷΛ͠·͢
ΫοΫύουͷ ϩάऩू(αʔϏεͰར༻͍ͯ͠Δͷ)
,JOFTJT4USFBNT4-BNCEBGVODUJPO%ZOBNP%##BUDI 4FSWFS3FETIJGU$MVTUFS"UIFOB&MBTUJDTFBSDIߏ"QQ4FSWFST"QQ4FSWFST"QQ4FSWFST-PH "HHSFHBUPS
-PH "HHSFHBUPS,JOFTJT4USFBNT"QQ4FSWFST4-BNCEBGVODUJPO%ZOBNP%##BUDI 4FSWFS3FETIJGU$MVTUFS"QQ4FSWFST"QQ4FSWFST"UIFOB&MBTUJDTFBSDIߏ
ΞϓϦέʔγϣϯαʔό͔Β• ΈΜͳେ͖ fluentd• ਖ਼֬ʹ td-agent• Ͱ͖ΔݶΓసૹઌগͳ͘• ूϊʔυ or Kinesis Streams
td-agent ूϊʔυ• ॾࣄʹΑΓ·ͩ 0.12• શϩά͕ूத͢ΔͨΊߴෛՙ• multiprocess plugin Ͱ͏·͍͜ͱ͍ͬͯΔ
Redshift• ࣾͷσʔλੳͷத৺ (DWH)• ϩά͚ͩͰͳࣾ͘ͷશ DB ͷσʔλΛ औΓࠐΜͰ͓Γ౷߹ੳ͕Մೳ• શࣾһ͕ར༻͢Δ (Tableau Redash)• S3 ͷϩάσʔλΛόονδϣϒͰऔΓࠐΈ• ϩʔυִؒΊ
Lambda + DynamoDB• ετϦʔϜूܭʹར༻ (ओʹࠂ)• Redshift ͷΠϯϙʔτִؒΛ͑Δ༻్• Kinesis Streams Ͱड͚ Lambda ͰूɺDynamoDB ʹॻ͘• ྲྀྔ͕େ͖͍ͨΊूϊʔυΛΘͳ͍• ϩάͦͷͷ S3 ʹॏෳૹ৴• ظతͳੳ Redshift Ͱߦ͏
Elasticsearch• ूܭμογϡϘʔυʹར༻• αʔϏεʹར༻͞Εͳ͍• ߴ͍ྲྀྔͷϩάʹΘͳ͍ or αϯϓϦϯά• Amazon Elasticsearch Service Λར༻
ͦͷଞ• getsentry/sentry• Τϥʔϩάऩूπʔϧ• Rails ΞϓϦέʔγϣϯͷΤϥʔΛૹ৴• Slack• Ϣʔβ͔Βͷ͝ҙݟͳͲΛૹ৴
Athena• ݕূத• Redshift ͷϩʔυִؒΛͯͳ͍͕ SQL Ͱੳ͍ͨ͠έʔε ʹ͏ఆ• Redshift Spectrum ͱ߹Θͤͯߟ͑த
શମ૾• σʔλετΞπʔϧʹΑΓಘҙෆಘҙ͕͋Δ• ΞϓϦ͔Β৮Δͱྑ͍ͷɺ ਓ͕ؒੳ͍͢͠ͷ (SQL Ͱ৮ΕΔͳͲ)• සൟͳॻ͖ࠐΈ͕ଟ͍ͷɺͦ͏Ͱͳ͍ͷ• αʔϏεϨϕϧڐ༰Ͱ͖ΔԆɺ༻్ͰબͿ
,JOFTJT4USFBNT4-BNCEBGVODUJPO%ZOBNP%##BUDI 4FSWFS3FETIJGU$MVTUFS"UIFOB&MBTUJDTFBSDIߏ"QQ4FSWFST"QQ4FSWFST"QQ4FSWFST-PH "HHSFHBUPSͰ͖Δ͚͍ͩԆͰΞϓϦ͔Βͷར༻ϝΠϯ͋ΔఔͷԆڐ༰Ͱ͖ΔΞϓϦ͔Βਓؒར༻αʔϏεӡ༻্ΫϦςΟΧϧͰͳ͍ϩάʹར༻
ΫοΫύουͷ ϩάऩू(ͦͷଞ)
ΞϓϦέʔγϣϯͷಈ࡞ϩά• td-agent Ͱऩू• CloudWatch Logs ʹૹ৴• fluent-plugin-cloudwatch-logs• Docker Խ͞Ε͍ͯΔΞϓϦඪ४උ• Τϥʔϩά Sentry ʹૹ৴
γεςϜϩά• ݱࡏ2ͭΛฒߦՔಈ• rsyslog• ;ͭ͏ͷϩάαʔό• td-agent Ͱऩूͯ͠ Kinesis Streams -> Lambda ͰGraylog • 2.3 ͔Β Elasticsearch Service ͕͑ΔΑ͏ʹ
·ͱΊ• ΫοΫύουͷϩάऩूج൫ʹ͍ͭͯ ͓͠·ͨ͠• த৺ʹͳ͍ͬͯΔͷ S3 ͱ Redshift• αʔϏεϨϕϧɺԆɺ༻్ʹΑͬͯ దͳͷΛબ͍ͯ͘͠