月間数千億リクエストをさばく技術 (ArchitectureNight公開用)

https://architecture-night.connpass.com/event/90666/

1. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़ גࣜձࣾαΠόʔΤʔδΣϯτ
   ΞυςΫຊ෦
   %ZOBMZTU
 ࠇ࡚
   ༏ଠ
   !LVSP@N
2. ࣗݾ঺հ w ࠇ࡚
   ༏ଠ
   !LVSP@N
   w גࣜձࣾαΠόʔΤʔδΣϯτ
 ΞυςΫຊ෦
   %ZOBMZTU
   όοΫΤϯυΤϯδχΞ
   w ೥
   ৽ଔೖࣾ
   w ೖࣾҎདྷ%ZOBMZTUͷೖࡳ
   ഑৴
   ܭଌ
   ूܭ෦෼ͷ։ൃ
   ΠϯϑϥΛ୲౰
   w ˞
   ࡱӨ࿥Ի౤ߘېࢭͷϖʔδʹ͸ࠨ্ʹˣͷΞΠίϯΛೖΕͯ͋Γ·͢
3. %ZOBMZTUͱ͸
4. Πϯλʔωοτ޿ࠂͱ͸
5. αΠόʔΤʔδΣϯτʹ͓͚ΔΞυςΫ
6. 35#ͷ͘͠Έ w 35#
   
   3FBM
   5JNF
   #JEEJOH 441 %41T
7. 35#ͷ͘͠Έ w 8FCϖʔδ͕ϩʔυ͞Εɺ޿ࠂλά͕ൃՐ 441 %41T
8. 35#ͷ͘͠Έ w ࿮৘ใ΍Ϣʔβ৘ใͱͱ΋ʹCJE
   SFRVFTU͕͘Δ 441 %41T CJE
   SFRVFTU
9. 35#ͷ͘͠Έ w ೖࡳֹͱදࣔ͢Δ޿ࠂΛܾఆ 441 %41T "% "% "% CJE
   SFRVFTU ԁ ԁ ԁ
10. 35#ͷ͘͠Έ w 0QFO
    35#ͱ͍͏ϓϩτίϧͰೖࡳ 441 %41T "% "% "% ԁ ԁ ԁ
11. 35#ͷ͘͠Έ w ೖࡳֹ͕Ұ൪ߴ͍ࣄۀऀ͕উར͢Δ 441 %41T "% "% "% ԁ ԁ ԁ XJO
12. 35#ͷ͘͠Έ w 441ʹ޿ࠂΫϦΤΠςΟϒ͕ૹΒΕΔ 441 %41T "% "% "% ԁ ԁ ԁ "% XJO
13. 35#ͷ͘͠Έ w 441ܦ༝Ͱམࡳͨ͠ࣄۀऀͷ޿ࠂ͕ల։͞ΕΔ 441 %41T "% "% "% ԁ ԁ ԁ "% XJO "%
14. 35#ͷ͘͠Έ w ͜ΕΛߴ଎ʹ܁Γฦ͢ 441 %41T "% "% "% ԁ ԁ ԁ "% XJO "% औҾ͸NTҎ಺ʹ׬ྃ͠ͳ͚Ε͹ͳΒͳ͍
15. %ZOBMZTUͱ͸ w %ZOBNJD
    3FUBSHFUJOH
    GPS
    (BNFT
    w εϚϗήʔϜ޲͚޿ࠂ഑৴ϓϥοτϑΥʔϜ
    w τοϓηʔϧε!೔ຊͷεϚϗήʔϜͷதͰ΋ߴ͍γΣΞ
    w ೔ຊɺΞϝϦΧΛؚΉΧࠃʹ഑৴த
    w Ϣʔβ͝ͱʹ࠷దԽͨ͠޿ࠂΛ഑৴ IUUQXXXEZOBMZTUJP
16. ޿ࠂΫϦΤΠςΟϒྫ
    ࡱӨ࿥Ի౤ߘېࢭ
17. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़
18. w "MM
    "84
    w ओཁར༻ίϯϙʔωϯτ %ZOBMZTUͷΞʔΩςΫνϟ Amazon EC2 Amazon ECR Amazon ECS AWS Lambda Elastic Load Balancing Amazon S3 Amazon RDS Amazon DynamoDB Amazon ElastiCache Amazon Redshift Amazon CloudFront Amazon Route 53 AWS Direct Connect Amazon VPC Amazon CloudWatch AWS OpsWorks AWS Trusted Advisor IAM AWS Certificate Manager Amazon Athena Amazon EMR Amazon Kinesis AWS Data Pipeline Amazon SQS Amazon SNS Amazon SES Amazon WorkMail
19. %ZOBMZTUͷΞʔΩςΫνϟ w "MM
    "84
    w BQOPSUIFBTU
    VTFBTUʹల։
    w ౰ॳΑΓάϩʔόϧͳϓϩμΫτΛ໨ࢦ͍ͯͨ͠ͷͰ"84Λબ୒
    w ໿ݸͷΞϓϦέʔγϣϯίϯϙʔωϯτͰߏ੒
20. %ZOBMZTUͷΞʔΩςΫνϟ ུ֓ BQOPSUIFBTU VTFBTU ೖࡳ഑৴ܥ ೖࡳ഑৴ܥ ܭଌܥ DBDIF
    DMVTUFS ूܭܥ DSPTT
    SFHJPO
    SFQMJDBUJPO DSPTT
    SFHJPO
    SFQMJDBUJPO DSPTT
    SFHJPO
    SFQMJDBUJPO ؅ཧը໘ DBDIF
    DMVTUFS όονܥ όονܥ
21. τϥϑΟοΫ֓گ ೔ຊͷೖࡳϦΫΤετඵ ΞϝϦΧͷೖࡳϦΫΤετඵ ೔ຊΞϝϦΧͷϨεϙϯελΠϜ NTFD
22. ͪͳΈʹ
    ࡱӨ࿥Ի౤ߘېࢭ ͋Δ೔ͷτϥϑΟοΫ
23. ͪͳΈʹ
24. ΠϯλʔωοττϥϑΟοΫ΋͍͍ͩͨ͜Μͳײ͡ IUUQXXXNGFFEDPKQTFSWJDFUSB⒏DIUNM
25. τϥϑΟοΫ֓گ w ͱ͋Δϲ݄ؒͷCJEϦΫΤετ਺ͷ߹ܭ
26. w DYMBSHF $
    (
    Y
    ໿୆
    ϐʔΫ
    64
    +1
    w DΠϯελϯε˕ େྔͷτϥϑΟοΫΛड͚ΔͨΊʹ DBDIF 3%# TMBWF DBDIF 3%# TMBWF BQOPSUIFBTUB BQOPSUIFBTUD ,74 3%# NBTUFS BQOPSUIFBTU
27. 4DBMBͱ͍͏બ୒ w %ZOBMZTUͰ͸΄΅શͯͷΞϓϦέʔγϣϯʹ4DBMBΛ࠾༻
    w ΠϯλϏϡʔهࣄ͋Γ·͢ http://gihyo.jp/dev/serial/01/cyberagent/0063
28. 4DBMBͱ͍͏બ୒ w 4DBMBΛબ୒ͨ͠ཧ༝
    w νϟϨϯδ
    ࠓͱͳͬͯ͸ී௨ʁ
    w Մಡੑɺϝϯςφϯεੑ͕ߴ͍
    w ϋΠύϑΥʔϚϯε
    w +BWBͷࢿ࢈͕࢖͑Δ
    w ΞυςΫελδΦ͸4DBMBͷ஌ݟͷմ X
29. 4DBMBʹΑΔฒྻॲཧ w ฒྻॲཧ͕ॻ͖΍͍͢
    w 'VUVSFͱ͍͏֓೦
30. 4DBMBʹΑΔฒྻॲཧ w ฒྻॲཧ͕ॻ͖΍͍͢
    w 'VUVSFͱ͍͏֓೦
    w ྫ
    3%#͔Βෳ਺ͷςʔϒϧΛҾ͖ͭͭɺ%ZOBNP%#΋Ҿ͘ ಉ࣌ʹ໰͍߹Θͤͯ ฒྻʹϨεϙϯεΛड͚Δ NT NT NT ௚ྻॲཧ
    NT
 ฒྻॲཧ
    NT
    
31. ϩά͸શͯ,JOFTJTʹྲྀ͢ w ೖࡳ΍ܭଌͷϩά͸શͯ,JOFTJT
    4USFBNTʹྲྀ͢
    w 8"1αʔό͸ϨεϙϯεΛฦ͢͜ͱ͚ͩʹूத
    w ֦ுੑ΍଱ো֐ੑͷ޲্ %#
    VQEBUF
    DPOTVNFS MPH
    BSDIJWF
    DPOTVNFS VQEBUF BSDIJWF ,JOFTJT
    4USFBNT DPOTVNF QSPEVDF
    MPHT BQQ
    TFSWFS %ZOBNP%# 4 1SPEVDFS 4USFBN $POTVNFS
32. 1SPEVDFS w ΞϓϦέʔγϣϯαʔό͕ు͍֤ͨछϩάΛ,JOFTJTʹ౤͛Δ ,JOFTJT
    4USFBNT qVFOUQMVHJOLJOFTJT BQQ
    TFSWFS 1SPEVDFS 4USFBN
33. ,JOFTJT
    1SPEVDFS
    -JCSBSZ w ,1-ͰτϥϑΟοΫઅ໿
    w ,JOFTJTͷ੍໿
    γϟʔυ͋ͨΓ SFDPSE
    165TFD
    PS
    .#T
    w ਺,#ͷϩάΛେྔʹసૹ͢Δʹ͸΋͍ͬͨͳ͍
    w ,1-Ͱ
    ,JOFTJTϨίʔυʹෳ਺ϝοηʔδΛ٧ΊࠐΉ
    QSPUPDPM
    CV⒎FST https://speakerdeck.com/kanny/logging-architecture-at-cookpad
34. $POTVNFS w ,JOFTJT͔ΒϨίʔυΛऔΓग़͢
    w DPOTVNFSΛ૿΍ͤ͹εέʔϧΞ΢τՄೳ ,JOFTJT
    4USFBN BQQ
    TFSWFS $POTVNFS 4USFBN %ZOBNP%#
35. ,JOFTJT
    $POTVNFS
    -JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT
    4IBSET BQQ
    TFSWFS $POTVNFS 4USFBN %ZOBNP%#
36. ,JOFTJT
    $POTVNFS
    -JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT
    4IBSET BQQ
    TFSWFS $POTVNFS 4USFBN %ZOBNP%# ো֐ൃੜ
37. ,JOFTJT
    $POTVNFS
    -JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT
    4IBSET BQQ
    TFSWFS $POTVNFS 4USFBN %ZOBNP%# ࣗಈ࠶ׂΓ౰ͯ
38. Ϣʔβσʔλͷѻ͍ w ޿ࠂ༻୺຤*% *%'"
    "EWFSUJTJOH*% ͱϢʔβσʔλͷ૊Έ߹Θͤ
    w ྫ
    ࠷ޙʹϩάΠϯͨ͠೔࣌
    ˠ
    ະϩάΠϯظ͕ؒΘ͔Δ
    w ޿ࠂೖࡳ࣌ʹେྔʹSFBE͕͞ΕΔͷͰਫฏεέʔϧͯ͠ཉ͍͠
    w Ϩίʔυ਺
    ਺ඦԯϨίʔυ
    w αϯϓϧςʔϒϧ ௒؆ུԽ %&7*$&@*% "%7&35*4&3@*% %&7*$&@5:1& -"45@-0(*/@"5 015065@"5 ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  J04  ʜʜ %&7*$&@  "OESPJE  ʜʜ
39. Ϣʔβσʔλͷѻ͍ w ,74ͱͯ͠%ZOBNP%#Λ࠾༻
    w )BTI
    ,FZͱ3BOHF
    ,FZΛ૊Έ߹ΘͤΔ
    w )BTI
    ,FZͰҾ͚͹3BOHF
    ,FZͰιʔτ͞Εͨෳ਺Ϩίʔυ͕ฦΔ %&7*$&@*% "%7&35*4&3@*% %&7*$&@5:1& -"45@-0(*/@"5 015065@"5 ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  J04  ʜʜ %&7*$&@  "OESPJE  ʜʜ )BTI
    ,FZ 3BOHF
    ,FZ
40. ϗοτύʔςΟγϣϯ໰୊ w ಛఆͷύʔςΟγϣϯʹΞΫηε͕ภΓɺύϑΥʔϚϯε͕ग़ͳ͍
41. ϗοτύʔςΟγϣϯ໰୊ w ,74ͷ໋॓
    w ಛఆͷύʔςΟγϣϯʹ3FBE8SJUF͕ภΔͱੑೳ͕ग़ͳ͍
    w ϗοτσʔλͱίʔϧυσʔλΛࠞͥͳ͍
    w ৽͍͠σʔλ
    ΞΫηε͞Ε΍͍͢
    ݹ͍σʔλ
    ΞΫηε͞Εʹ͍͘
    w ରࡦ
    w σʔλ͕ొ࿥͞Εͨ࣌ظ͝ͱʹςʔϒϧΛ෼͚Δ
    w ҰఆظؒΞΫηε͞Εͳ͍σʔλ͸࡟আ͢Δ OPEF OPEF OPEF ৽͍͠σʔλ ݹ͍σʔλ ଟ͍ গͳ͍ ී௨ ΞΫηεྔ
42. %ZOBNP%#ͷΩϟογϡ w Ωϟογϡͷಋೖ
    w ϨΠςϯγͷ࡟ݮ
    w %ZOBNP%#
    3FBE
    $BQBDJUZͷ࡟ݮ
    w ϗοτύʔςΟγϣϯ໰୊ͷ؇࿨
    w &MBTUJ$BDIF NFNDBDIFE Λ࠾༻
    w γϯϓϧɾߴ଎
    w ϚϧνεϨουͰಈ͘
    w SܥΠϯελϯε͸ωοτϫʔΫଳҬ͕(CQT·ͰόʔετՄೳ
    w IUUQTEFWDMBTTNFUIPEKQDMPVEBXTFDSOFUXPSLJOH %ZOBNP%# NFNDBDIFE BQQ  
43. %ZOBNP%#ͷΩϟογϡ w Ωϟογϡظ͕ؒ௕͍ͱϢʔβͷঢ়ଶมԽ͕͙͢ʹ൓ө͞Εͳ͍
    w Ϣʔβͷঢ়ଶΛར༻ͨ͠޿ࠂ͕΢ϦͳͷͰଈ࣌൓ө͍ͤͨ͞
    w %ZOBNP%#
    4USFBNT
    
    -BNCEB
    'VODUJPOͰΩϟογϡͷ࡟আ
    w %ZOBNP%#
    "DDFMFSBUPS͸ΫΤϦΩϟογϡ͸ࣗಈߋ৽͞Εͳ͍
    w ݱঢ়͸ࣗલͰ࣮૷͢Δ͔͠ͳ͍ %ZOBNP%# 61%"5& %ZOBNP%#
    4USFBNT %&-&5&
44. w ϦΫΤετΛड͚͚ͨͩͰ͸μϝɺूܭ͕ඞཁ
    w ϩάͷྔ
    5#EBZ
    w "QBDIF
    4QBSL
    PO
    "NB[PO
    &.3
    w ΫϥελίϯϐϡʔςΟϯάϑϨʔϜϫʔΫ
    w ෼ࢄڞ༗ϝϞϦϞσϧ
    w ϊʔυΛ૿΍͢͜ͱͰεέʔϧ͢Δ
    w 4QBSL
    .-JCͱ͍͏ػցֶशϥΠϒϥϦ΋ &.3
    $MVTUFS .BTUFS 4MBWF 4MBWF 4MBWF 4MBWF ϩάूܭ
45. ϩάूܭ w "QBDIF
    4QBSL
    PO
    &.3
    w ෳࡶͳूܭॲཧ͕ॻ͚Δ
    w ྫ
    ୯७ͳ࣌ؒ
    Y
    ޿ࠂओͷ
 ΫϦοΫ਺ूܭ 42-Ͱॻ͍ͨ৔߹ 4QBSLͰॻ͍ͨ৔߹
46. ϋΠύϑΥʔϚϯεͰ৴པੑͷඞཁͳ
 γεςϜΛӡ༻͢ΔͨΊʹ
47. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔ w ҰൠతͳαʔϏε ϝσΟΞ
    ήʔϜ
    w αʔϏεఏڙ͕Ͱ͖ͳ͍
    ˠ
    Ϣʔβͷ཭୤
    ऩӹ௿Լ
    w ޿ࠂ഑৴αʔϏε
    w ޿ࠂ͕഑৴Ͱ͖ͳ͍
    ˠ
    ػձଛࣦ
    ޿ࠂओͷ཭୤
    ऩӹ௿Լ
    w ޿ࠂ͕഑৴͞Εͳͯ͘΋Ϣʔβʹରͯ͠͸Өڹ͕ͳ͍ %ZOBMZTU Ϣʔβ ޿ࠂओ ൃ஫ ഑৴
48. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔ w αʔϏε͕੒௕͠ച্ͷن໛͕͕͖͋ͬͯͨ
    ˠ
    ച্ͷॏཁ౓͸มΘΒͳ͍
    w ച্ສԁ೔Ͱ࣌ؒ഑৴ఀࢭ
    
    
    
    
    
    
    
    
    
    
    
    
    ສԁͷଛࣦ
    w ച্ ສԁ೔Ͱ࣌ؒ഑৴ఀࢭ
     
    
    
    
    ສԁͷଛࣦ
    w γεςϜن໛΋େ͖͘ͳΔͱ
    w ো֐ͷӨڹ΋େ͖͘ͳ͖ͬͯͨ
    w ҰํίετΧοτͷҖྗ΋େ͖͘ͳ͖ͬͯͨ
49. ߴ͍৴པੑ͸ઃܭ͕શͯ w ୯Ұো֐఺Λ࡞Βͳ͍
    w Ϋϥ΢υΛ࢖ͬͯΔͳΒΫϥ΢υωΠςΟϒͳϕετϓϥΫςΟεʹدͤΔ
    w ΞϓϦ͸Ͱ͖Δ͚ͩεςʔτϨεʹ
    w εςʔτΛ࣋ͭ෦෼ΛϚωʔδυαʔϏεʹدͤΔ
    w ιϦϡʔγϣϯΞʔΩςΫτʹ૬ஊ͢Δ
    w ͱ͸͍͑ෆଌͷࣄଶ͸ى͜Δʜ
    w ෮چ؇࿨ͷͨΊͷखஈ͸༻ҙ͢Δ΂͖
50. ؂ࢹ w %ZOBMZTUͰ͸؂ࢹʹ%BUBEPHΛར༻
    w l%ZOBMZTU
    %BUBEPHzͰݕࡧ
51. ؂ࢹ
52. ؂ࢹ w ޿ࠂओ͝ͱͷ഑৴ϝτϦΫε
53. ϦΫΤετ͕٧·ͬͯ͠·ͬͨ࣌ w Ұ࣌తʹϨεϙϯε͕٧·Δͱʜ
    w αʔόʹϦΫΤετ͕ཷ·Δ
    w ಉ࣌ॲཧϦΫΤετ਺͕രൃ͢Δ
    w εϨουϓʔϧΛա৒ʹ࢖͏
    w 
    Ϩεϙϯε଎౓ͷେ෯ѱԽ
    w ॲཧΛ࢝Ί͔ͯΒҰఆ࣌ؒܦաͨ͠ॲཧΛதஅͯ͠ೖࡳ͠ͳ͍
    w ೖࡳ͠ͳ͍
    
    )551
     /P
    $POUFOU Λฦ͢
    w 4DBMBͳΒͦ͏͍͏ॲཧ΋ॻ͖΍͍͢ʂ
54. ύϑΥʔϚϯεௐࠪ w :PVS,JU
    ঎༻+BWBϓϩϑΝΠϥ
    ΛѪ༻
55. ࠓޙ
56. ࠓޙ΍͍͖͍ͬͯͨ͜ͱ w 4DIFEVMFE
    4DBMJOH
    
    "VUP
    4DBMJOH
    w "LLB
    )551
    w "LLB
    4USFBNΛར༻ͨ͠ॲཧ
    #BDL1SFTTVSFͱ͔
    w Ωϟογϡ
    %#૚ͷ"VUP
    4DBMJOH
    w ϏϧυϦϦʔεͷߴ଎Խ
    w &&ςετͷಋೖ
    w ͳͲͳͲʜ
57. ͋Γ͕ͱ͏͍͟͝·ͨ͠