SRE Conf 2022 - 91APP 在 AWS 上的 SRE 實踐之路

* Date: 2022/04/29
* Title: 91APP 在 AWS 上的 SRE 實踐之路
* SRE Conf 2022 https://sre.ithome.com.tw/
* Youtube: https://youtu.be/f-__anVimq0
* https://rickhw.github.io/2022/05/10/About/2022-SREConf2022/

Transcript

1. ‹$PD]RQ:HE6HUYLFHV,QFRULWV DIILOLDWHV ᤲᥥ45&2'(ሸᑃၥ༹ 㡿$:6&UHGLW⨾ඖ ච㈝ᘓ❧$:6ᖒ⹰ ༶ྍ⮳ᨦ఩㡿ྲྀ㝈㔞ዲ⚰
2. $33:D\ $33ᅾ$:6ୖⓗ65(ᐿ㋄அ㊰  5LFN+ZDQJ#$33 
3. $33,QF&RQILGHQWLDO 3URSULHWDU\ ရ∩᪂㞽၎ᅧᐙ㝲 ᖺᡂ❧┠๓ဨᕤே ᖺ㛤ጞᢞධ㟁ၟᖹྎ⥂⇯ 㟁ၟᖺ㈨ேᖺ䠈202ᖺ㈨ேᖺ බྖ⥂⇯つᶍ⌧ἣ 㤳ᐙୖᷬ6DD6㌾㧓᭹ົၟ ௦ᨲ㔠ὶつᶍ㏆൨ྎᖯ ⣼ィ᭹ົ㉸㐣ᐙ㞽၎ရ∩ ᭹ົᐈᡬ᱌౛ /90+䚸/µ2UpDO䚸9)ရ∩㞟ᅰ ඲⫃䚸඲ᐙ䚸ᗣ᫝⨾ ྎ⅂๓஬኱ޝ⾰䚸㐣༙ⓒ኱㈽ᐙ 
4. ᭦ከ᱌౛ㄳୖDSSFRPVKRZFDVHV ᖳຓ㉸㐣ᐙ ୰ᑠရ∩ᐿ⌧ᩝ఩㎈ᆺ $33,QF&RQILGHQWLDO 3URSULHWDU\ Æ
5. ㌾㧓㛤ⓐ⪅ 6RIWZDUH'HYHORSHU 䚸ᴗ㣾㡢ᵹឡዲ⪅䠈ᕤసゅⰍ⥂ Ṗ6RIWZDUH'HYHOSRHU䚸64$0DQDJHU䚸2SV,QIUD0DQDJHU䚸 $UFKLWHFW䚹ᑙ㛗Ⅽ⣔⤫ᯫᵓ䚸㌾㧓㛤ⓐ䚸 ㌾㧓 ヨ䚸⣔⤫⥔㐠䚹㏺ᗄ ᖺᑙὀᅾศᩓᘧ⣔⤫䚸$:6䚸'HY2SV65(䚸⥂⇯⟶⌮➼㡿ᇦ䠈ⴭ ᭷ᢏ⾡㒊ⴠ᱁䠖&RPSOHWH7KLQN䚸㆞ⴭ䠖ศᩓᘧ⣔⤫タィ䚹 ᕤసஅ㣾႐ṅ⛉ᗁᑠ䃠䚸㔠ᗤṊಘ䚸⥂඾ᩥᏥ䚸ဴᏥ䚸ேᩥ⸤⾡䚹ஓ ᫝ᴗ㣾㡢ᵹឡዲ⪅䠈⣙᭷ᘘᖺⓗ⥂㦩䠈ᾙ␎ᾰ⵹ ྜྷ௚䚸㘽┙䚸⦅᭤䚸 ᩍᏥ䠈ⴭ᭷㡢ᵹ㒊ⴠ᱁䠖႑࿼ၲ⪵㡢ᵹཬ㟁Ꮚ᭩䚹 5LFN+ZDQJ KWWSVZZZJWFDIHFRP KWWSVULFNKZJLWKXELR 
6.  KWWSVZZZIDFHERRNFRPJURXSVVUHWDLZDQ
7.  +RZ$33 V65(LPSOHPHQWV'HY2SVXVLQJ$:6  7KHDUFKLWHFWXUHRI65(DVD6HUYLFH $JHQGD 
8. ,QIUDVWUXFWXUH &RPSXWH_1HWZRUN_ 6WRUDJH0LGGOHZDUH_ 5XQWLPH_'DWD2SHUDWLRQV _6HFXULW\ 'HSOR\2SHUDWHDQG 0DQDJH $OODFWLYLWLHVQHHGHGWR GHSOR\DQGVXSSRUW SODWRIUPLQIUDVWUXFWXUHDQG DSSOLFDWLRQVLQSURFXGWLRQ 'HYHORS%XLOGDQG7HVW $OODFWLYLWLHVQHHGHGWR GHILQHDQGYDOLGDWH3ODWIRUP ,QIUDVWUXFWXUHRU%XVLQHVV $SSOLFDWLRQV  (QJLQHHULQJ 2SHUDWLRQV 3ODWIRUP $SSOLFDWLRQV $SSOLFDWLRQ (QJLQHHULQJ $SSOLFDWLRQ 2SHUDWLRQV 3ODWIRUP (QJLQHHULQJ 3ODWIRUP 2SHUDWLRQV 6RXUFH$:6:HOO$UFKLWHFWHG)UDPHZRUN2SHUDWLRQDO([FHOOHQFH3LOODU %XVLQHVV6RIWZDUH &XVWRP'HYHORSHGRU &RPPHUFLDORIIWKH6KHOI
9. බྖⓗ᪉ྥ ⮬ᕫ㛤ⓐ䠈⮬ᕫ⥔㐠 )HDWXUH7HDPVRU$37HDP 
10. 65(せ㜝ὀⓗ䠛  ␗ᖖ⹦⌮  6/2䠖┘᥍ᶵไ䚸⣔⤫ᯫᵓ  ㌾㧓ᕤ⛬䠖ே⟶⣔⤫䚸⣔⤫⟶᭹ົ 
11. 
12. 3DUW +RZ$33 V65(LPSOHPHQWV'HY2SVXVLQJ$:6 a 
13. ၥ㢟   ␗ᖖ⹦⌮䠖ᚘⓐ⏕฿ⓐ⌧䠈ᛠ㯟⹦⌮䠛 D ᅰ㝲ᛠ㯟㐠స䠛 E ⣔⤫ᯫᵓⓗ㈨イ୍⮴䠛  ┘᥍ᶵไ䠖ᕤ⛬㠃䠄᭷ᩀⓗ┘᥍䠅䚸ไᗘ㠃䠄ٲ⌜ไᗘ䠅  ⣔⤫ⓗྍ㠁ᗘ䠖୙ᐃᮇⓗᴗົ⾜㖽άື
14. ᡃ಼ⓗ 65( ᐿ㋄ཎ๎䠖  ᑙὀᅾ⁁㏻ὶ⛬䚸$3,௓㠃  ┗ྍ⬟౑⏝0DQDJHG6HUYLFHV  ල㧓ⓗつ⠊⯅ὶ⛬䠈⣔⤫ᯫᵓ㏱᫂໬  ౑⏝㌾㧓ᕤ⛬䠈ㆸ65(௵ົつᶍ໬ 
15.  (QJLQHHULQJ 2SHUDWLRQV 3ODWIRUP $SSOLFDWLRQV $SSOLFDWLRQ (QJLQHHULQJ $SSOLFDWLRQ 2SHUDWLRQV 3ODWIRUP(QJLQHHULQJ 0DQDJHG6HUYLFHV 6RXUFH$:6:HOO$UFKLWHFWHG)UDPHZRUN2SHUDWLRQDO([FHOOHQFH3LOODU 6HSDUDWHG$(2DQG,(2ZLWK*HQWUDOL]HG*RYHUQDQFH $(2$SSOLFDWLRQ(QJLQHHULQJDQG2SHUDWLRQV ,(2,QIUDVWUXFWXUH(QJLQHHULQJDQG2SHUDWLRQV
16. ᛠ㯟೴ ୍䚸஦௳⹦⌮⯅⟶⌮ὶ⛬ᶆ‽໬ ஧䚸౑⏝㧗ྍ㠁ⓗ┘᥍⣔⤫ ୕䚸ᥦ㧗㐠⾜ᖹྎⓗྍ㠁ᗘ ᅄ䚸㝆ప⣔⤫ᯫᵓⓗ」㞯ᗘ ஬䚸ᶆ‽໬䚸つ⠊໬ 
17. ୍䚸஦௳⹦⌮⯅⟶⌮ὶ⛬ᶆ‽໬   6ᐈッⓗ⹦⌮䠈⏤ᴗົ⥔ 㐠 %L]2SV ➨୍⥺⹦⌮䚹 ዴᯝ㔜኱άືᡈ⪅㔜኱ᐈッ 䠈๎㟂ᦚ㓄බ㜝⯅༴ᶵ⟶⌮ ⛬ᗎ䠈ᨃゞޝእ᠕ㆰィ␓䚹  6ᕤ⛬ⓗၥ㢟䠈⏤⣔⤫⥔ 㐠 6\V2SV65( ┘᥍ⓐ ⌧⯅⹦⌮䚹ീ᫝ฟ⌧␗ᖖⓗ ὶ㔞䚸⿕''R6ᨷ᧬䚸✺ⓐ ⥙㊰⇕㯶஦௳«➼䚹  6ޝ㒊ᕤ⛬ၥ㢟䠈㏺㢮ⓗ ஦௳ᒞ᪊㛤ⓐ㐣⛬୰ᖖつᛶ ␗ື䠈ዴ஺௜ὶ⛬⯅㒊ᒞ᪉ ἲἄ᭷⹦⌮ዲ䠈ྍ⬟᭳ᅉṈ 㐀ᡂ⣔⤫ᘏ᏾ⓗၥ㢟䚹  3ޝ㒊⥔㐠ၥ㢟䠈ᒞ᪊⥔ 㐠ᕤ⛬ᅰ㝲ⓗᕤస㡯┠䠈ໟ ྵ⮬ⓐᛶⓗ㦝ື䚸㠀㡸ᮇⓗ 㦝ື䚹
18. ୺⟶&KDQQHO HPHUJHQF\FKDQQHO JHQHUDOLQFLGHQWFKDQQHO ᴗົᅰ㝲 ⭒ရᅰ㝲 ᅇሗޗᐜ䠖ᙳ㡪᭹ົ⠊ᅩ⌧ἣ㛤ጞ᫬㛫┠๓⹦⌮㐍ᗘୗḟᅇሗ᫬㛫 ึṉุ᪇␗ᖖ➼⣭ ⿕ື㏻▱ ☜ㄆ➼⣭ 3 3 3 ୺ື┘᥍ ẗศ㚝᭦᪂ ẗศ㚝᭦᪂ ẗศ㚝᭦᪂ >⹦⌮᏶␌@ ಙ௳䃚᫂␗ᖖ ⹦⌮≪ἣ 35ⓐබ㜝✏ 35⯅୺⟶ウㄽ 㔜㯶ᐈᡬ 3 3 3 ୺⟶䠋බ㜝 HPHUJHQF\ FKDQQHO ᕤႝ⣔⤫ ᕤႝ⣔⤫ 㟂せᑞእ䃠᫂බ㜝✏ ㏻ሗ⟶㐨 
19. ஧䚸౑⏝㧗ྍ㠁ⓗ┘᥍⣔⤫  ᅾ㏺ಶ㝵ẁ䠈ᡃ಼ⓗཎ๎ Ɣ ᢏ⾡Ỵ⟇ⓗ㜝㘽䠖᭹ົᴗົ Ɣ 㑊ච㝢ධ䛄┘᥍䛅┘᥍⣔⤫ⓗᅇᅤ Ɣ ྍ௨᥋ཷⓗᡂᮏ
20.  7LS1*; 7LS1*; 7LS1*; 7LS1*; (& &ORXG:DWFK/RJV /RJ*URXSV /RJ6WUHDP$ /RJ6WUHDP% /RJ6WUHDP& /RJ6WUHDP1 $ODUPV )LOWHUV 'DVKERDUG 0HWULFV 6167RSLFV ,QJLVKW VWGRXW ILOHEDVH LXXTWVMGOL[KMXLYFMS ,1*(67 ⵔ㞟 6725( ඈᏑ 352&(66 ⹦⌮ศᯒ 9,68$/,=( 127,)< ࿊⌧㏻▱ &ORXG:DWFK $JQHW $:6&ORXG:DWFK඲ᐙᱩ
21. /RJศᯒ⯅ḛュ䠖$:6$WKHQD 
22. Ɣ ᖹྎ .Vᮏ㌟ (.6  ౑⏝⪅௓㠃 5DQFKHU  'DVKERDUG *UDIDQD Ɣ つ⠊㈨※ศ㓄ⓗసᴗつ⠊䠈 ໟྵᡂᮏ䚸⏦ㄳὶ⛬䚸ᩍ⫱ カ⦎䚸Ḓ㝈⟶⌮䚸ᖖぢၥ㢟  «➼䚹  ୕䚸ᥦ㧗㐠⾜ᖹྎⓗྍ㠁ᗘ䠖ᑟධ.V䠈ᖹྎ໬
23. ᅄ䚸㝆ప⣔⤫ᯫᵓⓗ」㞯ᗘ 
24. ᭹ົ౫㉈⯅㑔⏺  .V3ODWIRUP (53 :HE6WRUH 260 %DWFK ,QJUHVV $3,*DWHZD\ 3UR[\ /RDG %DODQFHUV 7UDFNLQJ &06 6HDUFK (QJLQH 1RWLILFDWLRQ 6HUYLFH 6HDUFK$3, /R\DOW\ 3D\PHQW L26 $QGURLG 'HVNWRS -HQNLQV *LW/DE 0HVVDJH4XHXH$0&0
25. 6RXUFH'HY2SV'D\Vᗔ㐀ᙛӁԬ䃽65(៿䇀㜳࣑Ⲻฯ佀 
26.  +LJK/HYHO9LHZ㊴ฟ౗䠈┳඲ᬒ ż ޝእ㒊᭹ົ౫㉈䚸㑔⏺ᐃ⩏  6HUYLFH'HILQLWLRQ┳㐍ཤ䠈ᢍᮏ㉁ ż ᢳ㇟ゅⰍ 5ROH ᐃ⩏⯅㜝ಀ䠈୙ㄯᢏ⾡ ż ㄯゅⰍⓗᐃ⩏ 5ROH 䚸ゅⰍ⯅ゅⰍஅ㛫ⓗ౫㉈㜝ಀ䠈 :HE䚸'%䚸$3,䚸%DWFK䚸/%« ➼ ż ゅⰍஅ㛫⁁㏻ⓗ㏻イ༠㆟  +7737&3 䚸㈨ᩱὶ䚸㏻イᶍᘧ ୺ື䚸⿕ື ż Ꮡྲྀ᥍⟶䠖3XEOLF䚸3URWHFWHG䚸3ULYDWH ż ᦚ㓄8VHU6WRULHV  *R/LYHືᡭ೴䠈ᢍᐿ㋄ ż ල㇟ᯫᵓᐿ㋄䠈ໟྵ ヨ⎔ቃ䚸ṇᘧ⎔ቃⓗᐿ㋄ᢏ⾡  .V$:6RURWKHUV ż ᢏ⾡ᐿ㋄᫝ᕤ⛬ᅰ㝲┦஫⁁㏻⏝ⓗ䚹 ż ⪃㔞ྍ ᛶ䠈ໟྵ⎔ቃᘓ⨨䚸㒊⨫䚸ᩀ⬟䚸ྍ ᛶ ż ⪃㔞⥔㐠䠈ໟྵ㒊⨫䚸ᘓ⨨䚸┘᥍䚸⥔ㆤ䚸㈨Ᏻ䚸ᡂᮏ䚸἞⌮䚹 ᯫᵓつ⠊ 
27. $FFRXQW% $FFRXQW& $FFRXQW' $FFRXQW$ +4 0DLQ 0DLQ 93&% 0DLQ 93&% 0DLQ 0DLQ 0DLQ 93&% 0DLQ $FFRXQW( 0DLQ 7RN\R 6LQJDSRUH 2UHJRQ $FFRXQW) 0DLQ 93&% 0DLQ 0DLQ 
28. 5HJLRQ$ 5HJLRQ% 5HJLRQ& ᫍ≪䠖7;*:!93& ⥙≪䠖7;*:!7;*: ᫍ≪䠖7;*:!2Q3UHP $:67UDQVLW*DWHZD\  ┠ᶆ䠖⥙≪ĺᫍ≪ +4
29. $FFRXQW% $FFRXQW& ,QIUD $FFRXQW$ +4 0DLQ 0DLQ 93&% 0DLQ 0DLQ 0DLQ 93&% 0DLQ $FFRXQW( 0DLQ 7RN\R 6LQJDSRUH 2UHJRQ $FFRXQW) 0DLQ 93&% 0DLQ 0DLQ 
30. ஬䚸ᶆ‽໬つ⠊໬ 
31.  ⣔⤫ᯫᵓつ⠊ $UFK&RQYHQWLRQ  ᭹ົ┠㗴 6HUYLFH&DWDORJ  ᭹ົⓐ⌧ 6HUYLFH'LVFRYHU\  ⭘ฟ≀⟶⌮ $UWLIDFW0DQDJHPHQW  㓄⨨⟶⌮ &RQILJ0DQDJHPHQW  ᐦ☞䚸ᐦ㛋⟶⌮ 6HFUHW0DQDJHPHQW 㛤ⓐ⥔㐠἞⌮   ㈨※⯅⎔ቃ㓄⨨ 3URYLVLRQLQJ  ᘓ⨨ᶆ‽໬ %XLOG6SHF  㛤ⓐὶỈ⥺ 3LSHOLQH  や 䠖᪥ㄅ⯅┘᥍ᣦᶆ  2EVHUYDWLRQ  ┘᥍䠖┘᥍ᖹྎ⯅൤⾲ᯈ  0RQLWRULQJ  ඲ᇦ࿌㆙⣔⤫ *OREDO$OHUW6\VWHP  ␗ᖖ⹦⌮⯅⟶⌮ ,QFLGHQW0DQDJPHQW
32. $:6㈨※సᴗつ⠊  Ɣ ㄆㆇ⯅ᤵḒ ż 662WR$:6&RQVROH ż ㊬ᖒ⹰ゅⰍ Ɣ 5HVRXUFH7DJV ż つ⠊ཎ๎ ż ඲ᇦつ⠊ ż (& Ɣ ,$0⟶⌮つ⠊ ż ,$08VHU ż ,$0*URXS ż ,$05ROH ż ,$03ROLF\ $:6㈨※⟶⌮つ⠊ Ɣ (& Ɣ (/% Ɣ 93& Ɣ 6 Ɣ &ORXG:DWFK Ɣ 6HFXULW\*URXSV Ɣ '\QDPR'% Ɣ :$) Ɣ &ORXG)URQW Ɣ $WKHQD Ɣ 3DUDPHWHU6WRUH Ɣ .LQHVLV'DWD)LUHKRVH Ɣ $3,*DWHZD\
33. ඼௚䠖ᘬධ඼௚ᴫᛕ⯅ᢏ⾡   㑌ྥ$3,⥂⃽$3,*DWHZD\ᑟධஅ᪑  ⪵⪵㌾㧓஺௜ⓗ℃゗ㄯ⭘ฟ≀⟶⌮  ᚤ᭹ົⓗᇶ♏ᘓタ6HUYLFH'LVFRYHU\$QGUHZ:X  ᚘ㞽㛤ጞⓗ&RQILJXUDWLRQ0DQDJHPHQW/HYL&KHQ  ⅏㞴₇⦎#$:6ᐿᡚศா$:6UH,QYHQWUH&$3  ᅾ▴㝕ᆺ⤌⧊⿹䠈ዴఱ᭷ᩀ⟶⌮$:6ⓗᡂᮏ⤖ᵓ⯅⣔⤫ᯫᵓ
34. ㏺㯟ከつ⠊ᡈᶆ‽䠈 ᭷㎨ἲⴠᆅ䠛 ኱ᐙ᭷㎨ἲ㑂ᏲႫ䠛 
35. 6RXUFH'HY2SV'D\Vᗔ㐀ᙛӁԬ䃽65(៿䇀㜳࣑Ⲻฯ佀 
36. ᡃ಼ⓗ 65( ᐿ㋄ཎ๎䠖  ᑙὀᅾ⁁㏻ὶ⛬䚸$3,௓㠃  ┗ྍ⬟౑⏝0DQDJHG6HUYLFHV  ල㧓ⓗつ⠊⯅ὶ⛬䠈⣔⤫ᯫᵓ㏱᫂໬  ౑⏝㌾㧓ᕤ⛬䠈ㆸ65(௵ົつᶍ໬ 
37. ᢕᡚ⥺䠈 ᕥ⛣ ᚘ㛤ⓐᑵ㛤ጞ‽ഛ 
38. 
39. 3DUW 7KHDUFKLHFWXUHRI65(DVD6HUYLFH 
40. ᛠ㯟೴ ୍䚸஦௳⹦⌮⯅⟶⌮ὶ⛬ ஧䚸౑⏝㧗ྍ㠁ⓗ┘᥍⣔⤫ ୕䚸ᥦ㧗㐠⾜ᖹྎⓗྍ㠁ᗘ ᅄ䚸㝆ప⣔⤫ᯫᵓⓗ」㞯ᗘ ஬䚸ᶆ‽໬䚸つ⠊໬  ᡂ 㛗 㝵 ẁ
41. බྖⓗ᪉ྥ ⮬ᕫ㛤ⓐ䠈⮬ᕫ⥔㐠 ㉺౗㉺ከ᭹ົ䠈㉺౗㉺ከᅰ㝲䠈 㠃ᑞᕷሙⓗᨵㆰ䠈ᛌ㏿཯᠕䚹 つ⠊㉺౗㉺ከ䠈ᛠ㯟ㆸつ⠊ⴠᆅ䠛 
42. 6RXUFH⯅␿ඹᏑ$33᥎䛂㣶㣧㟁ၟ䛃ຓ㣶㣧ရ∩ᦟᨷⓒ൨᪂ၟᶵ 
43. 7KLV³6HSDUDWHG$(2DQG,(2´PRGHOVHHNVWR HVWDEOLVKD³\RXEXLOGLW\RXUXQLW´PHWKRGRORJ\  6RXUFH$:6:HOO$UFKLWHFWHG)UDPHZRUN2SHUDWLRQDO([FHOOHQFH3LOODU
44. 㝆ప$SSOLFDWLRQ7HDP౑⏝2SHUDWLRQV6HUYLFHVⓗ㛛ᷦ⯅」㞯ᗘ  㛤ⓐⓗไᗘ໬䚸ᶆ‽໬  ⥔㐠ⓗไᗘ໬䚸ᶆ‽໬ 65(DVD6HUYLFH  ㏺ ಶ ㆰ ᡂ ᶏ ᯈ ໬ ᡚ⥺ᢼ฿㏺⿹
45. ᛠ㯟೴ ୍䚸ᖹྎᴫᛕ⸛ᅯ ஧䚸0DQDJHG6HUYLFHVDV$3, ୕䚸ᢕつ⠊ㆰྍᐿసⓗつ᱁ ᅄ䚸ၿ⏝0DQDJHG6HUYLFHV≉ᛶ䠈௨.V$GPLVVLRQ:HEKRRNⅭ౛ ஬䚸᭦ከ« 
46.  ,QIUD$3,2SV5HOHQJ$:6 5HVRXUFHV*HQHUDO6HUYLFHV  6WDQGDUG%XLOG3URFHGXUH  3LSHOLQH6SHF (QJLQHHULQJ 2SHUDWLRQV 3ODWIRUP $SSOLFDWLRQV $SSOLFDWLRQ (QJLQHHULQJ $SSOLFDWLRQ 2SHUDWLRQV 0DQDJHG6HUYLFHV %  ୍䚸ᖹྎᴫᛕ⸛ᅯ $ $3365(3ODWIRUP  $:60DQDJHG6HUYLFHV &ORXG:DWFK(.6$3,*DWHZD\ «HWF  .V3ODWIRUP/RJ&RQILJ 6HFUHW  $UWLIDFW0DQDJHPHQW  &RQILJ0DQDJHPHQW  /LEUDULHV6'.  &/,7HPSODWHV
47. ஧䚸0DQDJHG6HUYLFHVDV$3,,QIUD$3, 
48. $FWLRQZRUNHU &KHFNZRUNHU 646 &KHFN4XHXH 646 &KHFN4XHXH'HDG/HWWHU )DLOHGDODUP (& 3ULYDWH 3XEOLF 3URWHFWHG $FFHVV&RQWURO (/% $.6. 'DWDEDVH 7ULJJHU:RUNHU :ULWLQJ:RUNHU 7ULJJHU  $3,*DWHZD\ 8VHU $3,.H\ ,QIUD$3,*OREDO$OHUW6\VWHP
49. ୕䚸ᢕつ⠊ㆰྍᐿసⓗつ᱁ 
50. つ⠊ ⴠᆅ *DS )HHGEDFN 
51. つ⠊ ⴠᆅ *DS )HHGEDFN  ຍ㏿
52. ㆸつ⠊ⴠᆅⓗ୕ಶゅⰍ  ᑙὀᅾศᯒ㟂ồ䠈ㆸつ⠊ㆰᡂつ᱁ⓗே D 5)&䠈ྍ௨⏝௵ព⛬ᘧㄒゝᐿ㋄䠈⏝ྠᶏⓗ᪉ἲ㦩ᨲ  ⏝ᢏ⾡ᐿ㋄つ᱁ⓗே D ᢍ6\VWHP6ROXWLRQᡈ⪅⮬ᕫᖿ,QIUD E ᑒ⿶つ᱁䠖/LEUDULHV!)UDPHZRUN!3ODWIRUP  ⟶⌮Ỵ⟇⯅㈨※༠ㄪ⪅ D ⟶⌮㐍ᗘ䚸⁁㏻༠ㄪ E ᨲ㟂ồၥ㢟䚸㈨※༠ㄪ  
53. ᐿసⓗཎ๎  6SHF)LUVW䠖ᢕ㟂ồ㎈໬ᡂྍ௨㊬ㄒゝ䚸㊬ᖹྎⓗᐿసつ᱁䠈㢮ఝ᪊ 5)&  $3,໬䠖ᑒ⿶0DQDJHG6HUYLFHVⓗ」㞯ᗘ䠈㝆ప౑⏝㛛ᷦ ż ᑒ⿶$:6ᖖ⏝᭹ົ䠈ീ᫝(&㛤㜝ᶵ䚸ୖୗ(/%䚸6HUYLFH≪ែ  6'./LE)UDPHZRUN䠖⤥$SSOLFDWLRQ௓᥋6ROXWLRQⓗᶆ‽௓㠃 ż &RQILJ䚸/RJ䚸6HFUHW䚸1HWZRUN䚸6HFXULW\䚸6WDQGDUG'DWD0RGHO  7RROV䠖 ż &/,䠖ໟྵ662䚸&RGH7HPSODWHV䚸%XLOG6SHF«HWF ż 7HPSODWHV&RGH7HPSODWHV 1(73+3QRGHMV«HWF 䚸3LSHOLQH7HPSODWH䚸 2EVHUYDWLRQ7HPSODWHV䚸$UFKLWHFW7HPSODWHV ,D& ż ,'(LOL]H䠖96&RGH([WHQVLRQ 
54. *DS つ⠊ ⴠᆅ  つ᱁ 6ROXWLRQ 6'. &/, 7HPSODWHV 0DQDJHG6HUYLFHV 'HVLJQ6SHF
55. Ɣ $GPLVVLRQ&RQWUROOHU᭳ᨛᡖ⥂㐣DXWKHQWLFDWLRQ DXWKRUL]DWLRQஅᚋⓗ$3,UHTXHVW  ᅄ䚸ၿ⏝0DQDJHG6HUYLFHV≉ᛶ䠖.V$GPLVVLRQ:HEKRRN
56.  /RJ6KLSSLQJE\6LGHFDU3DWWHUQ
57. 
58.  ⣔⤫ᯫᵓつ⠊ $UFK&RQYHQWLRQ  ᭹ົ┠㗴 6HUYLFH&DWDORJ  ᭹ົⓐ⌧ 6HUYLFH'LVFRYHU\  ⭘ฟ≀⟶⌮ $UWLIDFW0DQDJHPHQW  㓄⨨⟶⌮ &RQILJ0DQDJHPHQW  ᐦ☞䚸ᐦ㛋⟶⌮ 6HFUHW0DQDJHPHQW ஬䚸㛤ⓐ⥔㐠἞⌮   ㈨※⯅⎔ቃ㓄⨨ 3URYLVLRQLQJ  ᘓ⨨ᶆ‽໬ %XLOG6SHF  㒊⨫ᶆ‽໬ 'HSOR\6SHF  㛤ⓐὶỈ⥺ 3LSHOLQH  や 䠖᪥ㄅ⯅┘᥍ᣦᶆ  2EVHUYDWLRQ  ┘᥍䠖┘᥍ᖹྎ⯅൤⾲ᯈ  0RQLWRULQJ  ඲ᇦ࿌㆙⣔⤫ *OREDO$OHUW6\VWHP  ␗ᖖ⹦⌮⯅⟶⌮ ,QFLGHQW0DQDJPHQW
59. 6WDQGDUG%XLOG3URFHGXUHV6SHF  3DFNB)ODJV 3URFHGXUHV1<3 6RXUFH&RGH %XLOG 3DFN 8SORDG $UWLIDFW6LWH 5HOHQJ$3, 6LWHV,QIUD7HDP 8VHU'HILQH7HDP %XLOGB)ODJV 8SORDGB)ODJV 0DQLIHVW 5HOHQJ 5HVRXUFHV7HDP &XVWRP %XLOG-RE (QY9DUV XQLYHUVDOEXLOGVSHFIRUGRFNHUQXJHWQSPUDZ«HWF
60. 6WDQGDUG'HSOR\PHQW6SHF      7HVW 3URGXFWLRQ %XLOG GHY  SURYLVLRQ %XLOG GHY  SURYLVLRQ %XLOG GHY  SURYLVLRQ %XLOG GHY  SURYLVLRQ W VHFUHW FRQILJ VHWWLQJV VHFUHW FRQILJ VHWWLQJV VHFUHW FRQILJ VHWWLQJV VHFUHW FRQILJ VHWWLQJV GHSOR\ GHSOR\ GHSOR\ GHSOR\ %XLOG UHO  SURYLVLRQ VHFUHW FRQILJ VHWWLQJV GHSOR\ SURYLVLRQ VHFUHW FRQILJ VHWWLQJV GHSOR\ SURYLVLRQ VHFUHW FRQILJ VHWWLQJV GHSOR\ SURYLVLRQ VHFUHW FRQILJ VHWWLQJV GHSOR\
61. 
62.  aQFOLGHSOR\^HQYLURQPHQW[`^EOXHJUHHQIORZ` (QYLURQPHQWV 6HFUHW &RQILJ (QY9DUV )ORZV 3UHSURFHGXUH 7HPSODWHUHQGHULQJ 'HSOR\SURFHVV 3RVWSURFHGXUH
63.  ⣔⤫ᯫᵓつ⠊ $UFK&RQYHQWLRQ  ᭹ົ┠㗴 6HUYLFH&DWDORJ  ᭹ົⓐ⌧ 6HUYLFH'LVFRYHU\  ⭘ฟ≀⟶⌮ $UWLIDFW0DQDJHPHQW  㓄⨨⟶⌮ &RQILJ0DQDJHPHQW  ᐦ☞䚸ᐦ㛋⟶⌮ 6HFUHW0DQDJHPHQW ᚇከ«   ㈨※⯅⎔ቃ㓄⨨ 3URYLVLRQLQJ  ᘓ⨨ᶆ‽໬ %XLOG6SHF  㒊⨫つ᱁ 'HSOR\6SHF  㛤ⓐὶỈ⥺ 3LSHOLQH  や 䠖᪥ㄅ⯅┘᥍ᣦᶆ  2EVHUYDWLRQ  ┘᥍䠖┘᥍ᖹྎ⯅൤⾲ᯈ  0RQLWRULQJ  ඲ᇦ࿌㆙⣔⤫ *OREDO$OHUW6\VWHP  ␗ᖖ⹦⌮⯅⟶⌮ ,QFLGHQW0DQDJPHQW つ⠊!つ᱁!6ROXWLRQ 7RROV
64. 㝆ప$SSOLFDWLRQ7HDP౑⏝2SHUDWLRQV6HUYLFHVⓗ㛛ᷦ⯅」㞯ᗘ  㛤ⓐⓗᶆ‽໬䠖&RQILJ䚸/RJ䚸6HFUHW䚸1HWZRUN䚸6HFXULW\«HWF  ⥔㐠ⓗᶆ‽໬䠖%XLOG䚸3LSHOLQH䚸'HSOR\PHQW䚸2EVHUYDWLRQ«HWF 65(DVD6HUYLFH 
65. 
66. ⤖ㄽ⯅᦬せ 
67. 6RXUFH(QWHUSULVH5RDGPDSWR65(+RZWREXLOGDQGVXVWDLQDQ65()XQFWLRQ )LJXUH7KHS\UDPLGVRIUHOLDELOLW\  )URP6FDOHXSWR6FDOHRXW
68. බྖⓗつᶍ 65(7HDPⓗつᶍ 65(7HDP(QJLQHHUV/HDGHU   ┘᥍䚸஦௳⟶⌮䚸⣔⤫ྍ㠁ᗘ   ⣔⤫ྍ㠁ᗘ䚸6/2䚸㝆ప⣔⤫」㞯ᗘ   '5䚸つ⠊໬䚸ᶆ‽໬䚸$3,   &RVW䚸ᖹྎ໬䚸つᶍ໬ 
69.  ,QIUDVWUXFWXUH &RPSXWH_1HWZRUN_ 6WRUDJH0LGGOHZDUH_ 5XQWLPH_'DWD2SHUDWLRQV _6HFXULW\ 'HSOR\2SHUDWHDQG 0DQDJH $OODFWLYLWLHVQHHGHGWR GHSOR\DQGVXSSRUW SODWRIUPLQIUDVWUXFWXUHDQG DSSOLFDWLRQVLQSURFXGWLRQ 'HYHORS%XLOGDQG7HVW $OODFWLYLWLHVQHHGHGWR GHILQHDQGYDOLGDWH3ODWIRUP ,QIUDVWUXFWXUHRU%XVLQHVV $SSOLFDWLRQV (QJLQHHULQJ 2SHUDWLRQV 3ODWIRUP $SSOLFDWLRQV $SSOLFDWLRQ (QJLQHHULQJ $SSOLFDWLRQ 2SHUDWLRQV 3ODWIRUP (QJLQHHULQJ 3ODWIRUP 2SHUDWLRQV %XVLQHVV6RIWZDUH &XVWRP'HYHORSHGRU &RPPHUFLDORIIWKH6KHOI 6RXUFH$:6:HOO$UFKLWHFWHG)UDPHZRUN2SHUDWLRQDO([FHOOHQFH3LOODU
70. (QJLQHHULQJ 2SHUDWLRQV 3ODWIRUP $SSOLFDWLRQV $SSOLFDWLRQ (QJLQHHULQJ $SSOLFDWLRQ 2SHUDWLRQV 0DQDJHG6HUYLFHV  $3365(3ODWIRUP $3365(3ODWIRUP $3365(3ODWIRUP 0DQDJHDEOH 6FDODEOH &RVW2SWLPL]HG 'HIDXOW5HOLDELOLW\ (DV\2SHUDWLRQDO 6HFXULW\'1$ %XVLQHVV6RIWZDUH &XVWRP'HYHORSHGRU &RPPHUFLDORIIWKH6KHOI &ORXG3URYLGHUV &RVW 5HOLDELOLW\ 3HUIRUPDQFH &RPSOLDQFH 6HFXULW\ )OL[DEOH 'HYHORS%XLOGDQG7HVW $OODFWLYLWLHVQHHGHGWR GHILQHDQGYDOLGDWH3ODWIRUP ,QIUDVWUXFWXUHRU%XVLQHVV $SSOLFDWLRQV
71.  (QWHUSULVHURDGPDSWR65(  $:6:HOO$UFKLWHFWHG)UDPHZRUN2SHUDWLRQDO([FHOOHQFH3LOODU ཨ⪃㈨ᩱ 
72. ┦㜝㈨ᩱ  ῦㄯ⣔⤫┘᥍⯅&ORXG:DWFKⓗ᠕⏝$:68VHU*URXS7DLZDQ  ᚘ⥭ᛴ஦௳ㄯ65(᠕ㆰ⬟ຊⓗᇵ㣴'HY2SV'D\V7DLSHL  2SVDV&RGHXVLQJ6HUYHUOHVVL7+RPH  㑌ྥ$3,⥂⃽$3,*DWHZD\ᑟධஅ᪑$:66XPPLW  ⪵⪵㌾㧓஺௜ⓗ℃゗ㄯ⭘ฟ≀⟶⌮'HY2SV0HHWXS  ᚤ᭹ົⓗᇶ♏ᘓタ6HUYLFH'LVFRYHU\$QGUHZ:X'HY2SV'D\V7DLSHL  ᚘ㞽㛤ጞⓗ&RQILJXUDWLRQ0DQDJHPHQW/HYL&KHQ 'HY2SV'D\  㞽၎ရ∩⥂⇯䛂㟁ၟ䛃ヱᢕ㈨※ᢞධᅾီ⿹䠛 5LFN+ZDQJ  ⅏㞴₇⦎#$:6ᐿᡚศா$:6UH,QYHQWUH&$3  ᅾ▴㝕ᆺ⤌⧊⿹䠈ዴఱ᭷ᩀ⟶⌮ $:6ⓗᡂᮏ⤖ᵓ⯅⣔⤫ᯫᵓ$:66XPPLW 
73.  KWWSVZZZIDFHERRNFRPJURXSVVUHWDLZDQ
74. 7KDQN\RX  ᤲᥥ45&2'(ሸᑃၥ༹ 㡿$:6&UHGLW⨾ඖ ච㈝ᘓ❧$:6ᖒ⹰ ༶ྍ⮳ᨦ఩㡿ྲྀ㝈㔞ዲ⚰