Slide 1

Slide 1 text

Kubernetes ӡ༻ઃܭΨΠυ Japan Container Days v1804, Apr 19, 2018

Slide 2

Slide 2 text

@spesnova SRE at Mercari,Inc. / Kubernetes tokyo community organizer

Slide 3

Slide 3 text

ࠓ೔ͷςʔϚ ˞͜͜Ͱड़΂Δ಺༰͸ॴଐ૊৫ͷެࣜݟղͰ͸ͳ͘ݸਓͷݟղͰ͢

Slide 4

Slide 4 text

Kubernetes ΛͲ͏࢖ͬͨΒ͍͍͔໎ͬͯΔํ λʔήοτ

Slide 5

Slide 5 text

Kubernetes Λ࢖͏໨తΛ໌֬ʹ͠Α͏ ໌֬ͳ໨తΛ࣋ͬͯར༻͢Ε͹ɺ ɹͲ͏࢖͑͹ྑ͍͔ࣗવʹݟ͑ͯ͘Δ

Slide 6

Slide 6 text

໌֬ͳ໨త(ઃܭํ਑)Λϕʔεʹ Kubernetes Λ Ͳ͏࢖͏͔ઃܭ͍ͯ͘͠ྫΛࣔ͢ ςʔϚ

Slide 7

Slide 7 text

ࣗ཯తͳνʔϜͱγεςϜΛ࡞Δ ໨త (ઃܭํ਑)

Slide 8

Slide 8 text

ࣗ཯తͱ͸ ͦͷ΋ͷࣗମ͚ͩͰௐ੔ΛߦͬͨΓɺ ໰୊Λղܾͨ͠ΓͳͲΛߦ͏͞· Ҿ༻IUUQTXXXXFCMJPKQDPOUFOUࣗ཯త

Slide 9

Slide 9 text

ͳͥࣗ཯త͔ʁ ྑ͍ϓϩμΫτΛ࡞ΔͨΊͷॏཁͳཁૉͩͱߟ͍͑ͯΔ͔Βɻ ࣗ཯తͳνʔϜ΍γεςϜ͸ͦ͏Ͱͳ͍৔߹ͱൺֱͯ͠ɺΑΓ଎͘ಈ ͘͜ͱ͕ՄೳͰɺΠϊϕʔγϣϯ΋ى͖΍͍͢ͱײ͍ͯ͡Δɻ

Slide 10

Slide 10 text

ࣗ཯తͳνʔϜͷྫ: ΞϝϦΧ܉ ਆग़َ຅ͳςϩ΍ήϦϥΛલʹͯ͠ɺΞϝϦΧ܉͸ϐϥϛουܕ૊৫ʹݶ քΛײ͡ɺωοτϫʔΫܕ૊৫ʹҠߦɻ্૚෦ʹҙࢥܾఆΛڼ͕ͣɺ࠷લ ઢ෦ୂ͕ࣗ਎Ͱҙࢥܾఆͯ͠ಈ͘͜ͱͰςϩͷεϐʔυײʹରԠͨ͠ɻ ࢀߟ5&".0'5&".4

Slide 11

Slide 11 text

ςϩ 1. ςϩΛݕ஌ 3. ҙࢥܾఆ 2. ্૚෦΁఻ୡ 5. ߦಈ 4. ݱ৔΁఻ୡ ϐϥϛουܕɺதԝूݖܕͷҙࢥܾఆ ࢀߟ5&".0'5&".4

Slide 12

Slide 12 text

ςϩ 3. ߦಈͱ৘ใڞ༗ ωοτϫʔΫܕɺ෼ࢄܕͷҙࢥܾఆ 2. ҙࢥܾఆ ࢀߟ5&".0'5&".4 1. ςϩΛݕ஌

Slide 13

Slide 13 text

ࣗ཯తͳγεςϜͷྫ: Netflix AWS EC2 ͷϝϯςφϯεϦϒʔτΛࣗಈ෮چͷ࢓૊ΈʹΑͬͯθϩ μ΢ϯλΠϜͰ৐Γ੾ͬͨɻ͜Ε͸Ϧϒʔτ͕͔͔Δ౓ʹ OnCall ୲ ౰ऀʹి࿩͕໐ΓɺखಈͰ෮چ͢ΔΑΓ΋଎͍ɻ ࢀߟIUUQTXXXTMJEFTIBSFOFUQMBOFUDBTTBOESBOFUqJYBTUBUFPGYFODIBPTNPOLFZDBTTBOESB

Slide 14

Slide 14 text

଎͞ͱ͍͏ͷ͸ٸ͙͜ͱ͔ΒͰ͸ͳ͘ɺ Կ͔Λແ͘͢͜ͱ͔Βੜ·ΕΔ ଎͘ಈ͘

Slide 15

Slide 15 text

Կ͔Λແͯ͘͠଎͘ͳͬͨྫ • ΞϝϦΧ܉ͷྫ: ্૚෦ͷҙࢥܾఆ଴ͪΛແ͘͢ • Netflix ͷྫ: ਓ΁ͷ࿈བྷͱख࡞ۀΛແ͘͢ • αʔόͷϨεϙϯελΠϜ: DB ΁ͷ໰͍߹ΘͤΛݮΒ͢ • Amazon ͓ٸ͗ศ: ૔ݿΛ૿΍ͯ͠༌ૹڑ཭ΛݮΒ͢

Slide 16

Slide 16 text

Kubernetes ࣗମͷૂ͍ͱζϨ͍ͯͳ͍͔ʁ Kubernetes is more than just a “container orchestrator”. It aims to eliminate the burden of orchestrating physical/ virtual compute, network, and storage infrastructure, and enable application operators and developers to focus entirely on container-centric primitives for self-service operation. Kubernetes Design and Architecutre Ҿ༻IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE

Slide 17

Slide 17 text

Kubernetes ࣗମͷૂ͍ͱζϨ͍ͯͳ͍͔ʁ 1. Portable 2. General-Purpose 3. Meet users partway 4. Flexible 5. Extensible 6. Automatable 7. Advance the state of the art Kubernetes Design and Architecutre Ҿ༻IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE

Slide 18

Slide 18 text

Kubernetes ࣗମͷૂ͍ Ҿ༻IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE 1. Πϯϑϥ؅ཧͷͨΊͷख࡞ۀΛܶతʹݮΒ͢͜ͱ 2. ηϧϑαʔϏεܕͷӡ༻ΛՄೳʹ͢Δ͜ͱ

Slide 19

Slide 19 text

ࣗ཯తͳνʔϜͱγεςϜΛ࡞ΔͨΊʹ 1. γεςϜͷந৅Խɺૄ݁߹Խɺ࡞ۀͷࣗಈԽ 2. ڞ௨໨తͷ໌֬Խɺద੾ͳ੹຿ͷ෼ղͱఆٛɺݖݶҕৡ

Slide 20

Slide 20 text

νʔϜͷઃܭ

Slide 21

Slide 21 text

ͳͥνʔϜͷઃܭ͔Βʁ 1. ٕज़ͱ૊৫͸දཪҰମ 2. Kubernetes ͸ਓͷͨΊʹɺਓʹΑͬͯ࡞ΒΕɺਓʹΑͬ ͯӡ༻͞ΕΔɺ࢖͏ଆ΋ਓʹ͍ͭͯߟ͑Δඞཁ͕͋Δ 3. ૊৫จԽͱΠϯϑϥΛ྆ྠͱͯ͠ಉ࣌ʹม͍͑ͯͬͯॳ ΊͯޮՌ͕ग़Δ

Slide 22

Slide 22 text

“૊৫ͷઃܭ͢ΔγεςϜ͸ɺͦͷ૊৫ͷ ɹɹίϛϡχέʔγϣϯߏ଄Λͦͷ··൓өͨ͠ઃܭʹͳΔ” ίϯ΢ΣΠͷ๏ଇ

Slide 23

Slide 23 text

։ൃڌ఺͕ 3 ͭ͋Δͱιʔείʔυ΋ 3 ͭʹ෼͔ΕΔ ։ൃڌ఺ ιʔείʔυ

Slide 24

Slide 24 text

։ൃڌ఺͕ 3 ͭ͋Δͱιʔείʔυ΋ 3 ͭʹ෼͔ΕΔ ։ൃڌ఺ ιʔείʔυ

Slide 25

Slide 25 text

։ൃڌ఺͕ 1 ͭͩͱւ֎ల։ͯ͠΋ιʔείʔυ͸ 1 ͭ ։ൃڌ఺ ιʔείʔυ αʔϏε

Slide 26

Slide 26 text

“࡞Γ͍ͨγεςϜͷߏ଄Λ൓өͨ͠ίϛϡχέʔγϣϯ͓Αͼ ɹɹ૊৫ߏ଄Λ·ͣ࡞ΔͱɺγεςϜ͕ظ଴ͨ͠ઃܭʹͳΔ” ίϯ΢ΣΠͷ๏ଇΛٯखʹऔΔ

Slide 27

Slide 27 text

Kubernetes ͷੈքͰ͸γεςϜ͸ 2 छྨʹେผͰ͖Δ Πϯϑϥڞ௨ج൫ܥ ΞϓϦέʔγϣϯܥ

Slide 28

Slide 28 text

Πϯϑϥڞ௨ج൫ܥ ΞϓϦέʔγϣϯܥ ΞϓϦέʔγϣϯͱڞ௨ج൫Λີ݁߹ͤͯ͞͸͍͚ͳ͍

Slide 29

Slide 29 text

2 छྨͷνʔϜΛ࡞Δ ΫϥελΞυϛϯνʔϜ ϓϩμΫτνʔϜ

Slide 30

Slide 30 text

੹೚ൣғͷઃܭ

Slide 31

Slide 31 text

γεςϜ͝ͱʹඞཁͳΤϯδχΞϦϯά࡞ۀ͕͋Δ Πϯϑϥڞ௨ج൫ܥ ΞϓϦέʔγϣϯܥ

Slide 32

Slide 32 text

• Ϋϥελͷϝϯςφϯε • ڞ௨ίϯϙʔωϯτͷϝϯςφϯε • σϓϩΠύΠϓϥΠϯ • ΫϥελϨϕϧͷϞχλϦϯά • ΫϥελϨϕϧͷηΩϡϦςΟ etc • ΞϓϦέʔγϣϯίʔυ • ςετ • ίϯςφԽ • σϓϩΠ • ΞϓϦέʔγϣϯϨϕϧͷϞχλϦϯά etc ΞϓϦέʔγϣϯܥ Πϯϑϥڞ௨ج൫ܥ γεςϜ͝ͱʹඞཁͳΤϯδχΞϦϯά࡞ۀ͕͋Δ

Slide 33

Slide 33 text

ࣗ཯తʹಈͨ͘Ίɺૄ݁߹ʹͳΔ੹೚ڥքΛܾΊΔ Ϋϥελ ίϯςφ ϊʔυ ιʔείʔυ

Slide 34

Slide 34 text

ϓϩμΫτνʔϜͷ੹೚ൣғ ސ٬ͷ՝୊Λղܾ͢Δ͜ͱɻͦͷͨΊʹϓϩμΫτͷاը͔Β σβΠϯɺ։ൃɺӡ༻·Ͱશͯʹ੹೚Λ࣋ͭɻ

Slide 35

Slide 35 text

ΫϥελΞυϛϯνʔϜͷ੹೚ൣғ ϓϩμΫτνʔϜͷύϑΥʔϚϯεΛ࠷େԽ͢Δ͜ͱɻͦͷͨ ΊʹɺσϓϩΠύΠϓϥΠϯ΍ϞχλϦϯάͳͲͷϓϩμΫτ νʔϜΛࢧ͑Δڞ௨ج൫ͷ։ൃ͔Βӡ༻·Ͱ੹೚Λ࣋ͭɻ

Slide 36

Slide 36 text

“You build it, you run it”

Slide 37

Slide 37 text

Ϋϥελͷઃܭ

Slide 38

Slide 38 text

։ൃ؀ڥɺຊ൪؀ڥ͝ͱʹΫϥελΛ࡞Δʁ Development Ϋϥελ Ϋϥελ Production

Slide 39

Slide 39 text

1. ࠷΋Ұൠతͳཻ౓ 2. εςʔδϯά؀ڥ͕ඞཁʹͳͬͨΒʁ QA ؀ڥ͸ʁ 3. ؀ڥ͕૿͑ΔͨͼʹΫϥελ͕૿͑ͯ؅ཧίετ্͕͕Δ 4. ։ൃ؀ڥͱຊ൪؀ڥ͕Ұகͯ͠Δ͜ͱ͕อূͮ͠Β͍ ։ൃ؀ڥɺຊ൪؀ڥ͝ͱʹΫϥελΛ࡞Δʁ

Slide 40

Slide 40 text

Ϧʔδϣϯ͝ͱʹ 1 ͚ͭͩΫϥελΛ࡞Δ London Ϋϥελ Tokyo Ϋϥελ California Ϋϥελ

Slide 41

Slide 41 text

Ϧʔδϣϯ͝ͱʹ 1 ͚ͭͩΫϥελΛ࡞Δ London Ϋϥελ Development Production Staging BranchLab QA Sandbox etc

Slide 42

Slide 42 text

Ϧʔδϣϯ͝ͱʹ 1 ͚ͭͩΫϥελΛ࡞Δ 1. ͋ΒΏΔ؀ڥΛड͚ೖΕΒΕΔ 2. ։ൃ؀ڥͱຊ൪؀ڥ͕Ұகͯ͠Δ͜ͱ͕อূ͠΍͍͢ 3. ϓϩμΫτνʔϜ͕ΫϥελΛҙࣝ͠ͳ͍͍ͯ͘ (ந৅Խ) 4. ։ൃ؀ڥ͕ຊ൪؀ڥʹѱӨڹΛ༩͑ͳ͍͔ෆ҆ʁ

Slide 43

Slide 43 text

؀ڥΛಛผࢹ͠ͳ͍ Service A Development Service B Production Service A Production Service B Production ։ൃ؀ڥͱຊ൪؀ڥΛ෼͚ͨͱͯ͠΋ɺ͋ΔαʔϏε͕͋ΔαʔϏεʹӨڹΛ༩͑ ͳ͍Α͏ʹ͠ͳ͍ͱ͍͚ͳ͍͜ͱʹ͸มΘΓ͸ͳ͍ɻͩͱ͢Ε͹։ൃ؀ڥͱຊ൪ ؀ڥΛࠞࡏͤͯ͞΋ಉ͡Ͱ͋Δ

Slide 44

Slide 44 text

Ϧʔδϣϯ͝ͱʹ 1 ͚ͭͩΫϥελΛ࡞Δ 1. AWS, GCP, Heroku ʹ։ൃ؀ڥઐ༻૭ޱ͸ͳ͍ 2. Ϣʔβʔ͕”։ൃ؀ڥ༻”ͱͯ͠ΞΧ΢ϯτΛ࡞͍ͬͯΔ͚ͩ 3. Google ΋ GitHub ΋ Cluster per region Ͱ͋Δ 4. ηΩϡΞͳαʔϏε΋ಉ͡ΫϥελʹೖΕΔͷ͔ʁ

Slide 45

Slide 45 text

ϓϩμΫτɺαʔϏε͝ͱʹΫϥελΛ෼͚Δʁ 1. ؀ڥ͝ͱʹΫϥελΛ࡞ΔΑΓ΋Ϋϥελ͕૿͑ͯ͠·͏ 2. ৴པͰ͖ͳ͍ୈࡾऀʹར༻ͯ͠΋Β͏౳ͷέʔε͸෼͚Δ΂͖ 3. ࣾ಺ʹ͋ΔଞͷαʔϏεͱશ͘௨৴͠ͳ͍ͳΒ͹༗Γ 4. ࣾ಺ʹ͋ΔଞͷαʔϏεͱશ͘௨৴͢Δ৔߹͸ʁ

Slide 46

Slide 46 text

ϓϩμΫτɺαʔϏε͝ͱʹΫϥελΛ෼͚Δʁ Secure Ϋϥελؒ௨৴ Service A Ϋϥελ಺௨৴ Secure Service A Default Network Policy ΍ Istio ͳͲͷଘࡏʹΑͬͯݱ࣌఺Ͱ͸Ϋϥελ಺௨৴ͷํ͕ωο τϫʔΫͷ੍ޚ͕͠΍͍͢ɻ

Slide 47

Slide 47 text

ಉ͡ϊʔυʹࡌ͍ͬͯΔ͜ͱ͕ґવͱͯ͠໰୊ Secure Ϋϥελ ϊʔυ Service A ίϯςφ

Slide 48

Slide 48 text

Ϋϥελ͸ 1ͭͷ··Ͱઐ༻ϊʔυΛ༻ҙ͢Δ Ϋϥελ ϊʔυ Service A Secure

Slide 49

Slide 49 text

ηΩϡΞʹ͍ͨ͠ίϯςφΛઐ༻ϊʔυʹ഑ஔ Ϋϥελ ϊʔυ Service A Secure

Slide 50

Slide 50 text

Ϋϥελͷઃܭ·ͱΊ 1. Ϧʔδϣϯ͝ͱʹ 1 ͚ͭͩΫϥελΛ࡞Δ 2. ؀ڥ͸Ϋϥελ಺෦ͷ isolation ٕज़ʹΑͬͯ෼཭͢Δ 3. ಛఆͷαʔϏεઐ༻ϊʔυ͸ຊ౰ʹඞཁͳ࣌ʹ͚ͩ༻ҙ͢Δ 4. ϓϩμΫτ/αʔϏε͝ͱʹΫϥελΛ࡞Δͷ͸࠷ޙͷखஈ

Slide 51

Slide 51 text

Namespace ͷઃܭ

Slide 52

Slide 52 text

Namespace ͰόʔνϟϧΫϥελ͕࡞੒Ͱ͖Δ Ϋϥελ όʔνϟϧΫϥελ

Slide 53

Slide 53 text

1 ͭͷΫϥελͰ͋ΒΏΔ؀ڥΛड͚ೖΕΔ London Ϋϥελ Development Production Staging BranchLab QA Sandbox etc

Slide 54

Slide 54 text

؀ڥ͝ͱʹ Namespace Λ෼͚Δ dev prod qa

Slide 55

Slide 55 text

Service A Development Service B Production Service A Production Service B Production ؀ڥ͚ͩͰͳ͘αʔϏε΋෼཭͍ͨ͠

Slide 56

Slide 56 text

αʔϏε໊+؀ڥ͝ͱʹ Namespace Λ෼͚Δ A-dev A-prod B-qa

Slide 57

Slide 57 text

Network Policy ͷઃܭ

Slide 58

Slide 58 text

Service A Development Service B Production Service A Production Service B Production Network Policy Ͱ Pod ؒͷ௨৴੍͕ޚͰ͖Δ

Slide 59

Slide 59 text

Namespace ϨϕϧͰ੍ޚ͢Δ A-prod ϙϦγʔ͕ෳࡶʹͳΓա͗ΔͷΛ๷͙ɻجຊతͳར༻έʔε͸αʔϏεؒͷ௨৴ Λ੍ޚͳͷͰɺαʔϏε͝ͱʹ Namespace Λ࡞͍ͬͯΔͷ͕׆͖ͯ͘Δɻ B-prod

Slide 60

Slide 60 text

جຊ͸ All Deny B-prod A-prod C-qa B-dev

Slide 61

Slide 61 text

ϗϫΠτϦετͰ௨৴Մೳͳ Namespace Λࢦఆ A-prod B-prod B-dev C-qa D-prod

Slide 62

Slide 62 text

RBAC ͷઃܭ

Slide 63

Slide 63 text

RBAC Ͱ Kubernetes ͷݖݶͷ؅ཧ͕Ͱ͖Δ • Deployment ͷ࡞੒ • Secrets ͷӾཡ • PVC ͷ࡟আ ϢʔβʔΞΧ΢ϯτ foo Role RoleBinding

Slide 64

Slide 64 text

RBAC Λݖݶҕৡʹར༻͢Δ Ϋϥελ Namespace “admin” Role ϓϩμΫτνʔϜ A Namespace A ΫϥελΞυϛϯνʔϜ “custom-cluster-admin” Role

Slide 65

Slide 65 text

Namespace Admin Role 1. ϓϦηοτͷ “admin” ΛϓϩμΫτνʔϜʹ෇༩ 2. ಛఆͷ Namespace ҎԼͷ؅ཧݖݶ ΛϓϩμΫτνʔϜʹݖݶҕৡ 3. ϓϩμΫτνʔϜଆͰඞཁʹԠͯ͡ edit(read-write) ΍ view(read-only) Λ࡞੒ 4. ૊৫తʹ੹೚ൣғΛ໌֬ʹఆٛ͢Δ͜ͱͱɺγεςϜతʹͦΕΛදݱ͢Δ͜ͱ ͸྆ྠɺͲͪΒ͕͚ܽͯ΋͍͚ͳ͍

Slide 66

Slide 66 text

Custom Cluster Admin Role 1. ϓϦηοτͷ “cluster-admin” ͸ԿͰ΋ग़དྷͯ͠·͏ 2. ݖݶΛ࣋ͭ͜ͱ͸ಉ࣌ʹ੹೚Λ࣋ͭ͜ͱΛҙຯ͢Δ 3. “cluster-admin” ͔ΒݖݶΛམͱͨ͠ “custom-cluster-admin” Λ༻ҙ 4. ڞ௨ج൫ʹར༻͢Δ namespace ΍ node ؅ཧʹඞཁͳݖݶΛ෇༩ 5. αʔϏε/ϓϩμΫτ༻ namespace ͸ secrets Λআ͖ view ݖݶΛ෇༩ 6. ͋͘·ͰΫϥελͷ؅ཧʹప͠ɺϓϩμΫτʹؔ͢Δ෦෼͸೚ͤΔ 7. ϓϩμΫτͷ৴པੑͷ୲อ͕ඞཁͳ৔߹ɺSRE ͸ΫϥελΞυϛϯͰ ͸ͳ͘ϓϩμΫτνʔϜʹॴଐͯ͠׆ಈ͢Δ

Slide 67

Slide 67 text

ΞϓϦέʔγϣϯίϯςφͷઃܭ

Slide 68

Slide 68 text

(Deployment / Service ౳ͷઃܭ)

Slide 69

Slide 69 text

1. ࣗ཯తͳΞϓϦέʔγϣϯίϯςφΛ໨ࢦ͢ 2. ίϯςφͷҟৗऴྃɺϊʔυো֐΍ϊʔυϝϯςφϯε࣌ʹඋ͑ͯࣗಈ෮چ 3. ෛՙͷ૿ݮʹඋ͑ͯࣗಈεέʔϧ 4. ͱ͸͍͑׬શʹશࣗಈ͸೉͍͠ͷͰ࣮֬ʹखಈ࡞ۀ͸ඞཁʹͳΔ 5. खಈ࡞ۀָ͕ʹͳΔΑ͏ʹ΋ߟྀ ΞϓϦέʔγϣϯίϯςφͷઃܭ

Slide 70

Slide 70 text

1. Observable: ίϯςφ͕ਖ਼ৗ͔ҟৗ͔൑ผ͕͚ͭΒΕΔ͔ɺ໰୊ൃੜ࣌ʹݪҼڀ໌͕Ͱ͖Δ͔ 2. Disposable: ҟৗऴྃͨ͠ίϯςφ΍ো֐தͷϊʔυ্ʹ͋ΔίϯςφΛ͙͢ʹࣺͯΒΕΔ͔ 3. Immutable: ϩʔϧόοΫ΍εέʔϧ࣌ʹಉ͡ίϯςφ͕ىಈ͢Δ͜ͱΛอূͰ͖Δ͔ 4. Scalable: ϩʔυςετΛܦͯεέʔϧͷ͖͍͠஋͕ఆ·͍ͬͯΔ͔ 5. Loosely Coupled: σϓϩΠɺϩʔϧόοΫɺεέʔϧ࣌ʹґଘؔ܎Λߟྀ͠ͳͯ͘ࡁΉ͔ 6. Graceful: ѱӨڹΛग़ͣ͞ʹىಈɺఀࢭ͕Ͱ͖Δ͔ ࣗಈ / खಈ෮چɺࣗಈ / खಈεέʔϧʹඞཁͳཁૉ

Slide 71

Slide 71 text

1. Liveness Probe ͷར༻ 2. Readiness Probe ͷར༻ 3. ϩά ͷऩू 4. ϝτϦΫεͷऩू 5. τϨʔγϯά ᶃ Observable ϩά ϝτϦΫε τϨʔε ϔϧενΣοΫ ίϯςφ

Slide 72

Slide 72 text

Liveness Probe Liveness Probe Ͱ͸ϔϧενΣοΫʹ௨Βͳ͍৔߹ Kubernetes ͕ Pod (ίϯς φ) Λ࠶ىಈ͢ΔɻͦͷͨΊɺΞϓϦέʔγϣϯ͕ਖ਼ৗʹىಈ͔ͨ͠Λ൑ผ͢Δͨ Ίʹར༻͢Δɻٯʹݴ͏ͱɺͲ͏͍͏ঢ়گͰࣗಈ࠶ىಈ͍͔ͤͨ͞Λදݱ͢Δ৔ ॴͰ͋ΓɺKubernetes ʹඋΘ͍ͬͯΔࣗಈ෮چͷ࢓૊ΈͷҰ෦Ͱ͋Δɻϔϧε νΣοΫͷਫ਼౓͕؁͍ͱෆඞཁʹ࠶ىಈͯ͠͠·͏ॾਕͷ݋ͳ໘΋͋Δɻ

Slide 73

Slide 73 text

Liveness Probe ΞϓϦέʔγϣϯίϯςφ ϔϧενΣοΫ ΞϓϦέʔγϣϯίϯςφ ϔϧενΣοΫ New!

Slide 74

Slide 74 text

Readiness Probe Readiness Probe Ͱ͸ϔϧενΣοΫʹ௨ͬͨ৔߹ Kubernetes ͕ͦͷ Pod(ίϯ ςφ)Λ Service (ϩʔυόϥϯα) ʹొ࿥͢ΔɻͦͷͨΊɺσʔλϕʔε઀ଓͳͲ ΋ؚΊͯΞϓϦέʔγϣϯ͕ਖ਼ৗʹϨεϙϯεΛฦͤΔঢ়ଶ = ४උ͕Ͱ͖͔ͨ (Ready)Λ൑ผ͢ΔͨΊʹར༻͢Δɻ

Slide 75

Slide 75 text

Readiness Probe ΞϓϦέʔγϣϯίϯςφ ϔϧενΣοΫ ΞϓϦέʔγϣϯίϯςφ ϔϧενΣοΫ σʔλϕʔε σʔλϕʔε Service(ϩʔυόϥϯα) Service(ϩʔυόϥϯα)

Slide 76

Slide 76 text

1. εςʔτϨεʹ͢Δ 2. σʔλ͸ӬଓԽετϨʔδʹ 3. ϩά͸ JSON ܗࣜͰඪ४ग़ྗʹ ᶄ Disposable ίϯςφ ϩά DB σʔλ ඪ४ग़ྗ

Slide 77

Slide 77 text

1. Latest tag ͸࢖Θͳ͍ (ྫ: hello:1.0.1) 2. ։ൃ؀ڥͱຊ൪؀ڥͰಉ͡ΠϝʔδΛ࢖͏ ᶅ Immutable hello:1.0.1 ։ൃ؀ڥ hello:1.0.1 ຊ൪؀ڥ

Slide 78

Slide 78 text

1. ϦϦʔεલʹϩʔυςετΛߦ͍εέʔϧͷ͖͍͠஋Λग़͓ͯ͘͠ 2. Horizontal Pod Autoscaler Λ࢖͏ 3. (Vertical Pod Autoscaler Λ࢖͏) 4. Pod Disruption Budget Λ࢖͏ 5. Pod Priority ᶆ Scalable

Slide 79

Slide 79 text

Horizontal Pod Autoscaler ίϯςφ CPU: 1 Memory: 1GB ίϯςφͷ਺: 3 ίϯςφͷ਺: 3 + 6

Slide 80

Slide 80 text

Vertical Pod Autoscaler ίϯςφ CPU: 1 Memory: 1GB ίϯςφ CPU: 1 Memory: 2GB ίϯςφ CPU: 1 Memory: 1GB ίϯςφ CPU: 2 Memory: 2GB

Slide 81

Slide 81 text

Pod Disruption Budget ίϯςφͷ਺: 10 PDB ͷྫ: ϊʔυϝϯςφϯε౳ͷܭըతͳϊʔυͷμ΢ϯ λΠϜ࣌ʹશମͷ 20% ·Ͱ͔͠ݮΔ͜ͱΛڐ༰͠ͳ͍ Քಇ͍ͯ͠Δίϯςφͷ਺: 8 ఀࢭͨ͠ίϯςφͷ਺: 2

Slide 82

Slide 82 text

Pod Disruption Budget ΫϥελΞυϛϯνʔϜ͕ɺܭըϝϯςφϯεʹΑͬͯϊʔυ͔Βίϯς φΛୀආͤ͞Α͏ͱͨ͠ͱ͖(kubectl drain)ɺKubernetes ͸ PDB ʹࢦఆ ͞Εͨ Pod ਺ΛԼճΒͳ͍Α͏ʹ͠ͳ͕ΒίϯςφΛୀආͤͯ͘͞ΕΔɻ ͜Ε͸ϓϩμΫτνʔϜͱΫϥελΞυϛϯνʔϜ͕͓ޓ͍ʹӨڹΛग़͞ ͳ͍Α͏ʹ͢ΔͨΊͷ࢓૊Έ = ૄ݁߹ԽͰ͋Δ

Slide 83

Slide 83 text

1. 1 ͭͷίϯςφʹ͸ 1 ͭͷ࢓ࣄΛͤ͞Δ 2. ϋʔυίʔυ΍ґଘؔ܎Λۃྗආ͚Δ 1. Label ͷར༻ 2. σϓϩΠॱং͸͋ΔΑΓͳ͍ํ͕͍͍ 3. Node Affinity ΋ۃྗආ͚Δ 4. Service Λ࢖ͬͯݻఆ IP ΋ආ͚Δ ᶇ Loosely Coupled

Slide 84

Slide 84 text

ᶇ Loosely Coupled ಛఆͷ؀ڥԼͰ͔͠ੜ͖ΒΕͳ͍ੜ෺ΑΓ΋ɺ͋ΒΏΔ؀ڥԼͰੜ͖ΒΕ Δੜ෺ͷํ͕αόΠόϧೳྗ͕ߴ͍ͱݴ͑Δɻಉ༷ʹಛఆͷ໊લɺಛఆͷ ॱংɺಛఆͷϊʔυɺಛఆͷ IP ʹґଘͨ͠ίϯςφΑΓ΋ɺ ґଘ͕ͳ͍ίϯςφͷํ͕৴པੑ͕ߴ͍ɻ৴པੑ͕ߴ͍ίϯςφ͸ͦͷ෼ ख͕͔͔Βͳͯ͘ྑ͍ɻ

Slide 85

Slide 85 text

1. 1 ίϯςφ 1 ϓϩηε͕جຊ 2. 1 ͭͷίϯςφ(Πϝʔδ)ʹෳ਺ͷ੹຿͕͋Δͱίϯςφͷྑ͕͞ͳ͘ͳΔ 1. Dockerfile ͕ෳࡶʹͳΔ 2. εέʔϧ৚͕݅ෳࡶʹͳΔ 3. ϞχλϦϯά͕ෳࡶʹͳΔ 4. ىಈॲཧɺఀࢭॲཧ͕ෳࡶʹͳΔ 1 ͭͷίϯςφʹ͸ 1 ͭͷ࢓ࣄΛͤ͞Δ

Slide 86

Slide 86 text

1. SIGTERM, SIGKILL ΛϋϯυϦϯάͰ͖ΔΑ͏ʹ 2. ѱӨڹΛग़ͣ͞ʹఀࢭͰ͖ΔΑ͏ʹ 3. جຊతʹ͸࢖Θͳ͍ํ͕ෳࡶʹͳΒͣʹ͍͍͕ඞཁͳΒ͹ىಈޙ ͷॲཧɺఀࢭલͷॲཧʹ preStop, postStart ϋϯυϥΛ࢖͏ ᶈ Graceful

Slide 87

Slide 87 text

12 Factor App

Slide 88

Slide 88 text

ΦϖϨʔγϣϯͷઃܭ

Slide 89

Slide 89 text

1. ͜͜Ͱ͍͏ΦϖϨʔγϣϯͱ͸ʁ 1. ίϯςφͷσϓϩΠ(ྫ: Deployment ͷ࡞੒) 2. ڞ௨ج൫ͷϝϯςφϯε(ྫ: Node ͷΞοϓάϨʔυ) 2. ۃྗࣗ཯తͳγεςϜʹ೚ͤΔ͜ͱɺਓྗ࡞ۀΛݮΒ͢͜ͱ͕ࢦ਑ ΦϖϨʔγϣϯͷઃܭ

Slide 90

Slide 90 text

Control Loop Kubernetes ͷࠜװʹ Control Loop ͱ͍͏࢓૊Έ͕͋ΔɻKubernetes ͸ pod ͳͲͷ resource Λ؅ཧ͢Δࡍʹɺdesired state(ཧ૝ঢ়ଶ)ͱ actual state(࣮ࡍͷঢ়ଶ)ͷ 2 ͭΛ͓࣋ͬͯΓɺactual state Λ desired state ʹۙ ͚ͮΑ͏ͱ͢ΔॲཧΛӬٱʹ܁Γฦ͍ͯ͠Δɻ

Slide 91

Slide 91 text

Control Loop ཧ૝ͷঢ়ଶͱ࣮ࡍͷঢ়ଶͷൺֱ (Diff) ࣮ࡍͷঢ়ଶΛ֬ೝ (Observe) ཧ૝ঢ়ଶʹ͚ۙͮΔॲཧΛ࣮ߦ (Act)

Slide 92

Slide 92 text

Desired ίϯςφͷ਺: 5 Kubernetes ར༻ऀ͕ཧ૝ঢ়ଶΛ఻͑Δ ίϯςφͷ਺Λཧ૝ͷঢ়ଶʹ͚͍ۙͮͯ͘ྫ

Slide 93

Slide 93 text

Actual ίϯςφͷ਺: 2 Desired ίϯςφͷ਺: 5 ࣮ࡍͷঢ়ଶΛ֬ೝ (Observe) ίϯςφͷ਺Λཧ૝ͷঢ়ଶʹ͚͍ۙͮͯ͘ྫ

Slide 94

Slide 94 text

ίϯςφͷ਺Λཧ૝ͷঢ়ଶʹ͚͍ۙͮͯ͘ྫ Actual ίϯςφͷ਺: 2 Desired ίϯςφͷ਺: 5 ཧ૝ͷঢ়ଶͱ࣮ࡍͷঢ়ଶͷൺ ֱ(Diff)

Slide 95

Slide 95 text

Actual ίϯςφͷ਺: 5 Desired ίϯςφͷ਺: 5 ཧ૝ঢ়ଶʹ͚ۙͮΔॲཧΛ࣮ߦ (Act) ίϯςφͷ਺Λཧ૝ͷঢ়ଶʹ͚͍ۙͮͯ͘ྫ

Slide 96

Slide 96 text

એݴతͳΞϓϩʔνΛऔΔ ࣮ࡍͷঢ়ଶΛཧ૝ঢ়ଶʹ͍ͯ͘͜͠ͱɺͦͯͦ͠ͷํ๏(HOW) ʹ੹೚Λ࣋ͭͷ͕ Kubernetes ɻͦΕʹରͯ͠ཧ૝ঢ়ଶΛܾΊ(WHAT)ɺKubernetes ʹ఻͑Δͷ͕ Kubernetes ར༻ऀͷ੹೚ɻཧ૝ঢ়ଶΛ఻͑ؒҧ͑͹໰୊͕ى͖ͯ͠·͏ͨΊɺཧ ૝ঢ়ଶΛόʔδϣϯ؅ཧ͢Δɻ·ͨɺKubernetes ར༻ଆ͕ HOW ͷ෦෼ΛӅṭ(ந ৅Խ) ͯ͘͠Ε͍ͯΔͷ΋ؔΘΒ໋ͣྩతʹΦϖϨʔγϣϯ͢Δͷ΋ຊདྷͷઃܭҙ ਤʹ൓͢Δɻ

Slide 97

Slide 97 text

એݴతͳΞϓϩʔνΛऔΔ The declarative approach is key to the system’s self-healing and autonomic capabilities. Kubernetes Design and Architecture Ҿ༻IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE

Slide 98

Slide 98 text

એݴతͳΞϓϩʔνΛऔΔ In particular, it should be straightforward (but not required) to manage declarative intent under version control, which is standard industry best practice and what Google does internally. Version control facilitates reproducibility, reversibility, and an audit trail. ... Version control enables the use of familiar tools and processes for change control, review, and conflict resolution. Declarative application management in Kubernetes Ҿ༻IUUQTEPDTHPPHMFDPNEPDVNFOUED-1(XF7&:S7R2W#-+HTY75S&3N./0#"@DY;186

Slide 99

Slide 99 text

ͳͥ Kubernetes ͕ YAML ϕʔεͳͷ͔ 1. DSL ʹൺ΂ͯଟ͘ͷݴޠͰαϙʔτ͞Ε͍ͯΔ 2. Lint πʔϧ΋ଟ͘ଘࡏ͢Δ 3. API schema ͷৄࡉΛ֮͑ͳ͍ͱॻ͚ͳ͍΋ͷͷ… 1. ஗͔Εૣ͔Ε API schema ΍ Kubernetes ͷ֓೦ʹֶ͍ͭͯͼͨ͘ͳΔɺ· ͨ͸ֶͿඞཁ͕ग़ͯ͘ΔͷͰແବʹͳΒͳ͍ 2. Ή͠ΖҰ؏ੑ͕͋ͬͯΑ͍ ࢀߟIUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE

Slide 100

Slide 100 text

YAML ϚχϑΣετ͕ॻ͚ΔͳΒ… FDIPb BQJ7FSTJPOBQQTWCFUB LJOE%FQMPZNFOU NFUBEBUB OBNFEFQMPZNFOUFYBNQMF TQFD SFQMJDBT SFWJTJPO)JTUPSZ-JNJU UFNQMBUF NFUBEBUB MBCFMT BQQOHJOY TQFD DPOUBJOFST OBNFOHJOY JNBHFOHJOY QPSUT DPOUBJOFS1PSU cLVCFDUMDSFBUFG

Slide 101

Slide 101 text

REST API ΋࢖͑ΔΑ͏ʹͳ͍ͬͯΔ DVSM91045)$POUFOU5ZQFBQQMJDBUJPOZBNMEBUB BQJ7FSTJPOBQQTWCFUB LJOE%FQMPZNFOU NFUBEBUB OBNFEFQMPZNFOUFYBNQMF TQFD SFQMJDBT SFWJTJPO)JTUPSZ-JNJU UFNQMBUF NFUBEBUB MBCFMT BQQOHJOY TQFD DPOUBJOFST OBNFOHJOY JNBHFOHJOY QPSUT DPOUBJOFS1PSU IUUQBQJTBQQTWOBNFTQBDFTEFGBVMUEFQMPZNFOU

Slide 102

Slide 102 text

YAML Λॻ͜͏ 1. DSL ΍ GUI ౳Ͱந৅Խ͞ΕͨϚχϑΣετΛॻ͘ͷ͸࣮͸ԕճΓͰ͸ͳ͍͔ 2. YAML Λॻ͘ͷ͸ΤϯδχΞϦϯάͰ͸ͳ͍ͱ͍͏ҙݟʹରͯ͠ 1. YAML Λॻ͘࡞ۀࣗମ͸ΤϯδχΞϦϯάͰ͸ͳ͍͔΋͠Εͳ͍͕ɺͦΕ͸ YAML Λॻ͘࡞ۀ෦෼͔͠ݟ͍ͯͳ͍ͱݴ͑Δ 2. ΤϯδχΞϦϯά͸ Kubernetes ʹΑͬͯ YAML ϑΝΠϧͷཪଆʹӅṭ͞Ε͍ͯΔ 3. YAML Λॻ͚ͩ͘ͰΠϯϑϥपΓͷ࡞ۀ͕׬݁͢Δɺखಈ࡞ۀ͕ෆཁʹͳ͍ͬͯΔ 4. ͦͷු͍ͨ࣌ؒͰผͷΤϯδχΞϦϯάλεΫΛ΍Δ΂͖Ͱ͋ΓɺYAML Λॻ͖ͨ ͘ͳ͍͔Β REST API Λ࢖ͬͯந৅Խ͢Δ౳ͷߦҝ͸ຊ຤స౗Ͱ͸ͳ͍͔

Slide 103

Slide 103 text

1 Ϧιʔε 1 YAML ϑΝΠϧ 1. ͲͷϑΝΠϧʹͲͷϦιʔε͕ఆٛ͞Ε͍ͯΔ͔໌֬Ͱ͋Δ 2. ͲͷϦιʔεʹରͯ͠ΦϖϨʔγϣϯ͢Δͷ͔໌֬Ͱ͋Δ 3. ࠶ར༻ੑ͕ߴ͍ 4. ϦιʔεΛ௥Ճ͢Δ࣌ʹͲ͜ʹॻ͘΂͖͔໌֬Ͱ͋Δ ࢀߟIUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTBSDIJUFDUVSFBSDIJUFDUVSFNE

Slide 104

Slide 104 text

ϞχλϦϯάͷઃܭ

Slide 105

Slide 105 text

ͳͥϞχλϦϯά͢Δͷ͔ 1. γεςϜ͕ਖ਼ৗʹՔಇ͍ͯ͠Δ͔Λ೺Ѳ͢ΔͨΊ 2. Ͱ͸ɺͲͷγεςϜΛϞχλϦϯάͨ͠Βྑ͍͔ʁ

Slide 106

Slide 106 text

Kubernetes ͷੈքͰ͸γεςϜ͸ 2 छྨʹେผͰ͖Δ Πϯϑϥڞ௨ج൫ܥ ΞϓϦέʔγϣϯܥ

Slide 107

Slide 107 text

ϞχλϦϯά͢ΔγεςϜ͸େ͖͘ 2 ͭ Πϯϑϥڞ௨ج൫(Kubernetes) ΞϓϦέʔγϣϯ

Slide 108

Slide 108 text

ΞϓϦέʔγϣϯ͕ਖ਼ৗʹՔಇ͍ͯ͠Δͱ͸ʁ 1. ΞϓϦέʔγϣϯίϯςφ͕ىಈ͓ͯ͠Γɺ 2. ҰఆͷϨεϙϯελΠϜ಺Ͱɺ 3. ਖ਼ৗͳϨεϙϯε͕ฦ͍ͤͯΔঢ়ଶ

Slide 109

Slide 109 text

ΞϓϦέʔγϣϯͷϞχλϦϯά 1. ΞϓϦέʔγϣϯίϯςφ͕ਖ਼ৗʹىಈ͍ͯ͠Δ͔ 2. ҰఆͷϨεϙϯελΠϜ಺Ͱ͋Δ͔ 3. ਖ਼ৗͳϨεϙϯε͕ฦ͍ͤͯΔ͔

Slide 110

Slide 110 text

ڞ௨ج൫(Kubernetes)͕ਖ਼ৗʹՔಇ͍ͯ͠Δͱ͸ʁ 1. Kubernetes Master ͱ Nodes ͕ىಈ͓ͯ͠Γɺ 2. ༩͑ΒΕͨ desire state (ཧ૝ͷঢ়ଶ) ௨Γʹίϯςφ΍ॾʑͷϦ ιʔε͕ଘࡏ͍ͯ͠Δ͜ͱ

Slide 111

Slide 111 text

ڞ௨ج൫(Kubernetes)ͷϞχλϦϯά 1. Kubernetes Master ͱ Nodes ͕ਖ਼ৗʹىಈ͍ͯ͠Δ͔ 2. Desire state (ཧ૝ͷঢ়ଶ) ͱ Actual state (࣮ࡍͷঢ়ଶ) ʹဃ཭͕ͳ ͍͔

Slide 112

Slide 112 text

͜͜·Ͱ͕ୈҰεςοϓ

Slide 113

Slide 113 text

Ͱ͸ਖ਼ৗՔಇ͍ͯ͠ͳ͍ͱ͖ʹͲ͏͢Δ͔ʁ 1. γεςϜ͕ਖ਼ৗՔಇ͍ͯ͠Δ͔Λ൑ผ͢Δσʔλ(ϝτϦΫε)Λݟ ͍ͯͯ΋ݪҼ͸Θ͔Βͳ͍ 2. γεςϜ͕ਖ਼ৗՔಇ͍ͯ͠ͳ͍ͱ͖͸ɺͦͷγεςϜͷߏ੒ཁૉͷ ͍ͣΕ͔Ͱ໰୊͕ੜ͍ͯ͡Δ 3. ͦͷߏ੒ཁૉ΋ϞχλϦϯά͓ͯ͘͜͠ͱͰݪҼڀ໌ʹܨ͛ΒΕΔ 4. ΞϓϦέʔγϣϯͱڞ௨ج൫(Kubernetes)ͷߏ੒ཁૉ͸ʁ

Slide 114

Slide 114 text

ΞϓϦέʔγϣϯΛߏ੒͢Δ΋ͷ ΞϓϦέʔγϣϯܥ CDN ϩʔυόϥϯα ΞϓϦέʔγϣϯ ϛυϧ΢ΣΞ Ϋϥ΢υαʔϏε ίϯςφ Namespace etc

Slide 115

Slide 115 text

Πϯϑϥڞ௨ج൫Λߏ੒͢Δ΋ͷ Πϯϑϥڞ௨ج൫ܥ Ϋϥ΢υαʔϏε Kubernetes Master Kubernetes Nodes kubelet kube-proxy kube-dns ϩάऩू daemon etc

Slide 116

Slide 116 text

1. ϝτϦΫε 2. (τϨʔε) 3. Πϕϯτͱϩά ֤ߏ੒ཁૉʹؔͯ͠ 2 (3) छྨͷσʔλΛूΊΔ

Slide 117

Slide 117 text

ϝτϦΫε 1. WHAT(Կ͕ى͖͍ͯΔ͔)Λ೺Ѳ͢Δ΋ͷ 2. ग़དྷΔݶΓ৭ʑͳϝτϦΫεΛूΊ͓ͯ͘ 3. ޙ͔ΒλάɺϥϕϧͰඞཁͳ΋ͷΛݟΔ 4. ྫ: Desired Pod ਺ɺCPU ࢖༻཰ɺϨεϙϯελΠϜ

Slide 118

Slide 118 text

τϨʔε 1. HOW (Ͳ͏ͳ͍ͬͯΔ͔)Λ೺Ѳ͢Δ΋ͷ 2. ϝτϦΫεΑΓ΋ৄࡉʹͲ͜Ͱ໰୊͕ى͖͍ͯΔ͔Θ͔Δ 3. ྫ: ͲͷΫΤϦ͕஗͍ͷ͔ɺͲͷؔ਺͕Τϥʔʹͳ͍ͬͯΔ ͷ͔

Slide 119

Slide 119 text

Πϕϯτͱϩά 1. WHY (ͳͥى͖͔ͨ)Λ೺Ѳ͢Δ΋ͷ 2. ϝτϦΫε΍τϨʔε͔Β͸Θ͔Βͳ͍໰୊ͷݪҼ΍ى఺͕ Θ͔Δ 3. ྫ: ΦʔτεέʔϧΠϕϯτɺΞΫηεϩά

Slide 120

Slide 120 text

ϞχλϦϯά·ͱΊ 1. ίϯςφͷϥΠϑαΠΫϧͷ୹͞ɺಈతͳ഑ஔɺࣗ཯తͳڍಈͱɺ Observability(ࢹೝੑ) ͕ͳ͍ͱԿ͕ى͍ͬͯ͜Δ͔Θ͔Βͳ͍ 2. ೝࣝͰ͖ͳ͍΋ͷ͸վળͰ͖ͳ͍ͨΊɺܧଓతͳ Kubernetes ӡ ༻ͷվળʹϞχλϦϯά͸ඞཁෆՄܽͰ͋Δ

Slide 121

Slide 121 text

·ͱΊͱิ଍

Slide 122

Slide 122 text

1. Kubernetes Λ࢖ͬͯԿΛ࣮ݱ͍ͨ͠ͷ͔ɺKubernetes ͷઃܭऀ͸ԿΛҙਤͯ͠࡞ͬͨ ͷ͔Λҙࣝ͢Δͱɺݸʑͷػೳͷ࢖͍ํ΍ཁૉͷଘࡏҙ͕ٛݟ͑ͯ͘Δ 2. ΫϥελͷωοτϫʔΫઃܭ΍ϊʔυपΓɺηΩϡϦςΟɺϞχλϦϯάৄࡉ౳৮ΕΒ Εͯͳ͍߲໨΋͋Δ͕ɺεϥΠυຕ਺͕๲େʹͳ͖ͬͯͨͷͰࠓճ͸͜͜·Ͱ… 3. ͜͜Ͱ঺հͨ͠ઃܭ΍ϓϥΫςΟεΛશͯద༻͠Α͏ͱࢥ͏ͱ೔͕฻ΕΔɻ͍ͭ·Ͱܦͬ ͯ΋ຊ൪ӡ༻͕։࢝Ͱ͖ͳ͍ͷͰɺࣄલʹ΍Δ͜ͱͱɺࣄޙʹ΍Δ͜ͱͷόϥϯεΛऔ Γ·͠ΐ͏ 1. Ͳ͕͜ڥք͔ͱ͍͏ͱɺຊ൪ӡ༻։࢝ޙʹɺࣗಈԽ΍͜͜ʹॻ͍ͨϓϥΫςΟεΛ ద༻Ͱ͖Δ༨༟(શମͷ50%)͕ग़དྷΔ͘Β͍ͷ४උΛ͢Δͱ͍͍ͱࢥ͏ 2. શ͘४උͤͣʹຊ൪ӡ༻Λ։࢝͢Δͱ໰୊ͷରԠ΍ख࡞ۀʹຒ΋ΕͯෛͷεύΠϥ ϧʹؕΔ ·ͱΊͱิ଍

Slide 123

Slide 123 text

End