はてなでのサービス信頼性向上のための取り組み事例

by dekokun

Slide 1

Slide 1 text

͸ͯͳͰͷ αʔϏε৴པੑ޲্ͷͨΊͷ औΓ૊Έࣄྫ SRE Tech Talks גࣜձࣾ͸ͯͳ id:dekokun

Slide 2

Slide 2 text

ࣗݾ঺հ 2

Slide 3

Slide 3 text

ࣗݾ঺հ • id:dekokun • ͸ͯͳͰΠϯϑϥ@౦ژ • WebΦϖϨʔγϣϯΤϯδχΞ • ͸ͯͳͷ͍Ζ͍ΖͳαʔϏεͷΠϯϑϥ୲౰ • ͸ͯͳϒϩάͱଞʹ΋͍͔ͭ͘ • ೖࣾ1೥ऑ • લ৬͸PHPͱJSॻ͍ͯ·ͨ͠ 3

Slide 4

Slide 4 text

4 • SREͬͯͦ΋ͦ΋ͳʹʁ • ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞͱ  WebΞϓϦέʔγϣϯΤϯδχΞͷڠྗମ੍ • DBసૹྔ࡟ݮ΁ͷऔΓ૊Έ • ίʔυͱݕূʹΑͬͯαʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ • AWSʹ͓͚ΔRedisͷ৑௕ߏ੒ߏங • ໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ • མͪͯ΋མͪͯ΋ࣗಈͰىಈͯ͘͠Δαʔό ຊ೔ͷൃද಺༰

Slide 5

Slide 5 text

5 • SREͬͯͦ΋ͦ΋ͳʹʁ • ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞͱ  WebΞϓϦέʔγϣϯΤϯδχΞͷڠྗମ੍ • DBసૹྔ࡟ݮ΁ͷऔΓ૊Έ • ίʔυͱݕূʹΑͬͯαʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ • AWSʹ͓͚ΔRedisͷ৑௕ߏ੒ߏங • ໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ • མͪͯ΋མͪͯ΋ࣗಈͰىಈͯ͘͠Δαʔό ຊ೔ͷൃද಺༰

Slide 6

Slide 6 text

SREͬͯͦ΋ͦ΋Կʁ 6

Slide 7

Slide 7 text

SREͱ͸ • SRE Tech Talksͱ͍͏໊લͷΠϕϯτͰ  ൃද͍ͤͯͨͩ͘͜͞ͱʹͳ͕ͬͨ  ͦ΋ͦ΋SRE͕Կ͔Λৄ͘͠஌Βͳ͔ͬͨ • ࢲͷ৬छ΋”SRE”ͱ͍͏໊લͰ͸ͳ͍ • SLAΛܾΊͯͦͷதͰ௅ઓ͍ͯ͘͠Πϝʔδ • ίʔυΛΑ͘ॻ͘Πϝʔδ • DevOpsͱ͸ҧ͏ͷ͔ͳʁ 7

Slide 8

Slide 8 text

SREͱ͸ • Site Reliability EngineeringಡΈ࢝Ί·ͨ͠ • ࣾ಺ྠಡձ࢝·Γ·ͨ͠ • ·ͩchapter 1,2,4͔͠ಡΜͰ͍ͳ͍ • ࠓճ͸ɺࢲͷൃදʹؔ܎͋Γͦ͏ͳ෦෼͚ͩ  ܰ͘આ໌͓ͭͭ͠࿩Λ͠·͢ • ҎԼɺ”SREຊ”ͱݺͼ·͢ 8 Site Reliability Engineering: http://shop.oreilly.com/product/0636920041528.do

Slide 9

Slide 9 text

SREͱ͸ • ͸ͯͳʹ͸SREͱ͍͏ݞॻ͸͋Γ·ͤΜ͕ɺ  ͸ͯͳͰ΋αʔϏεͷ৴པੑ޲্ͷͨΊʹ  ೔ʑ༷ʑͳ͜ͱΛ͍ͯ͠·͢ • ຊ೔͸ͦΕΒͷҰ෦Λ͝঺հ͍ͨ͠·͢ 9

Slide 10

Slide 10 text

༻ޠղઆ 10

Slide 11

Slide 11 text

༻ޠղઆ • WebΦϖϨʔγϣϯΤϯδχΞ • ͍ΘΏΔΠϯϑϥ୲౰ • ͜ͷࢿྉͰ͸”OpsΤϯδχΞ” • WebΞϓϦέʔγϣϯΤϯδχΞ • ͍ΘΏΔ։ൃΤϯδχΞ • ͜ͷࢿྉͰ͸”DevΤϯδχΞ” • ͸ͯͳʹ͓͍ͯ྆ऀ͕ߦ͏ࣄͷڥ໨͸͋Δఔ౓ᐆດ 11

Slide 12

Slide 12 text

12 • SREͬͯͦ΋ͦ΋ͳʹʁ • ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞͱ  WebΞϓϦέʔγϣϯΤϯδχΞͷڠྗମ੍ • DBసૹྔ࡟ݮ΁ͷऔΓ૊Έ • ίʔυͱݕূʹΑͬͯαʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ • AWSʹ͓͚ΔRedisͷ৑௕ߏ੒ߏங • ໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ • མͪͯ΋མͪͯ΋ࣗಈͰىಈͯ͘͠Δαʔό ຊ೔ͷൃද಺༰

Slide 13

Slide 13 text

OpsΤϯδχΞͱDevΤϯδχΞͷ ڠྗମ੍ DBͷసૹྔ࡟ݮ 13

Slide 14

Slide 14 text

SREͱ͸(࠶ܝ) • “In SRE we bring this conﬂict to the fore, and then resolve it with the introduction of an error budget” • “Site Reliability Engineering: How Google Runs Production Systems” 2016/3/23 Betsy Beyerɺ Chris Jones Chapter 1 ΑΓ 14

Slide 15

Slide 15 text

opsΤϯδχΞͷαʔϏε΁ͷؔΘΓํ • SREຊʹΑΕ͹ɺSREͰ͸ͳ͍Ops͸Devͱ͸  ໨ඪ͕ҟͳΔ͜ͱʹΑΓ໰୊͕ൃੜ͕ͪͩ͠ͱͷ͜ͱ • ͸ͯͳͰ͸͋·Γͳ͍Α͏ʹࢥ͑Δ • ͸ͯͳͰͷOpsΤϯδχΞͱDevΤϯδχΞͷ  αʔϏε΁ͷؔΘΓํΛ঺հ͠·͢ 15

Slide 16

Slide 16 text

OpsΤϯδχΞͷαʔϏε΁ͷؔΘΓํ WebΦϖϨʔγϣϯ ΤϯδχΞνʔϜ ֤αʔϏε ͸ͯͳϒϩά ͸ͯͳϒοΫϚʔΫ etc… • ֤αʔϏεʹOpsΤϯδχΞ͕  ୲౰ͱͯͭ͘͠ 16

Slide 17

Slide 17 text

OpsΤϯδχΞͷαʔϏε΁ͷؔΘΓํ • ͸ͯͳʹ͓͍ͯ྆ऀ͕ߦ͏ࣄͷڥ໨͸͋Δఔ౓ᐆດ • OpsΤϯδχΞ͕αʔϏεͷίʔυʹpull requestΛ  ग़ͨ͠Γ • DevΤϯδχΞ͕chefΛ͍ͬͨ͡Γ͢Δ 17

Slide 18

Slide 18 text

ڠྗମ੍ࣄྫɿ DBͷసૹྔ࡟ݮ 18

Slide 19

Slide 19 text

DBసૹྔ࡟ݮࣄྫ • ͋Δ೔ɺmaster DB(MySQL)ͷసૹྔ͕  ݶքۙ͘ͳ͍ͬͯΔ͜ͱʹؾ෇͘ • ೔ʹΑͬͯ͸ϐʔΫ࣌ʹ͸  ”ޙ10%૿͑ͨΒݶքಥഁͩͳ”  ͱ͍͏ײ͡ʹͳ͍ͬͯΔ৔߹΋… • खΛଧͭඞཁ͕͋Δ 19 ཧ࿦্ͷݶք஋ ΪϦΪϦͩ… Πϝʔδਤ

Slide 20

Slide 20 text

DBసૹྔ࡟ݮࣄྫ • CPU࢖༻཰͸ΊͪΌͪ͘Ό༨͍ͬͯͨͷͰMySQLͷ௨ ৴ѹॖ͕ྑͦ͞͏ͱ͍͏͜ͱͰݕূΛߦͬͨ • DSNʹ’mysql_compression=1’Λࢦఆ • ѹॖʹΑΓCPU͸৯͏͕ϚϧνίΞΛ࢖ͬͯ͘ΕΔ͠ ҆৺ 20

Slide 21

Slide 21 text

DBసૹྔ࡟ݮࣄྫ • ݕূ݁Ռ্͕ʑͩͬͨͷͰpull requestΛग़ͯ͠  OpsΤϯδχΞʹϨϏϡʔͯ͠΋ΒͬͨΓ  मਖ਼ͯ͠΋ΒͬͨΓͭͭ͠ϦϦʔε 21

Slide 22

Slide 22 text

DBసૹྔ࡟ݮࣄྫ • సૹྔܹݮ(໿1/4) ΊͰ͍ͨ • DBͷCPU࢖༻཰͕໰୊ͳ͍ൣғͰܹ૿ 22 సૹྔܹݮ CPU࢖༻཰ܹ૿

Slide 23

Slide 23 text

DBసૹྔ࡟ݮࣄྫ • సૹྔ͕ݮগ҆͠৺͍ͯͨ͋͠Δ໷ɺ͍͖ͳΓCPU࢖ ༻཰͕ܹ૿͠DB͕٧·Γ͔͚Δ • “Writing to net”ͷDBεϨου͕େྔʹଘࡏ • సૹྔ͕૿͑͗ͯ͢ѹॖʹΑͬͯCPUΛ৯͍ͭͿ͠ѹॖ ͕௥͍͍͍ͭͯͳ͍ • ѹॖʹΑΔ΋ͷ͕ͩɺ͔ͱ͍ͬͯΞϓϦέʔγϣϯαʔ όશ୆ͰѹॖΛղআ͢Δͱࠓ౓͸సૹྔͰ٧·Δ 23

Slide 24

Slide 24 text

DBసૹྔ࡟ݮࣄྫ Ұ࣍ରԠ • Ұ࣍ରԠͱͯ͠ΞϓϦέʔγϣϯαʔόͷҰ෦ͷΈͰ  ѹॖΛղআ͠೉ΛಀΕͨ 24 ѹॖ ѹॖ ඇѹॖ ඇѹॖ

Slide 25

Slide 25 text

DBసૹྔ࡟ݮࣄྫ Ұ࣍ରԠ • ཌ೔ɺࣄଶ͸·ͩ༧அΛڐ͞ͳ͍ঢ়گͳͷͰ  Ұ୴ΠϯελϯελΠϓΛ্͛ͯmasterDB੾Γସ͑Λ ߦ͍Ұ҆৺ • AWS͞ΜMHA͞Μ͋Γ͕ͱ͏ 25

Slide 26

Slide 26 text

DBసૹྔ࡟ݮࣄྫ ࠜຊରॲ ௐࠪ • ٸʹCPU࢖༻཰্͕͕Δͷ͸ௐ͕ࠪඞཁ • DevΤϯδχΞͱڞʹkibana౳͔ΒݪҼΛௐࠪ͠ɺ  ͋ΔΤϯυϙΠϯτ͕ϢʔβͷߦಈʹΑͬͯ  ܹ͘͠DBͷసૹྔΛফඅ͢ΔΑ͏ʹͳΔ͜ͱΛൃݟɾ  DevΤϯδχΞͱڞ༗ 26

Slide 27

Slide 27 text

DBసૹྔ࡟ݮࣄྫ ࠜຊରॲ • ཌ೔ʹ͸DevΤϯδχΞͷखʹΑͬͯରॲ൛͕  ϦϦʔε͞ΕɺDBͷసૹྔ΋ݮΓΊͰͨ͠ΊͰͨ͠ • ͦͷޙDevΤϯδχΞͷखͰɺߋʹDBͷసૹྔΛݮΒ ͢ϦϦʔε͕Կ౓͔ߦΘΕͨ 27

Slide 28

Slide 28 text

OpsΤϯδχΞͱDevΤϯδχΞ • SREຊͰ͸“(SREͰ͸ͳ͍چདྷͷ)Ops͸αΠτͷՄ༻ੑ ΛकΔͨΊʹૉૣ͍σϓϩΠͳͲʹରཱ͢Δ”ͱ͍͏Α ͏ͳ࿩΋͋Γ·ͨ͠ɻ͕ɺ • ࠓճݟͨΑ͏ʹɺڍಈ͕Ϣʔβͷߦಈʹґଘ͢Δ  ҰൠͷWebαʔϏεʹ͓͍ͯͦ΋ͦ΋”αΠτͷՄ༻ੑ ΛकΔ”ͱ͍͏จ຺Ͱ΋Ospଆ΋ૉૣ͍มԽΛ๬·ͳ͍ ͱࢮ͵ͷͰ͸ • ΋ͪΖΜαʔϏεͷ੒௕͕Ұ൪ 28

Slide 29

Slide 29 text

29 • ΤϯδχΞ͕ࢀՃ͢ΔɺຖिߦΘΕΔࣾ಺ษڧձͰ͸  ͦΕͧΕͷ࢓ࣄͷ಺༰Λൃද͍ͯ͘͠ͳͲ  ૬ޓͷཧղͷॿ͚ʹͳ͍ͬͯ·͢ • ۀ຿࣌ؒ֎ͷো֐ରԠΛݮΒͨ͢Ίɺ  ҎԼͷΑ͏ͳ։ൃνʔϜͱͷ߹ҙ΋ߦ͍ͬͯ·͢ • ༵ۚ೔ʹϦϦʔεΛ͠ͳ͍ • ༦ํҎ߱ʹϦϦʔεΛ͠ͳ͍ OpsΤϯδχΞͱDevΤϯδχΞ ଞʹ΋͍Ζ͍Ζ

Slide 30

Slide 30 text

30 • ͜ͷΑ͏ʹɺ͸ͯͳͰ͸೔ʑOpsΤϯδχΞͱ  DevΤϯδχΞ͕ڠྗͭͭ͠  αʔϏεͷ৴པੑ޲্΁ͷऔΓ૊ΈΛ  ߦ͍͍ͬͯͬͯ·͢ OpsΤϯδχΞͱDevΤϯδχΞ ·ͱΊ

Slide 31

Slide 31 text

31 • SREͬͯͦ΋ͦ΋ͳʹʁ • ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞͱ  WebΞϓϦέʔγϣϯΤϯδχΞͷڠྗମ੍ • DBసૹྔ࡟ݮ΁ͷऔΓ૊Έ • ίʔυͱݕূʹΑͬͯαʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ • AWSʹ͓͚ΔRedisͷ৑௕ߏ੒ߏங • ໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ • མͪͯ΋མͪͯ΋ࣗಈͰىಈͯ͘͠Δαʔό ຊ೔ͷൃද಺༰

Slide 32

Slide 32 text

ίʔυͱݕূʹΑͬͯ αʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ AWS؀ڥͷRedisͷ৑௕ߏ੒ߏங 32

Slide 33

Slide 33 text

SREͱ͸(࠶ܝ) • “Google caps operational work for SREs at 50% of their time. Their remaining time should be spent using their coding skills on project work.” • “Site Reliability Engineering: How Google Runs Production Systems” 2016/3/23 Betsy Beyerɺ Chris Jones Chapter 1 ΑΓ 33

Slide 34

Slide 34 text

AWS؀ڥͷRedisͷ৑௕ߏ੒ߏங • զʑ͸50%΋ίʔυ͸ॻ͍͍ͯͳ͍ʂ • ͔͠͠ɺ΋ͪΖΜίʔυ͸ॻ͍͍ͯ·͢ • ίʔυΛॻ͖ͭͭɺ໰୊ղܾΛߦͬͨ࿩Λ͠·͢ 34

Slide 35

Slide 35 text

AWS؀ڥͷRedisͷ৑௕ߏ੒ߏங • ͸ͯͳͰ͸σʔληϯλʔͱAWSͷ྆ํͰαʔϏεΛ ӡӦ͍ͯ͠·͢ • Redisͷ৑௕ߏ੒͸͜Ε·ͰσʔληϯλʔͰ͸  ߦ͍ͬͯ·͕ͨ͠ɺAWS্Ͱ͸Ͱ͖͍ͯ·ͤΜͰͨ͠ • AWS্Ͱ΋Redisͷ৑௕ߏ੒Λߏங͓ͨ͠࿩Λ͠·͢ 35

Slide 36

Slide 36 text

σʔληϯλʔͰͷRedisͷ৑௕ߏ੒ 36 Redis (master) keepalived IP: x.x.x.x Redis (slave) keepalived IP:y.y.y.y VRRP /w broadcast replication health check health check VIP: z.z.z.z VIPͰ઀ଓ

Slide 37

Slide 37 text

σʔληϯλʔͰͷRedisͷ৑௕ߏ੒ • keepalived + VRRPʹΑΔ৑௕ߏ੒ • ΞϓϦέʔγϣϯ͔Β͸VIPͰRedisʹ઀ଓ • Redisʹ໰୊͕͋ͬͨ৔߹͸VIP͕Ҡಈ • VIPͷҠಈͱಉ࣌ʹɺslaveͷmasterঢ֨/masterͷslave ߱֨Λ࣮ࢪ 37 Redis (master) keepalived IP: x.x.x.x Redis (slave) keepalived IP:y.y.y.y VRRP /w broadcast replication health check VIP: z.z.z.z VIPͰ઀ଓ health check

Slide 38

Slide 38 text

Redis (master) keepalived IP: x.x.x.x Redis (slave) keepalived IP:y.y.y.y VRRP /w broadcast replication health check health check VIP: z.z.z.z VIPͰ઀ଓ AWSͰσʔληϯλʔͱಉ༷ͷߏ੒ΛͱΔͨΊͷ໰୊ 38 VRRP /w broadcast VIP: z.z.z.z

Slide 39

Slide 39 text

Slide 40

Slide 40 text

40 • AWS VPCͰ͸IPͷbroadcast/multicast͸࢖༻Ͱ͖ͳ͍ ͷͰunicastΛ࢖༻ͨ͠VRRPΛߦ͏ඞཁ͕͋Δ • ͸ͯͳͰ࢖͍ͬͯΔkeepalivedͷߏจνΣοΧ͸ൺֱ త৽͍͠ػೳͰ͋ΔunicastʹରԠ͍ͯ͠ͳ͍ • keepalived͸ඪ४ͰߏจνΣοΧ͕͍͓ͭͯΒͣɺ ޡͬͨߏจͰreload͢Δͱਖ਼ৗʹಈ࡞͠ͳ͘ͳΔͷ ͰߏจνΣοΧͳ͠Ͱ͸ාͯ͘࢖͑ͳ͍ Redisͷ৑௕ߏ੒ unicastͱߏจνΣοΫ

Slide 41

Slide 41 text

41 • ͸ͯͳͷ࢖͍ͬͯͨߏจνΣοΧ͸Haskell੡ • ࢲ͕unicastରԠͷHaskellίʔυΛॻͨ͘Ίʹ  Ϗϧυ؀ڥΛ੔͑Α͏ͱ࢛ۤീ͍ۤͯ͠Δ಺ʹɺ  ผͷDevΤϯδχΞͷid:y_uuki ͞Μ͕  GoݴޠͰߏจνΣοΧΛ࡞ͬͯ͘Εͯղܾ Redisͷ৑௕ߏ੒ unicastͱߏจνΣοΫ

Slide 42

Slide 42 text

Redisͷ৑௕ߏ੒ unicastͱߏจνΣοΫ 42

Slide 43

Slide 43 text

Slide 44

Slide 44 text

44 • σʔληϯλʔ಺Ͱ͸VIPͷҠಈ͸  GARPʹΑͬͯαϒωοτʹप஌͞ΕΔ͕  AWS VPCͰ͸ͦͷํ๏͸࢖͑ͳ͍ • VIPͷҠಈ͸ENIΛҠಈͤ͞Δ͜ͱͰ࣮ݱ • ͜͜Ͱ΋id:y_uuki͞Μͷ࡞ͬͨ  Goݴޠ੡ͷιϑτ΢ΣΞɺgrabeniΛ࢖༻ Redisͷ৑௕ߏ੒ VIPͷҠಈ

Slide 45

Slide 45 text

Redisͷ৑௕ߏ੒ VIPͷҠಈ 45

Slide 46

Slide 46 text

46 • ͍ͭͰʹݕূதʹݟ͔༷ͭͬͨʑͳখ͍͞໰୊ΛຒΊΔ • keepalivedͷݱࡏͷstatusΛϑΝΠϧग़ྗ • keepalivedͷMASTER/BACKUPߏ੒͔Β  BACKUP/BACKUPߏ੒΁ • keepalivedͷstatus੾Γସ͑௨஌ϝʔϧͷ  λΠτϧʹϗετ໊͕ग़ྗ͞ΕΔΑ͏ʹ • Redisͷfailover࣌ʹconﬁg rewriteΛߦ͍ɺfailoverޙʹ Redis͕࠶ىಈͯ͠΋ઃఆ͕ר͖໭Βͳ͍Α͏ʹ Redisͷ৑௕ߏ੒ ͦͷଞ

Slide 47

Slide 47 text

47 • ͱ͍͏༷ʑͳιϑτ΢ΣΞ։ൃ΍ݕূʹΑͬͯɺAWS ʹ͓͚ΔRedisͷ৑௕ߏ੒͸࣮ݱ͠·ͨ͠ Redisͷ৑௕ߏ੒ ·ͱΊ

Slide 48

Slide 48 text

48 • SREͬͯͦ΋ͦ΋ͳʹʁ • ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞͱ  WebΞϓϦέʔγϣϯΤϯδχΞͷڠྗମ੍ • DBసૹྔ࡟ݮ΁ͷऔΓ૊Έ • ίʔυͱݕূʹΑͬͯαʔϏεͷ৴པੑΛ޲্ͤ͞Δ࿩ • AWSʹ͓͚ΔRedisͷ৑௕ߏ੒ߏங • ໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ • མͪͯ΋མͪͯ΋ࣗಈͰىಈͯ͘͠Δαʔό ຊ೔ͷൃද಺༰

Slide 49

Slide 49 text

໨ࢦ͢΂͖γεςϜͷ࢟ʹ͍ͭͯ མͪͯ΋མͪͯ΋ࣗಈͰ  ىಈͯ͘͠Δαʔό 49

Slide 50

Slide 50 text

SREͱ͸(࠶ܝ) • “Monitoring should never require a human to interpret any part of the alerting domain. Instead, software should do the interpreting, and humans should be notiﬁed only when they need to take action. ” • “Site Reliability Engineering: How Google Runs Production Systems” 2016/3/23 Betsy Beyerɺ Chris Jones Chapter 1 ΑΓ 50

Slide 51

Slide 51 text

51 • ਓؒ͸ਓؒͷͰ͖Δ͜ͱʹूத͢΂͖Ͱɺ  ػց͕Ͱ͖Δ͜ͱ͸ػցʹ೚ͤΔ΂͖ • ͸ͯͳʹೖͬͯ໘ന͍ͳͱײͨ͡  ࣗ཯෮چγεςϜʹ͍ͭͯ • ΋ͪΖΜଞʹ΋͍Ζ͍Ζͳ  ࣗಈγεςϜ͸͋ΔͷͰ͕͢ɺ  Ұ൪͓΋͠Ζ͔ͬͨ΋ͷΛ ໨ࢦ͢΂͖γεςϜͷ࢟

Slide 52

Slide 52 text

52 • ͸ͯͳͷσʔληϯλʔͰ͸  ෺ཧαʔόΛԾ૝Խͯ͠࢖༻ • Ծ૝Խج൫ʹ͸XenΛ࢖༻͠ɺ  1୆ͷ෺ཧαʔόʹෳ਺ͷOSΛฒྻʹಈ࡞͍ͤͯ͞Δ Xenʹ͍ͭͯ

Slide 53

Slide 53 text

Xenʹ͍ͭͯ(༻ޠ) • DomU: ͍ΘΏΔී௨ͷαʔόͱͯ͠࢖༻ • Dom0: DomUͷىಈ౳΋ؚΊͨXenͷ؅ཧ΍ϋʔυ΢ΣΞΞΫηεΛߦ͏ 53 ϋʔυ΢ΣΞ Dom0(Xenͷ؅ཧ) DomU(Webαʔό) DomU(Webαʔό) DomU(DBαʔό)

Slide 54

Slide 54 text

54 • Xenͷ্Ͱ࡞ΒΕͨɺੲͷαʔϏεʹΑ͘࢖ΘΕ͍ͯΔ ࣗ཯෮چγεςϜ ໨ࢦ͢΂͖γεςϜͷ࢟ ࣗ཯෮چ

Slide 55

Slide 55 text

໨ࢦ͢΂͖γεςϜͷ࢟ ࣗ཯෮چ • Dom0͕DomU্Ͱಈ͍͍ͯΔHTTPΞϓϦέʔγϣϯ ΛHTTPͰ؂ࢹ(monit࢖༻) 55 ؂ࢹ ؂ࢹ ϋʔυ΢ΣΞ Dom0(Xenͷ؅ཧ) DomU(Webαʔό) DomU(Webαʔό) DomU(DBαʔό)

Slide 56

Slide 56 text

໨ࢦ͢΂͖γεςϜͷ࢟ ࣗ཯෮چ • DomUͷHTTPαʔό͕ࢮΜͩ͜ͱΛݕ஌͢Δͱ 56 ؂ࢹ ࢮ๢ݕ஌ ϋʔυ΢ΣΞ Dom0(Xenͷ؅ཧ) DomU(Webαʔό) DomU(Webαʔό) DomU(DBαʔό)

Slide 57

Slide 57 text

໨ࢦ͢΂͖γεςϜͷ࢟ ࣗ཯෮چ • monit͕ͦͷDomUΛOS͝ͱstop/start • xm destroy && xm create 57 ؂ࢹ DomUΛOS͝ͱ stop/start ϋʔυ΢ΣΞ Dom0(Xenͷ؅ཧ) DomU(Webαʔό) DomU(Webαʔό) DomU(DBαʔό)

Slide 58

Slide 58 text

໨ࢦ͢΂͖γεςϜͷ࢟ ࣗ཯෮چ • উखʹ෮چ׬ྃ 58 ؂ࢹ ؂ࢹ ϋʔυ΢ΣΞ Dom0(Xenͷ؅ཧ) DomU(Webαʔό) DomU(Webαʔό) DomU(DBαʔό)

Slide 59

Slide 59 text

໨ࢦ͢΂͖γεςϜͷ࢟ • ͋Δ೔ߴෛՙͰαʔό͕མͪͯ͸ࣗಈ෮چ͍ͯ͠ΔͷΛ ݟͯײಈͨ࣌͠ͷൃݴ 59

Slide 60

Slide 60 text

60 • ਓؒ͸ਓؒʹ͔͠Ͱ͖ͳ͍͜ͱʹ  ूத͢Δ؀ڥΛ࡞͍ͬͯ͘ ໨ࢦ͢΂͖γεςϜͷ࢟

Slide 61

Slide 61 text

61 • ͸ͯͳͰͷࣄྫ঺հΛ͠·ͨ͠ • ͜Ε͔Β΋DevΤϯδχΞͱڞʹྑ͍αʔϏεΛ࡞ͬͯ ͍͖·͢ • ͜Ε͔Β΋೪Γڧࣗ͘ಈԽΛਐΊ͍͖ͯ·͢ • ػցͷͰ͖Δ͜ͱ͸શͯػցʹ೚ͤΔੈք΁… ·ͱΊ

Slide 62

Slide 62 text

62 • ͸ͯͳͰ͸OpsΤϯδχΞΛੵۃ࠾༻͍ͯ͠·͢ʂ • ౦ژͰ΋ژ౎Ͱ΋ʂ • ࢲ͸౦ژͰژ౎ͷDev/OpsΤϯδχΞͱ  Ұॹʹ࢓ࣄ͍ͯ͠·͢ʂ • ࠓ೔ͷൃද΍౦ژͱژ౎ͷϦϞʔτϫʔΫͷ࿩ͳͲͰ΋  ฉ͖͍ͨ͜ͱ͋Γ·ͨ͠Βੋඇ࠙਌ձͰʂ ࠷ޙʹ