5years-history-of-heteml

Slide 1

Slide 1 text

2013/11/9 Shinya Tsunematsu @tnmt ๻ͱαʔϏεͷ5೥ͷาΈ Πϯϑϥվળͷྺ࢙

Slide 2

Slide 2 text

5೥ଓ͘αʔϏεͷӡ༻ͬͯ

Slide 3

Slide 3 text

௕͘ଓ͘αʔϏεʹ͖ͭ·ͱ͏ ✤ ୔ࢁ͋ΔαʔόؒͰͷඍົͳ࢓༷ࠩ! ✤ ن໛͕େ͖͍৔߹ͷมߋͷखؒ! ✤ ΦϯϓϨϛεͷ৔߹ͷεϖοΫΞοϓʹ͍͍͚ͭͯͳ͍ॊೈੑͷແ ͞! ✤ ϧʔνϯԽ͞Εͨ࡞ۀ͕লྗ͞Ε͍ͯͳ͍͜ͱʹΑΔӡ༻ίετ૿! ✤ Կ͕ى͖͍ͯΔ͔೺Ѳग़དྷ͍ͯͳ͍αʔϏεϔϧενΣοΫ඼࣭ͷ ྼԽ

Slide 4

Slide 4 text

paperboy&co. ϔςϜϧνʔϜ

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

heteml (ϔςϜϧ) ✤ ϩϦϙοϓʂͱฒͿϨϯλϧαʔόͷϥΠϯφοϓ! ✤ 2005೥ϦϦʔε! ✤ 2013೥10݄ݱࡏαʔό400୆ऑ! ✤ ৗদ͸2009೥͔ΒδϣΠϯ

Slide 7

Slide 7 text

ٕज़తෛ࠴

Slide 8

Slide 8 text

ෛ࠴ͱ͍͏ͱωΨςΟϒͳҹ৅ʁ ✤ ͔͠͠ඞͣ෇ਵ͢Δ໰୊Ͱࣗવͳ΋ͷ! ✤ ະવʹ๷͙ɺൃੜͨ͠৔߹ʹ͸ରॲ͍ͯ͘͠ͷΛ͓Ζ ͔ͦʹ͢Δͱޙʑࣗ෼ͱपΓʹৼΓ͔͔Δ

Slide 9

Slide 9 text

ΞδΣϯμ ✤ ఏڙ͍ͯ͠ΔαʔϏε͸͢΂ͯόʔδϣϯ؅ཧԽͰ؅ཧ͢Δ! ✤ σϓϩΠ΍αʔόʹର͢ΔมߋπʔϧΛ੔උɺଐਓԽ΍൥ࡶ͞ Λഉআ! ✤ OSηοτΞοϓΛলྗԽ! ✤ γεςϜߏ੒؅ཧπʔϧΛ༻͍ͯαʔόߏஙͷলྗԽɾ࢓༷Խ Λߦ͏! ✤ ؂ࢹγεςϜઃఆͷϨϏϡʔɺ؂ࢹઃఆͷࣗಈੜ੒

Slide 10

Slide 10 text

͢΂ͯΛόʔδϣϯ؅ཧԼʹ

Slide 11

Slide 11 text

ԿΛόʔδϣϯ؅ཧ͢Δ͔ ✤ جຊతʹ͸ʮ͢΂ͯʯ

Slide 12

Slide 12 text

ͱ͸͍͍͖͑ͳΓશ෦͸ແཧ ✤ ༏ઌ౓Λ͚ͭΔ! ✤ OSϨΠϠɺηΩϡϦςΟपΓ! ✤ ϛυϧ΢ΣΞઃఆ! ✤ cron, ӡ༻εΫϦϓτ! ✤ ؂ࢹιϑτ΢ΣΞઃఆ

Slide 13

Slide 13 text

ΑΓԼͷϨΠϠ͔Β؅ཧ͢Δ ✤ ҙຯͷॏෳΛආ͚ΒΕΔͨΊͦͷ͋ͱͷϝϯςφϯε ੑ΋ߴ͘ͳΔ! ✤ ϛυϧ΢ΣΞͷηΩϡΞͳઃఆΛܾఆ͢Δ͜ͱ͕༰қ ʹͳΔ! ✤ ҙ֎ͱ͜Ε͕͞Εͯͳ͍έʔε͕ଟ͍

Slide 14

Slide 14 text

࠷ॳ͸શવ؅ཧग़དྷͯͳ͔ͬͨ… ✤ ߏங࣌ظʹΑͬͯઃఆ͕ҧ͏! ✤ ͋Γ·ͤΜ͔ʁ! ✤ iptablesͱtcp_wrapperͰಉ͡ϧʔϧॻ͍ͯΔ! ✤ ϦϓϨΠεΛܦͯɺ΋͸΍ଘࡏ͍ͯ͠ͳ͍ϗετ͔ ΒͷڐՄઃఆ

Slide 15

Slide 15 text

όʔδϣϯ؅ཧԽʹೖΕ࢓༷Խ ✤ Ұ୴ݱߦಈ͍͍ͯΔαʔόϕʔεͰ΋ͱΓ͋͑ͣϦϙ δτϦʹಥͬࠐΉ! ✤ ๬·͍͠ઃఆͷݟ௚͠͸ͦͷޙ੔ཧ͢Δ! ✤ มߋ౳͸τϨʔεՄೳʹͳΔͨΊޙͰݟฦͤΔ

Slide 16

Slide 16 text

ΦϖϨʔγϣϯπʔϧͷݟ௚͠

Slide 17

Slide 17 text

ෳ਺αʔόʹมߋΛՃ͑Δखஈ ✤ όʔδϣϯ؅ཧ͕ग़དྷͨΒͦΕΛల։͢Δज़͕ཁΔ! ✤ ୆਺͕૿͑Δͱมߋͷख͕ؒେม! ✤ ख͕ؒ૿͑Δͱαʔόؒͷဃ཭͕૿͑Δѱ॥؀

Slide 18

Slide 18 text

sshϧʔϓظ ✤ ຖճଧͭͷ໘౗ɺઈରϛε͢Δ! ✤ 1୆ຖ࣮ߦͰγʔέϯγϟϧɺ͕͔͔࣌ؒΔ for host in web{1..100}.heteml.lan;! do! echo ${host}! ssh ${host} ‘perl -pi -e “s/foo/var/g” /path/to/ﬁle’! done

Slide 19

Slide 19 text

Archerظ ✤ Perl੡ͷσϓϩΠπʔϧ! ✤ YAMLͰఆٛ͞Εͨઃఆʹ΋ͱ͖ͮγΣϧίϚϯυ΍ ϑΝΠϧͷσϓϩΠ͕ߦ͑Δ

Slide 20

Slide 20 text

Archerظ ✤ Pluggable! ✤ Archer::Plugin::RoleLoader! ✤ roles.yaml! ✤ ࢦఆͨ͠ϩʔϧʹ! ✤ ίϚϯυ࣮ߦ! ✤ ϑΝΠϧσϓϩΠ …! ! web:! - web001.heteml.lan! - web002.heteml.lan! mail:! - mail001.heteml.lan! - mail002.heteml.lan! - mail003.heteml.lan! ! …

Slide 21

Slide 21 text

Archerظ ✤ େྔϗετʹͳΔͱίϚϯυ࣮ߦ݁Ռͷ֬ೝ͕͠ΜͲ ͍! ✤ جຊతʹઃఆϑΝΠϧʹॻ͍ͨ಺༰Λ࣮ߦ͢ΔͨΊɺ ࡉ͔ͳಈత࡞ۀͷ༻్ʹ͙ͦΘͳ͔ͬͨ! ✤ ಛʹ2ͭΊͷཧ༝͕େมɺಥൃ͢ΔλεΫʹରԠ͖͠ Εͳ͍

Slide 22

Slide 22 text

ࣗ࡞πʔϧheteloyظ ✤ ༻్ಛԽʢίϚϯυ࣮ߦɺϑΝΠϧσϓϩΠʣ! ✤ ࣗ੡ͷPerlπʔϧ! ✤ ʮϔςϜϧʯʹʮσϓϩΠʯ! ✤ σϓϩΠपΓ͸ޙड़͢ΔPuppetͱͷ਌࿨ੑΛߴΊͨ! ✤ PuppetͷϦϙδτϦͰ؅ཧ͞Ε͍ͯΔϑΝΠϧΛ഑෍͠ ΍͘͢ͳ͍ͬͯΔ

Slide 23

Slide 23 text

ࣗ࡞πʔϧheteloyظ ✤ /var/lib/puppet/data/ ͕ PuppetϦϙδτϦͱͯ͋͠Δ! ✤ /var/lib/puppet/data/modules/web/files/etc/httpd/httpd.conf Λ! ✤ webϩʔϧʢલड़ͷroles.yaml) ͷϗετʹ഑෍͠! ✤ Apache ͷ configtest ͯ͠ OK ͳ৔߹͚ͩ restart ɺNG ͳ৔߹͸ϑΝΠϧϩʔϧόο Ϋ sudo heteloy.pl --role web \! --dist /etc/httpd/httpd.conf \! --mode 644 \! --exec ‘service httpd configtest’ \! --exec ‘service httpd restart’!

Slide 24

Slide 24 text

ࣗ࡞πʔϧheteloyظ ✤ Α࣮͘ߦ͢ΔλεΫ͸λεΫΛyamlܗࣜʹdump! ✤ Ҏ߱͸ͦͷyamlΛࢦఆͯ͠ಉ͡ॲཧΛ࣮ߦՄೳ sudo heteloy.pl --role web \! ! --dist /etc/httpd/httpd.conf \! --mode 644 \! --exec ‘service httpd configtest’ \! --exec ‘service httpd restart’ \! --configdump > httpd_task.yaml! sudo heteloy.pl --configloada httpd_task.yaml!

Slide 25

Slide 25 text

ࣗ࡞πʔϧheteloyظ ✤ ؀ڥґଘΛڐ༰͠൚༻ੑΛࣺͯͨ! ✤ ͷͰެ։͸ͯ͠ͳ͍Ͱ͢…! ✤ ೔ʑͷӡ༻ͷதͰߦ͏ཻ౓͸খ͍͕͞εϐʔυΛٻΊ ΒΕΔ࡞ۀʹ޲͍͍ͯΔ! ✤ PuppetͰͷߏ੒؅ཧ͸࣠ʹͦΕΛαϙʔτ͢Δҝͷ πʔϧ͕͋Δͱศར

Slide 26

Slide 26 text

OSηοτΞοϓࣗಈԽ

Slide 27

Slide 27 text

৽͍͠ϕϯμͷαʔόΛߪೖ ✤ εϖοΫΞοϓ͕ओ؟! ✤ औΓճ͕͠͠΍͍͢Α͏IPMIΛ౥ࡌ͠ɺԕִ஍ʢΦ ϑΟεʣ͔Βͷ࡞ۀ͕ߦ͑ΔΑ͏ʹͨ͠

Slide 28

Slide 28 text

OSͷΠϯετʔϧ͸DC೚ͤͩͬͨ ✤ IPMI͕࢖͑ΔΑ͏ʹͳΓɺࣗ෼ͨͪͰΦϖϨʔγϣϯ Մೳʹ! ✤ ͱ͸͍͑ɺҰ୆ͣͭΠϯετʔϧ͢Δͷ͸େมख͕ؒ ͔͔Δ

Slide 29

Slide 29 text

Cobbler ✤ http://cobbler.github.com! ✤ ωοτϫʔΫϒʔτͰOSΛΠϯετʔϧ͢Δࡍʹඞཁ ͳίϯϙʔωϯτ͕ͦΖ͍ͬͯΔ! ✤ ݸผͷαʔόͷઃఆ৘ใΛதԝͰूத؅ཧՄೳ! ✤ RedHat Enterprise Linux͓ΑͼͦͷΫϩʔϯOSͰར ༻Մೳ

Slide 30

Slide 30 text

Cobbler ✤ ૉͷPXEϒʔτͰؤுΕͳ͘͸ͳ͍͕! ✤ Cobbler͸dhcpd, pxe, tftp, httpdͳͲΛݸผʹηοτ Ξοϓ͢Δख͕ؒେ෼ল͚Δ! ✤ ूத؅ཧ͕༏लɺαʔόઃఆ৘ใ (system) ͰIPॏෳ͕ ͋ΔͱΤϥʔʹͳΔͳͲ

Slide 31

Slide 31 text

CobblerͱPuppetͷॅΈ෼͚ ✤ Cobbler͸ςϯϓϨʔτ΍εχϖοτ͕࢖͑Δ! ✤ ͜ΕΒΛར༻ͯ͠ɺෳ਺ϩʔϧ͋ΔαʔόΛΠϯετʔ ϧ׬ྃޙଈαʔϏε౤ೖͱ͍ͬͨ͜ͱ΋ՄೳͩΖ͏! ✤ kickstartϑΝΠϧ͸ϝϯς͠ʹ͍͘! ✤ OSΠϯετʔϧ͑͞ग़དྷΕ͹ޙ͸Puppetʹ೚ͤΔ

Slide 32

Slide 32 text

Cobbler ✤ ϗετ௥Ճ࣌͸ɺ͜ͷॲཧޙαʔόΛىಈ͢ΔͱOS ͕Πϯετʔϧ͞ΕΔ ʢωοτϫʔΫ෦෼ׂѪʣ cobbler system add \! --name web001.heteml.lan \! --hostname web001.heteml.lan \! --proﬁle CentOS6.4-x86_64 \! --kopts “ksdevide=XX:XX:XX:XX:XX syslog=192.168.X.X” \! --kickstart /var/lib/cobbler/kickstarts/CentOS6-base.ks! ! cobbler sync

Slide 33

Slide 33 text

γεςϜߏ੒؅ཧπʔϧͷಋೖ

Slide 34

Slide 34 text

ಋೖͷඞཁੑ ✤ αʔόಋೖϖʔε͕Ճ଎! ✤ ߏஙͷख͕ؒϘτϧωοΫʹ

Slide 35

Slide 35 text

طଘαʔϏε΁ͷPuppetಋೖ ✤ खॱॻΛಡΈղ͘! ✤ ඞཁͳ৔߹͸खॱॻͷ੔ཧ΋! ✤ ͖ͪΜͱඥղ͖ɺߏ੒؅ཧπʔϧͷઃఆϑΝΠϧʹམͱ ͨ࣌͠ʹͦΕ͕αʔόͷ”࢓༷”ͱͳΔ! ✤ Ұ࣌ظΛࢧ͑ͨൿ఻ͷλϨ͸ܾͯ͠ѱͷΈͰ͸ͳ͍! ✤ ݱ࣮ੈքͷͦΕͱҟͳΔͷ͸֧፩ʢ͔͘͸Μʣ͠ͳ͍ҝ

Slide 36

Slide 36 text

ݕূ͢Δ؀ڥ ✤ ࢼͯ͠ഁغͯ͠΍Γ௚͕͠ग़དྷΔVMϕʔεͰߦ͏ͷ ͕΍͸Γ΍Γ΍͍͢! ✤ ϔςϜϧͰ͸ϖύϘࣾ಺؀ڥͷKVM؅ཧπʔϧ MaglicaΛ࢖ͬͯखܰʹVM࡞੒ɺഁغΛߦͳͬͨ! ✤ ݱࡏͳΒVagrantͳͲ͕͋ΔͨΊͦΕΛར༻͢Δͱ༗ ༻ͦ͏

Slide 37

Slide 37 text

·ͣ͸৽ن௥Ճϗετ͔Βಋೖ ✤ ৽نϗετ͸طଘαʔόͱͷ͕͠ΒΈ͕ͳ͍! ✤ ೺Ѳ͍ͯ͠ͳ͍࢓༷࿙ΕͳͲ͕͋ΔՄೳੑ΋͋Γطଘ αʔό͸৻ॏʹߦ͏ඞཁ͕͋Δ! ✤ طଘϗετ΁ͷద༻͸noop(dry-run)࣮ߦ͠ɺࠩ෼֬ ೝɺඞཁ͕͋Ε͹मਖ਼ͳͲ΋ߦ͍ͭͭॱ࣍ಋೖΛਐΊ ͍ͯΔ

Slide 38

Slide 38 text

ಋೖ͕׬ྃͨ͠ޙ΋ଓ͘ ✤ ߏ੒؅ཧπʔϧͷهड़ํࣜɺϕετϓϥΫςΟε͕ม ΘΔ͜ͱ΍ɺӡ༻͍ͯ͘͠தͰෆཁͳهࡌͳͲ΋ग़ͯ ͘Δ! ✤ ࣺͯΔ༐ؾ! ✤ ఆظతʹݟ௚͠Λ͢Δඞཁ͕͋Δ

Slide 39

Slide 39 text

ಋೖ͕׬ྃͨ͠ޙ΋ଓ͘ ✤ ඞཁͳઃఆ͸όʔδϣϯ؅ཧ͞Ε͍ͯΔ! ✤ ࢓༷ͷมߋ΍ݟ௚͠͸ઃఆϑΝΠϧϕʔεͰٞ࿦͢Δ! ✤ ϖύϘͰ͸GH:EΛར༻͍ͯ͠ΔͨΊɺPull Requestϕʔ εͰ্هΛߦͳ͍ͬͯΔ! ✤ ϨϏϡʔͯ͠΋Β͑Δͱਫ਼౓͕͕͋Δ! ✤ ͋ͱ҆৺͢Δ

Slide 40

Slide 40 text

؂ࢹγεςϜͷӡ༻վળ

Slide 41

Slide 41 text

؂ࢹ͕ἧͬͯͳ͍ ✤ ؂ࢹ߲໨͕ࡏΔ΋ͷͱແ͍΋ͷ͕͋Δ! ✤ ϗετΛ௥Ճ͢Δ͝ͱʹख௥Ճ! ✤ ͦ΋ͦ΋؂ࢹ͞Ε͍ͯͳ͍ϗετ΋…ʁ

Slide 42

Slide 42 text

αʔόͷҰཡ͸࣋ͬͯΔ ✤ લड़ͷroles.yaml! ✤ ͜͜ʹ؅ཧ͍ͯ͠Δαʔό͸໢ཏ͞Ε͍ͯΔ

Slide 43

Slide 43 text

؂ࢹઃఆͷࣗಈੜ੒ ✤ Nagios! ✤ munin! ✤ roles.yaml Λ΋ͱʹ conf Λࣗಈੜ੒͢Δ! ✤ શαʔόڞ௨ͷ؂ࢹ (σΟεΫI/O, L/A)! ✤ ϩʔϧݻ༗ͷ؂ࢹ (webͩͬͨΒHTTPૄ௨౳)! ✤ ϗετݻ༗ͷઃఆ (ಛఆαʔό͸αʔό্Ͱύϥϝʔλ্ॻ͖ʣ

Slide 44

Slide 44 text

มߋ࣌ ✤ طଘͷ؂ࢹγεςϜʢखಈ௥Ճʣͱ৽؂ࢹγεςϜʢઃ ఆࣗಈੜ੒ʣΛฒߦͯ͠Քಇͤ͞Δ! ✤ Nagios৽؂ࢹγεςϜ͸Ξϥʔτඈ͹͞ͳ͍! ✤ े෼ʹ֬ೝ͔ͯ͠Β͍͑΍Ͱ੾Γସ͑

Slide 45

Slide 45 text

ৼΓฦͬͯ

Slide 46

Slide 46 text

ྺ࢙͕௕͍ͷͰτϨϯυ΋มΘΔ ✤ Puppet͸master agentܕͰ΍ͬͯΔ͚Ͳɺ࠷ۙ͸ελ ϯυΞϩϯܕ͕ྲྀߦͬͯΔʢςετۦಈΠϯϑϥߏஙʣ ✤ SerfͳͲʹΑΔࣗ཯తʢಈతʣͳϊʔυ௥Ճɾઃఆ௥ ՃͳͲ! ✤ ͳͲͳͲ

Slide 47

Slide 47 text

౎౓΍Δ͔͠ͳ͍ ✤ WebͷαʔϏεͳݶΓ׬੒͕ͳ͍ɺӡ༻ʹ΋ऴΘΓ͕ ͳ͍! ✤ ͚Ͳޮ཰తʹɺָ͘͢͠Δ͜ͱ͸ग़དྷΔ! ✤ Ұ൪ϘτϧωοΫʢ൥ࡶɺ͕͔͔࣌ؒΔʣ࡞ۀ͔Βૣ ΊʹऔΓ૊Ή! ✤ ஍ಓʹ΍Δ

Slide 48

Slide 48 text

ྫ ✤ ࠷ۙϝϯς͕ଟ͍! ✤ ର৅ϗετ͕ҧ͏͚ͩͰखॱ͸࢖͍ಓ͕ฉ͘! ✤ खॱॻ΋ੜ੒ʢςϯϓϨʔτԽʣ͠Α͏ʂ

Slide 49

Slide 49 text

खॱΛςϯϓϨʔτԽ export OLDHOST=<%= @conﬁg['src1_hostname'] %>! export NEWHOST=<%= @conﬁg['dst_lan_ip'] %>! rsync -rlptDv /data/backup/heteml/$OLDHOST/path/from/ $NEWHOST::data_to/ src1_hostname : web001! src1_lan_ip : 192.168.1.10! ! dst_hostname : web002! dst_lan_ip : 192.168.1.11!

Slide 50

Slide 50 text

खॱΛςϯϓϨʔτԽ genecon -t README.md -c conﬁg.yaml > web001maintenance.md export OLDHOST=web001! export NEWHOST=192.168.1.11! rsync -rlptDv /data/backup/heteml/$OLDHOST/path/from/ $NEWHOST::data_to/ ✤ genecon https://github.com/glidenote/genecon

Slide 51

Slide 51 text

͋Δఔ౓ͷ౓ڳ͸ඞཁ ✤ ৽͍͠΋ͷͷಋೖ΍Կ͔ͷ࡮৽͸༐ؾ͕ཁΔ! ✤ Ͱ΋΍Δ࣌ʹ͸΍Βͳ͍ͱ͍͚ͳ͍! ✤ ཪ෇͚Ͱ͋Δ֬ೝ͸ςετͳͲͰཪ෇͚Λͯ҆͠৺ ͢Δ

Slide 52

Slide 52 text

ࠓճͷ಺༰ɺ͞Βʹৄ͍͠಺༰͸ ✤ WEB+DB PRESS Vol.75! ✤ ʙෳࡶੑͷ૿େͱ؀ڥͷมԽʹରԠ͢Δʙ ܧଓతWebαʔϏεվ ળΨΠυ! ✤ ΞϓϦɺνʔϜϏϧσΟϯάͷվળͳͲଞͷτϐοΫ΋߹Θͤͯࡌͬ ͯ·͢

Slide 53

Slide 53 text

Ұॹʹಇ͘ϝϯόʔΛืू͍ͯ͠· ͢ ✤ http://www.paperboy.co.jp/recruit/engineer_career/! ✤ ϔςϜϧνʔϜͷΠϯϑϥΤϯδχΞ΋ืूதͰ͢

Slide 54

Slide 54 text

͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠