Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Goでサーバの健全性を確保する / Keeping servers healthy with Go

linyows
April 24, 2021

Goでサーバの健全性を確保する / Keeping servers healthy with Go

2021年4月24日 FUKUOKA Engineers Day 2021 Online での資料です。

linyows

April 24, 2021
Tweet

More Decks by linyows

Other Decks in Programming

Transcript

  1. FUKUOKA Engineers Day 2021 Online
    Tomohisa Oda / Apr 24, 2021
    Keeping Servers Healthywith Go
    GoͰαʔόͷ݈શੑΛ֬อ͢Δ

    View Slide

  2. FUKUOKA Engineers Day 2021 Online
    @linyows
    Blog: tomohisaoda.com


    Affiliation: GMO pepabo, inc.


    Fukuoka.go, Fukuoka.ts Organizer
    ࠷ۙɺNotionʹࣄྫ঺հهࣄͰࣅإֆΛॻ͍ͯ΋Β͍·ͨ͠ɻ


    ࠓ೔͸ͳΜͱɺGo Conference Tokyo 2021͕ࠓ͍͋ͬͯ·͢ɻ#gocon

    View Slide

  3. FUKUOKA Engineers Day 2021 Online
    ๏ ϗεςΟϯάࣄۀ෦ Senior Engineering Lead


    ࣄۀ෦ͷCTO΍VPoEͷΑ͏ͳ΋ͷ


    ๏ ϖύϘݚڀॴ ۝भେֶڞಉݚڀ


    2020 Պݚඅ࠾୒ https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-20K11791/


    ܰྔίϯςφʹΑΔେن໛ߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
    In Pepabo…

    View Slide

  4. FUKUOKA Engineers Day 2021 Online
    Private Projects
    ΞϓϦέʔγϣϯࣗ਎͕ϦϙδτϦ͔
    Β࠷৽ͷίʔυΛݕ஌ͯࣗ͠ಈͰσϓϩ
    ΠͱσϦόϦʔΛߦ͏πʔϧ
    linuxͷuser໊લղܾΛGitHubͷ
    Teamϝϯόʔ΍ϦϙδτϦͷΞΫηε
    ݖݶऀ͔Βߦ͏πʔϧ
    ϝʔϧૹ৴༻ͷಁաܕSMTPϓϩΩγͰ
    ϦΫΤετͱϨεϙϯεΛϩά΍σʔλ
    ϕʔεʹग़ྗ͢Δ͜ͱ͕Ͱ͖Δ
    New!!

    View Slide

  5. FUKUOKA Engineers Day 2021 Online
    GoͰαʔόͷ݈શੑΛ֬อ͢Δ 👨⚕

    View Slide

  6. FUKUOKA Engineers Day 2021 Online
    ๏ ϜʔϜʔυϝΠϯ — https://muumuu-domain.com


    ๏ ΁ςϜϧ — https://heteml.jp


    ๏ ϩϦϙοϓ — https://lolipop.jp


    ๏ ϚωʔδυΫϥ΢υ — https://mc.lolipop.jp
    ϗεςΟϯάࣄۀ෦

    View Slide

  7. FUKUOKA Engineers Day 2021 Online
    ๏ Domain — DNS


    ๏ Compute — HTTP, SSH, FTP and WebDav…


    ๏ Mail — SMTP, POP, IMAP…


    ๏ Database — MySQL, SQLite…
    ֤αʔόϦιʔεͷ҆ఆతఏڙͱӡ༻

    View Slide

  8. FUKUOKA Engineers Day 2021 Online
    αʔόΛ؅ཧ͢Δ͜ͱ͕ͱͯ΋ॏཁ ⚠

    View Slide

  9. FUKUOKA Engineers Day 2021 Online
    ෺ཧαʔόͷ؅ཧ
    ๏ ϗεςΟϯάࣄۀ෦͚ͩͰ෺ཧαʔό͕େମ1000୆Ҏ্͋Δ


    ๏ αʔό͸ނো͢Δ΋ͷ


    ๏ อकظؒ͸௕ͯ͘5೥ؒɺյΕͨΒഇغ͢Δ͔ɺߴֹͳඅ༻Λ෷ͬͯमཧ͢Δ͔


    ๏ αʔόͷݮՁঈ٫͸5೥ɺͭ·Γαʔόͷ଱༻೥਺͕লྩͰ5೥ͱͳ͍ͬͯΔ


    ๏ ఆظతʹαʔόΛϦϓϨʔε͢Δඞཁ͕͋Δ


    ๏ ͪͳΈʹϕϯμʔʹΑͬͯn೥΋ͭɺ΋ͨͳ͍Έ͍ͨͳͷ͸ͳΜͱͳ͋͘ΔͬΆ͍

    View Slide

  10. FUKUOKA Engineers Day 2021 Online
    Ծ૝αʔόͷ؅ཧ
    ๏ ෺ཧαʔό্ʹ૥ͷKVM΍OpenStackΛ࢖ͬͯԾ૝؀ڥΛ࡞͍ͬͯΔ


    ๏ ࠷ۙ͸OpenStack্ʹKubernetes؀ڥΛ࡞ͬͯਫ਼ྗతʹίϯςφԽΛਐΊ͍ͯΔ


    ๏ VM΋ͬ͟ͱ600Πϯελϯε΄Ͳ͋Δ


    ๏ ౰વϋʔυ΢ΣΞҎ֎ʹιϑτ΢ΣΞ΋ݹ͘ͳΔͷͰɺ੬ऑੑରԠͳͲɺOS΍ϛυϧ
    ΢ΣΞͷΞοϓσʔτ΋ܧଓతʹඞཁ


    ๏ ϑΝΠϧͷ੔߹ੑ؂ࢹ΍ڴҖݕग़΍੬ऑੑݕग़ʹWazuhΛར༻͍ͯ͠Δ͕໢ཏతͰ͸ͳ
    ͍

    View Slide

  11. FUKUOKA Engineers Day 2021 Online
    ηΩϡϦςΟͷ؅ཧ
    ๏ IDSʢ৵ೖݕ஌γεςϜʣ
    ʹWazuhΛར༻͍ͯ͠Δ


    ๏ Wazuh͸ڴҖݕ஌Ҏ֎ʹ
    ϑΝΠϧͷ੔߹ੑ؂ࢹ΍
    ੬ऑੑݕग़ʢCVE
    databaseʣ΋Ͱ͖Δ

    View Slide

  12. FUKUOKA Engineers Day 2021 Online
    ࢿ࢈ͱͯ͠ͷαʔό؅ཧ
    ๏ ߴֹͳαʔό͸ձࣾࢿ࢈ͷ̍ͭ


    ๏ ͍ͭɺͩΕ͕ɺͲ͔͜ΒɺͳʹΛɺͳΜͷͨΊʹɺ͕෼͔ΔΑ͏ʹ؅ཧ͍ͯ͠Δ


    ๏ ࣗલͷΞϓϦέʔγϣϯͱDBʹΑ࣮ͬͯݱ͍ͯ͠Δ


    ๏ Ұൠతʹ͸ɺDCIMʢData Center Infrastructure Managementʣ΍IPAMʢIP
    Adress ManagerʣͷൣᙝͰ͋Δαʔό͕ऩ༰͞ΕΔϥοΫͷ৔ॴ΍αʔόͷεϖοΫɺ
    αʔό͕࢖͍ͬͯΔIPΞυϨεͷ؅ཧ΋ͦͷΞϓϦέʔγϣϯʹೖ͍ͬͯΔ


    ๏ ࢿ࢈؅ཧ͸ͪΌΜͱͰ͖͍ͯΔ

    View Slide

  13. FUKUOKA Engineers Day 2021 Online
    😭 ؅ཧ΍ӡ༻͢Δ͜ͱ͕ΊͬͪΌ͋Δ 😭

    View Slide

  14. FUKUOKA Engineers Day 2021 Online
    ՝୊
    ๏ ෳ਺ͷαʔϏεɺෳ਺ͷνʔϜɺෳ਺ͷαʔό؀ڥʹΑΓશମ૾ͷ೺Ѳ͕Ͱ
    ͖͍ͯͳ͍


    👉 ໢ཏతʹશମ೺Ѳ͕Ͱ͖Δঢ়ଶʹ͍ͨ͠ʂ


    ๏ αʔό؅ཧΛϝΠϯʹ΍͍ͬͯΔνʔϜ͸ͳ͘ӡ༻ͷยखؒͳͨΊɺରԠ͕
    ޙखʹͳΓ͕ͪ


    👉 ੵۃతʹαʔόͷ݈શੑΛҡ࣋Ͱ͖Δঢ়گΛͭ͘Γ͍ͨʂ

    View Slide

  15. FUKUOKA Engineers Day 2021 Online
    GoͰ͋ΒΏΔαʔόͷInventoryΛͭ͘Δ

    View Slide

  16. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ

    View Slide

  17. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ

    View Slide

  18. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ

    View Slide

  19. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ

    View Slide

  20. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ

    View Slide

  21. FUKUOKA Engineers Day 2021 Online
    Architecture
    - ෺ཧαʔόೲ඼࣌ʹਓ͕ࢿ࢈DBʹొ࿥͢Δ


    - ࢿ࢈DB͔Β෺ཧαʔόͷ৘ใऔಘ͢Δ


    - KVMϗετʹSSHͯ͠ήετ৘ใΛऔಘ͢Δ


    - OpenStack͸API͔Βςφϯτ͝ͱʹ
    Instanceͷ৘ใΛऔಘ͢Δ


    - શͯΛϦετԽͨ͠ϑΝΠϧΛgit؅ཧͯ͠ຖ
    िࠩ෼Λࢦࠩ֬͠ೝ͢Δ


    - Wazuhαʔόͱagentͷ໊دͤΛͯ͠શ୆ಋ
    ೖΛ࿙Εͳ͘΍Δʢಋೖ཰ͷՄࢹԽʣ
    0
    150
    300
    450
    600
    A B C D

    View Slide

  22. FUKUOKA Engineers Day 2021 Online
    ग़དྷͨ΋ͷ

    View Slide

  23. FUKUOKA Engineers Day 2021 Online

    View Slide

  24. FUKUOKA Engineers Day 2021 Online
    ಈ͘΋ͷ͕Ͱ͖Δ·Ͱ̍िؒ
    - ։ൃͨ͠΋ͷΛҰൠԽ͠ʹ͍ͨ͘Ίɺιʔε
    ίʔυ͸ඇެ։


    - Architecture͔Β෼͔Δ௨Γฐࣾͷӡ༻ମ
    ੍ґଘ͕େ͖͍


    - Go͸ඪ४Ϟδϡʔϧ͕ॆ࣮͍ͯ͠ΔͷͰ֎
    ෦ϥΠϒϥϦ͸ൺֱతʹগͳ͍

    View Slide

  25. FUKUOKA Engineers Day 2021 Online
    Test
    - Integrationςετ͸ྗٕͬΆ͍͕ɺSSH
    αʔόΛGoͰىಈͤͯ͞Stub͢ΔΑ͏ʹ͠
    ͍ͯΔ


    - ͨͩɺSSHαʔόॻ͘ͷ΋݁ߏखܰͰ170ߦ
    ͙Β͍Ͱॻ͚Δ


    $ wc -l ssh_test.go


    170

    View Slide

  26. FUKUOKA Engineers Day 2021 Online
    SSHӽ͠ʹTest
    - SSHαʔό͕ड͚औͬͨೖྗ͔Βग़ྗ͢Δ
    ಺༰ΛఆٛΛ͢Δ


    - goroutineͰSSHαʔόΛىಈ͢Δ


    - ର৅ͷϝιουΛݺͼग़͢


    - ظ଴͢ΔίϚϯυΛSSHαʔό͕ड͚औͬ
    ͍ͯΔ͔Λݕূ͢Δ

    View Slide

  27. FUKUOKA Engineers Day 2021 Online
    SSHӽ͠ʹTest
    - SSHαʔό͕ड͚औͬͨೖྗ͔Βग़ྗ͢Δ
    ಺༰ΛఆٛΛ͢Δ


    - goroutineͰSSHαʔόΛىಈ͢Δ


    - ର৅ͷϝιουΛݺͼग़͢


    - ظ଴͢ΔίϚϯυΛSSHαʔό͕ड͚औͬ
    ͍ͯΔ͔Λݕূ͢Δ

    View Slide

  28. FUKUOKA Engineers Day 2021 Online
    SSHӽ͠ʹTest
    - SSHαʔό͕ड͚औͬͨೖྗ͔Βग़ྗ͢Δ
    ಺༰ΛఆٛΛ͢Δ


    - goroutineͰSSHαʔόΛىಈ͢Δ


    - ର৅ͷϝιουΛݺͼग़͢


    - ظ଴͢ΔίϚϯυΛSSHαʔό͕ड͚औͬ
    ͍ͯΔ͔Λݕূ͢Δ

    View Slide

  29. FUKUOKA Engineers Day 2021 Online
    SSHӽ͠ʹTest
    - SSHαʔό͕ड͚औͬͨೖྗ͔Βग़ྗ͢Δ
    ಺༰ΛఆٛΛ͢Δ


    - goroutineͰSSHαʔόΛىಈ͢Δ


    - ର৅ͷϝιουΛݺͼग़͢


    - ظ଴͢ΔίϚϯυΛSSHαʔό͕ड͚औͬ
    ͍ͯΔ͔Λݕূ͢Δ

    View Slide

  30. FUKUOKA Engineers Day 2021 Online
    Metrics
    - ύϑΥʔϚϯεϞχλϦϯά͸PrometheusΛ࢖͍ͬͯΔ


    - PrometheusͷΧελϜίϨΫλͱͯ͠ϑΝΠϧΛग़ྗ


    - ෺ཧαʔόͷՔಇ೥਺Λyoung, middle,oldͱͯ͠෼ྨ͠ɺ
    αʔϏε͝ͱͷαʔόͷ݈શੑΛՄࢹԽ͍ͯ͠Δ


    - ՄࢹԽ͢Δ͜ͱͰੵۃతͳαʔόϦϓϨʔε͕ՄೳʹͳΔͱߟ
    ͍͑ͯΔ

    View Slide

  31. FUKUOKA Engineers Day 2021 Online
    ๏ Ϗδωε্͋ΒΏΔαʔόΛ؅ཧ͍ͯ͘͜͠ͱ͕ॏཁͰ͋Δ


    ๏ ෺ཧαʔό΍ෳ਺؀ڥͷԾ૝αʔόʹରͯ͠໢ཏతͳ୆ாΛGoΛ࢖ͬͯ࡞੒ͨ͠


    ๏ ୆ாͷ৘ใ͔Βશαʔό࿙Εͳ͘IDSΛಋೖ͢Δ͜ͱ΍ɺੵۃతͳαʔόϦϓϨʔεΛߦ
    ͏؀ڥΛ੔උͰ͖ͨ


    ๏ ΍΍ෳࡶͳߏ੒ͷεΫϦϓτͰ΋ൺֱత؆୯ʹIntegration TestͰ͖Δ
    Conclusion

    View Slide

  32. FUKUOKA Engineers Day 2021 Online
    Fukuoka.go͸ɺGoݴޠΛத৺ʹιϑτ΢ΣΞΤϯδχΞϦϯάͷڞ༗Λߦ͏ίϛϡ
    χςΟͰ͢ɻ2020೥ͷΤϯδχΞϑϨϯυϦʔγςΟ෱ԬΞϫʔυίϛϡχςΟ෦໳
    Ͱड৆͠·ͨ͠ɻ


    ڵຯ͕͋Γ·ͨ͠ΒɺConnpassͳͲ͔Β͝ࢀՃ͍ͩ͘͞ʂ


    https://fukuokago.dev/
    Fukuoka.go

    View Slide

  33. FUKUOKA Engineers Day 2021 Online
    Thank you \ʕ⊙౪⚆ʔ/

    View Slide