Upgrade to Pro — share decks privately, control downloads, hide ads and more …

たまにはオンプレもいいぞ?GKE と Armadillo IoT で実現するハイブリッドクラウド

たまにはオンプレもいいぞ?GKE と Armadillo IoT で実現するハイブリッドクラウド

More Decks by 株式会社ビットキー / Bitkey Inc.

Other Decks in Technology

Transcript

  1. ࠤʑ໦ྃ 3ZP4BTBLJ 2012/04 2017/07 ๭ ௨৴ܥSIer ೖࣾ ɾݩʑ͸NWΤϯδχΞͩͬͨͷͰBGP࢖ͬͨଞڌ఺NW࡞ͬͨΓͱ͔ ɾ࣏ࣗମͱ͔׭ެிܥγεςϜʹܞΘͬͨΓ͢Δ͜ͱ͕ଟ͔͔ͬͨͳ ɾڮྊ఺ݕϩϘοτ։ൃ΍ͬͨΓͱ͔

    ϑϦʔϥϯεʹస޲ ɾϖωτϨʔγϣϯςετ΍ͬͨΓͱ͔ ɾOpenStatkͱOpenShift݁ߏ΍ͬͯͨΓͱ͔ ɾϋΠϒϦουΫϥ΢υ࡞ͬͨΓͱ͔ 2021/11 ϏοτΩʔʹSREͱͯ͠δϣΠϯ ※ͿͬͪΌ͚͋·ΓSRE΍ͬͯͳ͍͚ͲͶ ެͷ৔Ͱਓʹઆ໌Ͱ͖ΔΑ͏ͳً͔͍͠ܦྺ͸࣋ͬͯͳ͍ͷͰ αΫοͱྲྀ͠·͢ʂʂʂ
  2. CONFIDENTIAL © Bitkey Inc. All rights reserved. 8 Connect everything

    through the power of technology, securely and conveniently in a way, that simply feels good ςΫϊϩδʔͷྗͰɺ͋ΒΏΔ΋ͷΛ ҆શͰ ศརͰ ؾ࣋ͪΑ͘ ʮͭͳ͛Δʯ Our Mission
  3. 13

  4. 14

  5. 17

  6. 19

  7. ࠓ·Ͱͷߏ੒ͱ໰୊఺ 25 • ϏοτΩʔͷϓϩμΫτͷதͰ
 །Ұ෺ཧαʔόʔΛ༻͍ΔΦϯϓϨߏ੒ ◦ ͦΕҎ֎͸શͯϑϧαʔόʔϨεߏ੒ ◦ GCF /

    CloudRun / GKE (ࠓճ͸ৄࡉΛׂѪ) • ࣾ಺ʹΦϯϓϨଆΛѻ͏ٕज़తͳϊ΢ϋ΢͕๡͍͠ • Մ༻ੑ͕௿͘ɺো֐࣌ʹ͸खಈͰͷ੾Γସ͕͑લఏ • ෺ཧͳΒͰ͸ͷ௕ظؒ࢖͍ଓ͚ΔͨΊͷӡ༻อकҙ͕ࣝ๡͍͠ ໰୊఺
  8. ࠓ·Ͱͷߏ੒ͱ໰୊఺ • HWతͳނোʹର͢Δඋ͕͑ෆे෼ͩͬͨ ◦ ؂ࢹମ੍͕ͳ͍ ◦ NWػث΍αʔόʔʹԿ͔͕ى͖ͨ࣌ͷ෮چखॱΛཱ֬Ͱ͖͍ͯͳ͍ • Ξοϓσʔτ͕ࠔ೉/೉ղ ◦

    ͦ΋ͦ΋24365ՔಇͳͷͰ؆୯ʹࢭΊΒΕͳ͍ͱ͍͏ཁૉ͕େ͖͍ ◦ ͦͷ্ͰɺࣗಈతͳσϓϩΠͷ࢓૊Έ͕͋ͬͨΘ͚Ͱ΋ͳ͍ͷͰɺ
 ຖճ͓٬༷ͱௐ੔ͷ্ɺϝϯςφϯε࣌ؒΛ΋Βͬͯ
 σϓϩΠίϚϯυΛୟ͘ɾɾɾΈ͍ͨͳײ͡ 27 ໰୊఺
  9. ৽ߏ੒ʹ޲͚ͯ 33 • ͱʹ͔͘໘౗͝ͱ͕ଟ͍ΦϯϓϨଆͷཁૉΛݮΒ͍ͨ͠ ◦ ࣾ಺ʹৄ͍͠ΤϯδχΞ͕গͳ͍ ◦ ݱঢ়͸ࣾ֎ϕϯμʔͱڠྗ͕ඞਢ ◦ ͦΜͳঢ়ଶ͔Βൈ͚ग़͍ͨ͠

    • ՄೳͳݶΓࣗ෼ͨͪͷಘҙ෼໺ʹدͤͯεΩϧͷԠ༻ੑΛߴΊ͍ͨ ◦ ͦ͏͢Δ͜ͱͰɺࣗ෼ͨͪࣗ਎Ͱਖ਼͘͠ߏ੒΍࢓૊ΈΛઃܭ/ධՁ/൑அ͍ͨ͠ ૝͍
  10. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ ίϯηϓτ 46 • ΦϯϓϨଆʹ͸αʔόʔ͸ஔ͔ͳ͍ • ৑௕ੑΛ࣋ͨͤͨNWػثΛ࠷খݶஔ͘ • ΞϓϦέʔγϣϯػೳ͸શͯGCPͷGKEʹஔ͘

    ◦ ϚϧνϦʔδϣϯߏ੒ͰՄ༻ੑΛ͞Βʹ޲্ ◦ AutopilotϞʔυʹΑΔӡ༻ͷ؆ུԽ • ϞόΠϧճઢΛ࢖ͬͨϚϧνΩϟϦΞܦ࿏Λ࣋ͨͤΔ ◦ VPNʹΑͬͯϩʔΧϧΤϦΞΛΫϥ΢υ·ͰԆ৳ • ୯Ұ෺ཧߏ੒Ͱ͋ΒΏΔ͓٬༷ͷ؀ڥͱ੍ޚ൫ʹରԠ ※͝ΊΜͳ͍͞ɻࣾ಺ࣄ৘͕͋Γ·ͯ͠ɺ͜ͷ෦෼͸ࠓ೔͸ࡱӨ͠ͳ͍Ͱ͍ͩ͘͞ɻ͋ͱͰඞͣࢿྉެ։͠·͢ɻ
  11. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 47 ͜Ε͑͞ೖΕͯ͘ΕͨΒɾɾɾ • ؆୯ʹ੍ޚ൫࿈ܞͰ͖·ͬͤʂ • Ϗϧશମ੍ޚͰ͖ͪΌ͏Αʂ ͱ͍͏ ϏϧΦʔτϝʔγϣϯͷ


    ਨ௚౷߹ΞϓϥΠΞϯεΛఏڙ ※࣮͸·໊ͩલ͕ܾ·ͬͯ·ͤΜ ※͝ΊΜͳ͍͞ɻࣾ಺ࣄ৘͕͋Γ·ͯ͠ɺ͜ͷ෦෼͸ࠓ೔͸ࡱӨ͠ͳ͍Ͱ͍ͩ͘͞ɻ͋ͱͰඞͣࢿྉެ։͠·͢ɻ
  12. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 48 • ࣮ࡍʹ੍ޚ൫࿈ܞΛߦ͏ΞϓϦέʔγϣϯ͸GoͰ࣮૷ • ੍ޚ൫࿈ܞΛந৅Խ ◦ ༷ʑͳ੍ޚ൫༻ͷϓϩτίϧΛαϙʔτ͠ɺෳ਺ϕϯμʔͷ੍ޚ൫ΛૢΕΔ ◦

    TCP / UDP / ಠࣗϓϩτίϧ etc. • ͦͷΞϓϦέʔγϣϯΛίϯςφԽ͠GKEʹͯՔಇͤ͞Δ • VPNΛ௨ͯ͡Ϗϧ಺ͷNWʹೖΓࠐΈϩʔΧϧ௨৴ͤ͞Δ
  13. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 51 • ෺ཧߏ੒ύλʔϯ͸1ͭͷΈ • ͦͷ1ͭͰ͢΂ͯͷ͓٬༷ʹରԠͰ͖Δ ◦ Θ͟Θ͟ճઢΛҾ͖ࠐΈͨ͘ͳ͍έʔεͷNW ◦

    ճઢΛҾ͖ࠐΉέʔε ◦ طઃNW͕ଘࡏ͢Δέʔε • Ͳͷ͓٬༷Ͱ͋ͬͯ΋૊ΈࠐΈՄೳͰɺ
 ͔ͭɺ૝ఆ͢ΔՄ༻ੑΛൃشͰ͖ΔΑ͏ͳNWతͳॊೈੑΛ֬อ
  14. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 53 • ϑΝʔϜ΢ΣΞ͕ೋॏԽ͞Ε͓ͯΓສ͕ҰͷϦεΫ௿ݮ͕Մೳ ◦ A໘͕ͿͬյΕͯ΋B໘ʹ੾Γସ͑ͯಀ͛ΕΔ ◦ ෺ཧతʹॿ͚Δ͜ͱ͕Ͱ͖ͳ͍ԕִ؀ڥͰ΋҆৺҆શʹΞοϓσʔτ͕Մೳ •

    ϑΝʔϜ΢ΣΞΛؙ͝ͱωοτܦ༝ͰೖΕସ͑ΒΕΔ ◦ ։ൃ؀ڥଆͰ࡞ΓࠐΈɺ࣮ػʹରͯ͠͸ؙ͝ͱϑΝʔϜ΢ΣΞΛॻ͖ࠐΉײ͡ ◦ Ansible΍TerraformͳͲͷIaCʹཔΒͳͯ͘΋OK (࢖ͬͯ΋͍͍) IoTͱͯ͠ͷ࠾༻ྫ͕ଟ͍ArmadilloΛ࠾༻
  15. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 54 • NIC΍ిݯ͕ೋॏԽ͞Ε͍ͯΔΘ͚Ͱ͸ͳ͍ • LTE΋ػث୯ମͰ͸1ΩϟϦΞ͔͠ѻ͑ͳ͍ • ͔͠͠ɺܰྔͰ҆ՁͳArmadilloΛͨ͘͞Μ༻ҙͯ͠εέʔϧΞ΢τੑΛ࣋ͨͤΔ ◦

    ୯ҰͷArmadillo͕μϝʹͳͬͯ΋෭ܥʹউखʹ੾ΓସΘΕ͹͍͍͚ͩ ◦ ߴ͍͓ۚΛ෷ͬͯڧ͍αʔόʔͰ಺෦తʹೋॏԽͤ͞Δ΄ͲͰ΋ͳ͍ ◦ έʔϒϧܨ͔͗͑ͯ͸͍ऴΘΓʂͷΑ͏ͳަ׵༰қੑΛ͓࣋ͨͤͯ͘ IoTͱͯ͠ͷ࠾༻ྫ͕ଟ͍ArmadilloΛ࠾༻
  16. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ Flexibility 55 • GCPଆʹϝΠϯͷΞϓϦέʔγϣϯػೳ͕͋ΔͨΊͲ͏ʹͰ΋ͳΔ • ݪଇGKEͰϗεςΟϯά͍ͯ͠Δ͕ɺ౰વଞͷGCPαʔϏε΋ར༻ͯ͠࿈ܞͰ͖Δ • k8sͱͯ͠ͷڧྗͳΤίγεςϜʹ৐͔ͬΓͭͭɺ


    AutopilotʹΑͬͯ໘౗ͳϊʔυ؅ཧ͸ࣗ෼ୡͷख͔Β཭͢ ◦ AutopilotͱStandardͷҧ͍: https://cloud.google.com/kubernetes-engine/docs/resources/autopilot-standard-feature-comparison?hl=ja • ࠓޙɺPod΍ServiceΛ૿΍ͨ͘͠ͳͬͯ΋͍͘ΒͰ΋Ͳ͏ʹͰ΋ͳΔ
  17. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 57 • NWػೳ͸ strongswan + BIRDͰVPN + BGPΛ࣮ݱ

    ◦ ArmadilloʹOSSΛ૊Έ߹ΘͤΔ͜ͱͰɺ
 ࢢ৔ʹग़ճ͍ͬͯΔ·ͱ΋ͳNWػثʹෛ͚ͳ͍ػೳΛಘΒΕΔ ◦ ΋ͪΖΜIPsecͷHWΦϑϩʔυͱ͔Ͱ͖ͳ͍͠ɺ
 ϧʔςΟϯάੑೳ΋εΠονϯάೳྗ΋ຊ৬͸ෛ͚Δ͚Ͳɺ
 ࠓճͷϫʔΫϩʔυతʹ͸શવ༨༟ Flexibility
  18. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ Scalability 59 • GKEʹΑͬͯɺΞϓϦέʔγϣϯଆͰ৭ʑ΍Γͨ͘ͳͬͯ΋Ͳ͏ʹͰ΋Ͱ͖Δ ◦ ඞཁͳͱ͖ʹඞཁͳ͚ͩ૿΍ͤΔ • ݪଇɺΦϯϓϨଆ΋εέʔϧΞ΢τͤ͞Δํ޲ͰͲΜͲΜڧ͍͚ͯ͘͠Δ

    ◦ ΋ͪΖΜ͸Ͳ͔͜Ͱݶք͸ܴ͑Δ͕ɺϫʔΫϩʔυతʹ
 ͦ͜·Ͱڧ྽ʹεέʔϧͤ͞Δ͜ͱ͸ͳ͍ ◦ લड़ͷ௨Γϙʔτ਺΋ͦͦ͋͜͜ΔL3εΠονͷ͓͔͛Ͱ͔ͳΓԿͰ΋΍ΕΔ
  19. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ Agility 60 • ύϒϦοΫΫϥ΢υଆ͸ݴΘͣ΋͕ͳॠ࣌ʹཉ͍͠΋ͷ͕ἧ͏ • ΦϯϓϨଆ͸͋͑ͯࠃ࢈ϝʔΧʔ඼ͷೲظ୹Ίͷௐୡ͠΍͍͢ϞϊΛબఆ • ͜ͷ൒ಋମෆ଍ͷ؀ڥԼʹ͓͍ͯ΋ɺ͓͓ΉͶ1िؒͰௐୡՄೳ

    ◦ ࠃ֎ϝʔΧʔ඼ͩͱϞϊʹΑͬͯ͸Լख͢Ε͹ௐୡ·Ͱ਺ϲ݄͔͔Δɾɾɾ • “յΕͳ͍”͜ͱΛ໨ࢦ͢ͷͰ͸ͳ͘ɺ҆Ձʹૉૣ͘ަ׵ͯ͠෮چͰ͖ΔػಈྗΛಘΔ
  20. ৽ߏ੒: ϋΠϒϦουΫϥ΢υ 61 • Մ༻ੑ ◦ ϚϧνϦʔδϣϯߏ੒ / ϚϧνΩϟϦΞNWΛલఏʹͨ͜͠ͱͰେ෯ʹ޲্ ◦

    چߏ੒Ͱ͸ਓؒʹΑΔݱ஍࡞ۀͷ੾Γସ͕͑ඞཁ͕ͩͬͨ৽ߏ੒Ͱ͸ࣗಈ੾Γସ͑ • ίετ ◦ چߏ੒ͷ1/3ఔ౓ʹѹॖ • ӡ༻อकੑ ◦ ॳظઃܭ࣌఺Ͱ24365Λݟਾ͑ͨϞχλϦϯάΛ૝ఆ ◦ ӡ༻อकϑϩʔ΋੔උ͠ɺඇΤϯδχΞͰ͋ͬͯ΋Ұ࣍ରԠ͕Մೳ
  21. ঎༻ϋΠϒϦουΫϥ΢υͷͨΊͷໝ૝ྗ 65 ݪଇɺͲͷύϒϦοΫΫϥ΢υ΋ׂͱ؆୯ʹϋΠϒϦουΫϥ΢υࣗମ͸࡞ΕΔ • VPN: IPsec + BGPʹΑΔSite-to-Site͔ͭμΠφϛοΫͳNWΛߏங • ઐ༻ઢ:

    ࣮֬ / ߴ଎ / ҆ఆʹ௚઀తͳϩʔΧϧNWΛߏங ͲͪΒ΋ɺͲͷύϒϦοΫΫϥ΢υϓϥοτϑΥʔϜͰαϙʔτ͍ͯ͠Δɻ ࣄྫ΋ׂͱੈͷதʹ͍͘ΒͰ΋͋ΔͷͰɺ࣮͸ϋΠϒϦουΫϥ΢υࣗମΛ࡞Δͷ͸؆୯ɻ
  22. ঎༻ϋΠϒϦουΫϥ΢υͷͨΊͷໝ૝ྗ ώϯτ 74 • αʔόʔଆʹ͸Կ΋໰୊͸ͳ͍ ◦ ෦඼͕ނো͍ͯ͠Δͱ͔ɺOSͷજࡏతόάͱ͔Ͱ΋ͳ͍ • NWػثʹ΋Կ΋໰୊͸ͳ͍ •

    ͨͩ1୆ʹ͚ͩࣄ৅͕ൃੜ ◦ ಉ͡NWʹܨ͕͍ͬͯΔଞͷαʔόʔʹ͸Կ΋໰୊͕ͳ͍ • ͦͷࣄ৅͕ൃੜ͢Δ௚લʹ͸গ͠ϥοΫΛ৮Δ࡞ۀ͕͋ͬͨ
  23. ঎༻ϋΠϒϦουΫϥ΢υͷͨΊͷໝ૝ྗ ໝ૝ྗ: Ϋϥ΢υ 81 • Ϋϥ΢υͷ୯Ұκʔϯͷো֐ : ౰વ͋Δ͔ΒϦʔδϣφϧͳߏ੒ʹ͠Α͏ • Ϋϥ΢υͷ୯ҰϦʔδϣϯͷো֐:

    ͋Γ͏ΔΑͶɺϚϧνϦʔδϣϯͳߏ੒ʹ͠Α͏ • ೔ຊࠃ಺ͷ2Ϧʔδϣϯͷಉ࣌ো֐: ͳ͘͸ͳ͍͚Ͳɺͦ͜·Ͱ΍Δʁʁ • Googleͷւఈέʔϒϧͷো֐: ΋͸΍ϚϧνΫϥ΢υʹ͠ͳ͍ͱແཧ
  24. ঎༻ϋΠϒϦουΫϥ΢υͷͨΊͷໝ૝ྗ ໝ૝ྗ: ΦϯϓϨ 82 • ෺ཧܥ ◦ ిݯϢχοτ͕ࢮ͵ ◦ NIC͕ࢮ͵

    (Ͳ͔͜ͷϙʔτ͚ͩࢮ͵) ◦ ࢮ͵ͳΒ·ͩϚγ͕ͩɺத్൒୺ͳ൒ࢮঢ়ଶʹͳΔ ◦ ϝϞϦͷECCΤϥʔ͕ଟൃͯ͠அଓతʹ࠶ىಈ͔͔Δ ◦ έʔϒϦϯάෆඋ ◦ ͦ΋ͦ΋έʔϒϧͷ඼࣭͕௿͍ ◦ पғ؀ڥͷϊΠζ͕ܹ͍͠ ◦ ۭௐ͕ͳ͍ or ۭௐͷௐࢠ͕ѱͯ͘ॵ͍ ◦ ΠϯλʔωοτճઢͷΩϟϦΞো֐ ͜Μͳͷংͷޱɻ ·ͩ·ͩͨ͘͞Μ͋Δɻ
  25. ঎༻ϋΠϒϦουΫϥ΢υͷͨΊͷໝ૝ྗ 85 • ΋ͪΖΜྺ࢙͕͋ΔͷͰɺϕετϓϥΫςΟεతͳͷ͸ͨ͘͞Μ͋Δ ◦ Ϋϥ΢υଆ΋ͦ͏ͩ͠ • ॴҦϕετϓϥΫςΟεతͳΫϥ΢υΞʔΩςΫνϟ΍෺ཧߏ੒Λऔ͍ͬͯͯ΋ɺ
 ࣮͸͜Μͳ͜ͱ͕͋Δͱো֐ʹؕΔΑͶɾɾɾʁͱ͍͏ͷ͸શવ͋Δ ◦

    ϕετϓϥΫςΟε͸͜͏΍Ε͹ྑ͍ײ͡ʹͳΔͥͷಓҊ಺ΨΠυͳ͚ͩͰɺ
 શ͕ͯ໢ཏ͞ΕͨΞ˓ςΟϚχΞͰ͸ͳ͍ ◦ ͖ͬ͞ͷޫϑΝΠόʔͷྫͳΜͯڭՊॻతͳϕετϓϥΫςΟεʹ͸ࡌͬͯͳ͍ ͦ͜Λࣗ෼ୡͷߏ੒ͱϏδωεʹরΒ͠߹ΘͤͯᖰΓग़͢ͷ͕ໝ૝ྗ
  26. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ 90 • GCPͷ৔߹ɺϕʔεͱͳΔVPC͸NWతͳՄ༻ੑͱͯͦ͠΋ͦ΋ϚϧνϦʔδϣϯ ◦ VPN͸ݪଇϦʔδϣφϧ (Ϛϧνκʔϯ) ◦ ౰વɺτϯωϧΛ2ຊҎ্͸Δඞཁ͸͋Δ ▪

    ୯ҰϦʔδϣϯʹ2ຊҎ্͸ͬͯϦʔδϣφϧ ◦ ͦΕͧΕͷϦʔδϣϯʹରͯ͠VPNΛ2ຊҎ্(߹ܭ:࠷௿4ຊ)
 ͋Ε͹ϚϧνϦʔδϣϯͳVPNܦ࿏Λߏ੒Ͱ͖Δ ઌ΄Ͳͷߏ੒ਤͩͱਅΜதͷ͜ͷ෦෼
  27. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ 93 • GKEͷ৔߹͸VPCωΠςΟϒʹՔಇͤ͞ΒΕΔ ◦ k8sͷ஌ࣝΛ׆͔ͤΔͷͰӡ༻΋ɺ·͊΍Γ΍͍͢ ◦ AutopilotΛલఏʹ͢Δ͜ͱͰϑϧϚωʔδυʹ΍ΕΔͱͳ͓ྑ͠ ▪ ΋ͪΖΜཁ݅࣍ୈͰ͸͋Δ͕ɺΑ΄Ͳॊೈʹ΍Γ͍ͨέʔεͰ΋ͳ͚Ε͹ɺAutopilotʹ

    Αͬͯk8sͱͯ͠ͷڧྗͳΦʔέετϨʔγϣϯೳྗ͸ڗडͭͭ͠ϊʔυ؅ཧͳͲͷ໘౗ ͝ͱ͸೚ͤͪΌ͏ ◦ ࢀߟ: https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview?hl=ja
  28. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ 95 • HTTPS௨৴͕લఏͳΒ Multi Cluster Ingress ͕؆୯ ◦ ࢀߟ:

    https://cloud.google.com/kubernetes-engine/docs/concepts/multi-cluster-ingress?hl=ja ◦ ࠓճͷ৔߹ɺHTTPʹݶΒͳ͍༷ʑͳϓϩτίϧΛѻ͏ඞཁ͕͋Δ ◦ ͭ·Γ࢖͑ͳ͍ɺෆ࠾༻ • ࣅͨཧ༝ͰAnthos Service Mesh΋ෆՄ ◦ UDPΛѻ͑ͳ͍ɾɾɾϚϧνϦʔδϣϯͳLBͳΒ݁ہ֎෦LB͕લఏ ◦ ࢀߟ: https://cloud.google.com/service-mesh/docs?hl=ja
  29. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ߟ͑͏Δ༗ޮࡦ 100 • େલఏͱͯ͠NWͰϚϧνϦʔδϣϯͳμΠφϛοΫϧʔςΟϯάΛ࣮ݱͤ͞Δ • ͦ͏΍ͬͯNWతͳ৑௕ܦ࿏Λ୲อ্ͨ͠Ͱɺ
 GCPͷCloudDNSϧʔςΟϯάϙϦγʔ&ϔϧενΣοΫΛซ༻ ◦ ໊લղܾΛ༻͍ͨϑΣΠϧΦʔόʔύλʔϯͱ͍͏͜ͱ

    ◦ ެࣜΨΠυ: https://cloud.google.com/dns/docs/zones/manage-routing-policies ◦ ެࣜϒϩά: https://cloud.google.com/blog/ja/products/networking/introducing-automated-failover-for-private- workloads-using-cloud-dns-routing-policies-with-health-checks
  30. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ߟ͑͏Δ༗ޮࡦ 101 • ಺෦ར༻ͷͨΊͷϓϥΠϕʔτκʔϯΛ࡞͓ͬͯ͘ • ͦͷκʔϯʹର͢ΔAϨίʔυͱͯ͠ɺFailover PolicyΛ࣋ͨͤΔ ◦ PrimaryܥͷClusterͷServiceʹϔϧενΣοΫΛ͔͚͓ͯ͘

    ◦ ݪଇతʹ໊લղܾ݁Ռ͸ͦͪΒʹͳΔ ◦ ো֐ʹؕͬͨΒSecondaryଆʹ໊લղܾ݁Ռ͕੾ΓସΘΔײ͡ • GEO Policiesͱ͍͏ܗͰΑΓ͍ۙϦʔδϣϯʹܨ͕ΔΑ͏ʹ໊લղܾ݁ՌΛม͑Δ͜ͱ΋Մೳ • ͨͩ͠ࠓճͷྫͷΑ͏ʹສ͕ҰͷࡍʹผϦʔδϣϯʹಀ͛ΕͨΒͦΕͰྑ͍͍ͬͯ͏࣌͸ɺFailover Policies Ͱे෼
  31. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ஫: ݱঢ়GUIαϙʔτ͕ͳ͍ͷͰCLIΛ࢖͏ඞཁΞϦ 103 # gce-vm-ipͳNGEΛ࡞੒ gcloud compute network-endpoint-groups create

    tokyo-neg \ --network=default \ --subnet=default \ --network-endpoint-type=gce-vm-ip \ --zone=asia-northeast1-b # backend-serviceΛ࡞Δ࣌఺Ͱগ͠ύϥϝʔλʔΛ૿΍͢ gcloud compute backend-services create tokyo-backend-service \ --region=asia-northeast1 \ --load-balancing-scheme=INTERNAL \ --protocol=TCP \ --health-checks=your-healthcheck \ --health-checks-region=asia-northeast1 # gce-vm-ipͳNEGΛόοΫΤϯυͱͯ͠௥Ճ͢Δ gcloud compute backend-services add-backend tokyo-backend-service \ --network-endpoint-group=tokyo-neg \ --network-endpoint-group-zone=asia-northeast1-b \ --region=asia-northeast1
  32. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ 104 # Routing Policyͱͯ͠Ξλον͢ΔͨΊͷForwarding Rule gcloud compute forwarding-rules tokyo-forwarding-rule

    \ --region=asia-northeast1 \ --load-balancing-scheme=internal \ --network=default \ --subnet=default \ --ip-protocol=TCP \ --ports=8080 \ --backend-service=your-backend-service \ --backend-service-region=asia-northeast1 # ౦੢৑௕ͷͨΊͷForwardingRuleΛͦΕͧΕΞλον͠ͳ͕ΒFailoverՄೳͳAϨίʔυΛ࡞੒ gcloud dns record-sets create ha.example.local \ --ttl=30 \ --type=A \ --zone=your-private-zone \ --routing-policy-type=FAILOVER \ --enable-geo-fencing \ --routing-policy-primary-data=tokyo-forwarding-rule \ --routing-policy-backup-data-type=GEO \ --routing-policy-backup-data="asia-northeast2=osaka-forwarding-rule" \ --backup-data-trickle-ratio=0 \ --enable-health-checking
  33. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ͳΒͲ͏͢Δʁ 108 • ࠓճɺ୯७ʹো֐࣌ʹผϦʔδϣϯʹಀ͛ΒΕ͹ྑ͍͚ͩ (ϑΣΠϧΦʔόʔͰOK) • Armadillo಺෦Ͱ֤GKEΫϥελʔͷServiceʹରͯ͠ϔϧενΣοΫΛ͔͚ͯ
 ͦΕ͕མͪͨΒτϯωϧΛམͱ͢σʔϞϯίϯςφΛՔಇͤ͞Δ ◦

    τϯωϧ͕མͪΕ͹BGP Keepalive΋౰વམͪΔͷͰϧʔτ͕੾ΓସΘͬͯɺ
 ηΧϯμϦͷผϦʔδϣϯʹܨ͕Δ ◦ CiscoͰ͍͏IP SLA Object TrackingͷΠϝʔδ • ΫϥελʔԣஅͰActive-ActiveͰಈ͚ΔΞϓϦέʔγϣϯͷܗͱ͓͖ͯͭͭ͠ɺ
 ్தͰԿ͕͋ͬͯ΋ؼΓͷ௨৴͕ಧ͘ܗΛ࡞Γ্͛Δ
  34. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ GCP Autopilot؀ڥʹ͓͚ΔγϯάϧϦʔδϣϯ؀ڥͰͷμ΢ϯλΠϜܰݮࡦ 110 • αʔδΞοϓάϨʔυͱPodDisruptionBudgetͷซ༻Ͱ
 ϝϯςφϯε࣌ʹ͓͍ͯ΋αʔϏεػೳͱͯ͠ͷμ΢ϯλΠϜΛ͔ͳΓܰݮͰ͖Δ ◦ ॱ࣍ɺϊʔυͷΞοϓάϨʔυͳͲ͕ೖͬͯ΋γʔέϯγϟϧʹ࣮ߦ͞Εɺ
 ͔ͭɺPodͷ࠷௿਺͕อͨΕͨ··ʹͳΔͷͰɺ


    ಛఆ࣌ؒͰPod͕શ໓͢Δ͜ͱ͕ͳ͍ = ࣄ্࣮ͷμ΢ϯλΠϜൃੜ͠ͳ͍ • αʔδΞοϓάϨʔυ͸Autopilotͷ৔߹σϑΥϧτͰઃఆ͞Ε͍ͯΔͷͰ
 ͋·Γҙࣝ͠ͳͯ͘΋OK • ࢀߟ: https://cloud.google.com/blog/ja/products/containers-kubernetes/introducing-surge-upgrades-for-anthos-gke?hl=ja
  35. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ArmadilloͷԼ४උ 111 • ArmadilloͷσϑΥϧτLinux Kernel͸࠷௿ݶͷػೳ͔͠૊Έࠐ·Ε͍ͯͳ͍ ◦ ೝূ΍҉߸ԽΛ࢘ΔAH΍ESPؔ࿈ͷΧʔωϧϞδϡʔϧ͕૊Έࠐ·Ε͍ͯͳ͍ ◦ ͭ·ΓσϑΥͰ͸IPsecΛ͸Εͳ͍ͷͰKernelΛ࠶Ϗϧυ͢Δඞཁ͕͋Δ

    # Networking Support > Networking options > ESP Transformation ͱ AH Transformation Λ༗ޮԽ $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig # ͜ͷ࣌఺ͰΧʔωϧ࠶Ϗϧυ͸׬ྃ $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5 # ࣮ࡍʹ͸͜͜ͰArmadilloͳΒͰ͸ͷΰχϣΰχϣ͕ඞཁׂ͕ͩѪ # ੒Ռ෺ͷ.swu͕ArmadilloͰॻ͖ࠐΊΔϑΝʔϜ΢ΣΞΠϝʔδ $ mkswu -o update-kernel.swu update-kernel/update-kernel.desc
  36. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ArmadilloͷԼ४උ 112 • Atmark Techno Development Environmentͱ͍͏ެࣜʹఏڙ͞ΕΔVMΛ࢖͑͹ɺ Armadillo༻ͷΧʔωϧϏϧυͳͲΛ؆୯ʹ΍Γ࢝ΊΒΕΔͷͰΦεεϝ ◦

    ࢀߟ: https://armadillo.atmark-techno.com/guide/atde • ͨͩͦ͠ͷ··Ͱ͸୯ҰϚγϯʹର͢ΔVMͱ͔ͯ͠͠ѻ͑ͳ͍ͷͰɺ
 ࠓճ͸GCEʹΠϯϙʔτͤͯ͞ڞ༗VMͷΑ͏ͳײ͡ʹͨ͠ ◦ ؔ࿈ϝϯόʔ͕୭Ͱ΋࢖͑ΔΑ͏ʹ # VMͷ.vmdkΛࣄલʹGCSʹΞοϓϩʔυ͓ͯ͘͠ # ͦͷvmdkΛΠϯϙʔτͤͯ͞GCEΠϯελϯεΛ࡞੒ $ gcloud compute images import [your-vm-name] --source-file "gs://path/to/atde.vmdk"
  37. ࠓճͷࣄྫʹ͓͚Δٕज़ཁૉͷ঺հ ArmadilloͷԼ४උ 113 • ࡞੒ͨ͠Πϝʔδ͸ωοτϫʔΫܦ༝ͰΠϯετʔϧͤ͞Δ͜ͱ΋Մೳ • ͞ΒʹΦϒδΣΫτετϨʔδͳͲΛซ༻ͯ͠؆୯ʹDFUͰ͖Δ࢓૊Έ΋੔͑Δͱྑ͠ # ࣗલͷͲ͔͜ʹಠࣗϏϧυͨ͠ΧελϜΠϝʔδΛ͓͍ͯɺ͔ͦ͜Βμ΢ϯϩʔυͯ͠ॻ͖ࠐΉ $

    swupdate -d '-u http://your.image-publisher.io/update-kernel.swu' # ެࣜతʹఏڙ͞Ε͍ͯΔ࠷৽ΠϝʔδΛॻ͖ࠐΉ $ swupdate -d '-u https://download.atmark-techno.com/armadillo-iot-g4/image/baseos-x2-latest.swu'
  38. ೗Կʹໝ૝͢Δ͔ ෺ཧతʹ৸Δ 124 • ਺೔ܦͬͯઃܭΛݟ௚͢ͱΞϥ͕໨ʹ෇͘ (Έͳ͞Μ΋ܦݧ͋Δͱࢥ͏͕) • ਆΤϯδχΞͳΒ·ͩ͠΋ɺࢲΛؚΊຌਓ͕1೔Ͱ࡞ͬͨઃܭ͸ΫιofΫι • ݪଇɺϩʔϚ͸1೔ͯ͠੒Βͳ͍

    • ͱʹ͔࣌ؒ͘Λஔ͍ͯɺ৽઱ͳ໨Ͱཁ݅΍ઃܭΛݟ௚͢͜ͱΛηΫγϣϯຖʹ܁Γฦ͢ ◦ Ϋϥ΢υͷxxͷ෦෼ / ΦϯϓϨͷNWͷ͜ͷ෦෼ / ͜͏͍͏ো֐ύλʔϯͷͱ͖ etc.
  39. ೗Կʹໝ૝͢Δ͔ ઃܭΛ৸͔ͤΔ 125 • ΄ͱΜͲ෺ཧతʹ৸Δ͜ͱͱಉٛͳͷ͕ͩ • ׬੒ͨ͠ʂͬͯࢥͬͨઃܭɺͪΐͬͱ଴ͯʂ 1ϲ݄৸͔ͤΑ͏ • ͦͷ1ϲ݄ޙʹ·ͨݟ௚͢ɺ୭͔ʹઆ໌ͯ͠ΈΔ

    (ॏཁ) ◦ ͋Εɾɾɾʁ ͜Εຊ౰ʹ͜ΕͰ͍͍Μ͚ͩͬɾɾɾʁ 
 ͜Μͳ͜ͱ/͋Μͳ͜ͱ΋༗Γಘͳ͍ɾɾɾʁ
 ͬͯͳΔ • ͜ΕͰख़੒͕ਐΜͰ͞Βʹѱ͍Օॴ͕ᖰΓग़͞Εͯશମతʹྑ͘ͳΔɻ
  40. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 133 • ϋΠϒϦουΫϥ΢υΛ࣮૷͢Δʹ౰ͨͬͯଞʹ΋બ୒ࢶ͸͋ͬͨ • ͔͠͠ɺίϯηϓτʹরΒ͠߹Θͤͯߟ͑Δͱཁ݅Λຬͨ͞ͳ͍͜ͱ͕ଟ͍ ◦ GCP Anthos Clusters

    on bare metal ◦ ઐ༻ઢ(InterConnect) ◦ ดҬ໢ ◦ ֤ࣾͷϚΠΫϩαʔόʔ ◦ Ϛϧνܦ࿏ͳઐ༻NWػث ◦ ※͜ΕΒ͸Ұ෦ͷྫͰɺ࣮ࡍʹ͸΋ͬͱͨ͘͞Μͷબ୒ࢶͱऔࣺબ୒͕͋Γ·ͨ͠
  41. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 134 • Anthos Clusters on bare metal ͷՄೳੑ ◦

    ΦϯϓϨଆͰϚωʔδυͳk8sΛಈ͔ͯ͠GCP͔Β؅ཧͰ͖Δ ◦ ΊͪΌͪ͘Όศརʂ ຊདྷͳΒ໘౗ͳΦϯϓϨଆͷk8sΛϚωʔδυʹѻ͑Δ • ෆ࠾༻ͷཧ༝ ◦ ཁٻεϖοΫ͕ߴ͘ɺͦΕΛຬͨͨ͢ΊͷػثΛങ͓͏ͱ͢Δͱίετ͕ਹΉ ◦ ΦϯϓϨଆͰΰϦΰϦʹk8sಈ͔͍ͨ͠΄ͲͷϫʔΫϩʔυ͸ͳ͍ ◦ ͦΜͳಷॏͳߏ੒ʹͨ͘͠ͳ͍
  42. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 135 • தن໛ʙେن໛ͳϋΠϒϦουߏ੒Λ΍ΔͳΒઐ༻ઢΛಋೖ͢΂͖ ◦ AWS: DirectConnect ◦ GCP: InterConnect

    ◦ Azure: ExpressRoute • ѹ౗తʹ଎ͯ҆͘ఆత • ͦ͏͍͏ن໛ͷέʔεͳΒࣗ෼Ͱ΋ͦ͏͢Δ͠ɺաڈͦ͏ͨ͠ ◦ ͕ɺࠓճͷ༻్ʹ͸߹Θͳ͍
  43. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 137 • ϞόΠϧճઢ + ดҬ໢Λ࢖͏έʔε ◦ ಛʹԿ΋ߏ੒ͤͣͱ΋SIMૠ͚ͩ͢ͰGCPͱϩʔΧϧ௨৴͕ՄೳʹͳΔ ◦ VPNͷ؅ཧ/ϝϯςφϯείετΛ௿ݮՄೳ

    ◦ ͓ͦΒ͘ɺࠓճͷΑ͏ͳଞڌ఺ɾෆಛఆଟ਺ͷϋΠϒϦουΫϥ΢υߏஙͰ΋ͳ͍ ݶΓ͸ɺਖ਼௚ดҬ໢Λ࢖ͬͨํָ͕ͩͱࢥ͏
  44. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 138 • ดҬ໢Λ࠾༻͠ͳ͔ͬͨཧ༝ ◦ ϥϯχϯάίετ͕ߴ͍ ◦ ϑϨΩγϏϦςΟ͕௿͍ ▪ ϕϯμʔ͕ઃఆ͍͔ͨͭ͘͠ͷΦϓγϣϯҎ֎ͷ͜ͱ͸Ͱ͖ͳ͍

    ◦ εέʔϥϏϦςΟ͕௿͍ ▪ Կ͔͠ΒτϥϑΟοΫతʹݶքΛܴ͑ͯ΋ͳ͔ͳ্͔͛ΒΕͳ͍ ◦ ΞδϦςΟ͕௿͍ ▪ ௐୡ·ͰͷϦʔυλΠϜ͕௕͘ൃ஫͔͚͔ͯΒ1-2ϲ݄͔͔ͬͯ͠·͏
  45. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 139 • ઐ༻ઢ͕ѱ͍Θ͚͡Όͳ͍ / ดҬ໢͕ѱ͍Θ͚͡Όͳ͍ ◦ ؒҧ͍ͳ͘ߴ଎Ͱ҆ఆతͩ͠ɺҰ౓ಋೖͯ͠͠·͑͹ޙ͸ָ ◦ ؅ཧ΍੹೚Λͪ͜ΒͰ࣋ͭඞཁ͕ͳ͍

    ◦ ͓ۚ͑͞ग़ͤ͹ɺ౦੢৑௕΋ͯ͘͠ΕΔͷͰՄ༻ੑ΋Ҿ্͖͛ΒΕΔ ◦ ͔͠͠ͳ͕ΒϞϊͷྑ͞ͱɺϏδωεͱরΒ͠߹Θ্ͤͨͰͷྑ͞͸ҟͳΔ ◦ ࠓճͷέʔεͩͱ߹க͠ͳ͔͚ͬͨͩ
  46. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 140 • ઐ༻NWػثͷݕ౼΋౰વͯ͠Έͨ ◦ CiscoͰ͍͑͹ C1111-8PLTELA ͱ͔ͳΒ༗ઢGbE΋LTE΋࢖͑Δ ◦ ֤ࣾগͳ͔ΒͣࣅͨΑ͏ͳͷ͕͋Δ

    • ࠷ॳʹݕ౼ͨ͠ͷ͕͜Εܥͩͬͨͷ͕ɺສ͕Ұͷ௵͕͠ޮ͔ͳ͍ ◦ ͦΓΌIPsecͷHWΦϑϩʔυޮ͔ͤΔΑ͏ͳεϧʔϓοτ΍ΒNATੑೳ΍Β͕
 ཉ͍͠ͳΒ͜͏͍͏ܥΛ࢖͏΂͖͕ͩࠓճ͸ͦ͜·ͰͷύϑΥʔϚϯε͸ෆཁ ◦ ͩͱ͢Ε͹ɺ൚༻ੑ͕ߴ͍ํΛબͼ͔ͨͬͨ = LinuxͷNWػثԽ • ௐୡϦʔυλΠϜ͕௕͘ͳΓ͕ͪ ◦ ্ड़ͷC1111-8PLTELAͩͱೲظ໿10ϲ݄ɻɻɻ
  47. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ 141 • ֤αʔόʔϕϯμʔ͔ΒϚΠΫϩαʔόʔͱ͍͏΋ͷ͕ϦϦʔε͞Ε͍ͯΔ ◦ 20ສʙ͙Β͍ͷ҆͞Ͱ྆खαΠζ͙Β͍ͷখ͞ͳαʔόʔ͕खʹೖΔ ◦ ίϯγϡʔϚʔ޲͚ͷIntel NUCΛͻͱ·ΘΓେ͖ͨ͘͠ײ͡ͷ΍ͭ •

    ͪ͜Βͷํ͕௵͕͠ޮ͘ ◦ ͕ɺࠓޙͷϏδωεల։Λߟ͑ͨ࣌ʹ࠷খߏ੒Ͱ͋ͬͯ΋ΦʔόʔεϖοΫ ◦ ઈରతʹ͸͍͚҆ͲɺզʑͷϏδωεΛߟ͑ͨ࣌ʹ͸૬ରతʹߴ͍ ◦ ͓٬༷΁ͷՁ֨తૌٻྗΛऑΊͯ͠·͏
  48. ཧ૝ͱݱ࣮ͷΪϟοϓΛຒΊΔ ఏڙϓϥϯͱ͓٬༷ࣗ਎ͷબ୒Ͱ͖Δࣗ༝Λ 144 • High Availabillity ϓϥϯ: ೥ؒ਺෼΋ࢭΊͨ͘ͳ͍͓٬༷޲͚ ◦ ׬શͳϚϧνϦʔδϣϯߏ੒ʹΑΔHigh

    Availabillityͳ؀ڥ • Regular ϓϥϯ: ೥ؒ਺े෼͙Β͍ͷμ΢ϯλΠϜͳΒڐ༰Ͱ͖Δ͓٬༷޲͚ ◦ ໓ଟʹϠό͍͜ͱ͸ى͖ͳ͍ΑͶ͍ͬͯ͏͜ͱͰϦʔδϣφϧߏ੒͕લఏ ͓٬༷ͷࣄ৘ʹԠ͍ͯ҆͡ϓϥϯͰ΋ߴ͍ϓϥϯͰ΋બ୒Ͱ͖ΔΑ͏ʹ
  49. ͓఻͍͑ͨ͜͠ͱ 145 • ϋΠϒϦουΫϥ΢υʹ͸ໝ૝ྗ͕ඞཁෆՄܽ • ৸Α͏ / ৸͔ͤΑ͏ • ࠷ڧ͸࠷ળͰ͸ͳ͍

    ಄ͷย۱ʹͰ΋ஔ͍͓͍͍͚ͯͯͨͩΔͱɺݱ࣮తͳϋΠϒϦουΫϥ΢υઃܭ͕Ͱ͖Δɾɾɾ͔΋ɻ