Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINEをdual stack環境に変更した話

LINEをdual stack環境に変更した話

JANOG50での登壇資料です

Avatar for LINE Developers

LINE Developers

July 14, 2022
Tweet

More Decks by LINE Developers

Other Decks in Technology

Transcript

  1. n໊લ தߔ ࢲՎ(ͳ͔Έͧ ͻ͔ͦ) nॴଐ LINEגࣜձࣾ ITSC ωοτϫʔΫࣨ αʔϏεωοτϫʔΫ2νʔϜ n࢓ࣄ

    2019೥ΑΓLINEʹೖࣾɺLINEͷIPv6ରԠͷPM΍Fintech શൠʹؔ࿈͢ΔωοτϫʔΫҊ݅Λ୲౰த ࣗݾ঺հ
  2. ϓϩδΣΫτͷλʔήοτ2 nLINEͷϑϩϯτΤϯυΛIPv4/IPv6 Dual StackԽ͢Δ͜ͱͰIPv6ରԠ͠ɺ όοΫΤϯυ͸Ҿ͖ଓ͖IPv4ͷΈͰ௨৴͢Δߏ੒Λ࠾༻ Collaboration Services LINE Client Software

    LB LEGY LINE event delivery gateway Talk Server IPv4 IPv4 Storage OBS Object storage IPv4/IPv6 VoIP Voice over IP IPv4 IPv4/IPv6 IPv4/ IPv6 IPv4/ IPv6 n ֤ίϯϙʔωϯτͷ໾ׂ l LEGY(LINE Event delivery GatewaY): LINE Client͕֤छ ίϯϙʔωϯτͱ઀ଓ͢ΔͨΊʹඞཁͳήʔτ΢ΣΠɺϦόʔεϓϩΩγ l OBS(OBject Storage): ը૾ɺಈըͳͲͷσʔλอ؅ઌ l VoIP(Voice over IP): LINEͰͷԻ੠௨৴ɺϏσΦ௨৴
  3. Dual Stack؀ڥ΁ͷҠߦ·ͰͷಓͷΓ ཁ݅ఆٛɺ ઃܭɺݕূ Transit/IX޲͚ͷ઀ଓΛDual StackԽ Backbone NWɺ ։ൃ؀ڥ޲͚NWɺ Verda(։ൃ؀ڥ)Λ

    Dual StackԽ ։ൃ؀ڥͰ ΞϓϦଆͷ ಈ࡞νΣοΫ ঎༻؀ڥ޲͚NWɺ Verda(঎༻؀ڥ)ΛDual StackԽ ΞϓϦΛ Dual StackԽ 1 2 3 4 5 7 nԼهͷΑ͏ͳϓϩηεΛܦͯɺҠߦ ӡ༻πʔϧͳͲͷIPv6ରԠ 6
  4. ϓϩδΣΫτͷي੻ 2019 2020 2021 2022 10݄ Kickoff 11݄ ݕ౼։࢝ 2݄

    IPv6ΞυϨεମܥܾఆ 6݄ ։ൃ؀ڥ޲͚Dual Stack NWఏڙ׬ྃ 12݄ ։ൃ؀ڥͰͷΞϓϦ ςετ׬ྃ 10݄ ঎༻؀ڥ޲͚Dual Stack NWఏڙ׬ྃ 11݄ OBS JPରԠ׬ྃ 12݄ Transit/IX޲͚ͷ Dual StackԽରԠ׬ྃ 5݄-?݄ VoIP JPରԠ׬ྃ༧ఆ 6݄ OBS TWରԠ׬ྃ 7݄ LEGY JPରԠ׬ྃ 7݄-8݄ LEGY TWରԠ׬ྃ༧ఆ 9݄-12݄ VoIP TWରԠ׬ྃ༧ఆ n2022೥౓ͷ׬ྃΛ໨ࢦͯ͠ϓϩδΣΫτਪਐதɺVoIPͰҰ෦૝ఆ֎ ͷಈ࡞Λ͍ͯ͠Δ͕ɺҠߦʹ൐͏େ͖ͳτϥϒϧ͸ͳ͠
  5. ՝୊1 ϓϩδΣΫτείʔϓ͕ᐆດ n ૊৫Λލ͍ͩෳ਺ͷϝϯόʔ͕ࢀը l NWࣨɿ঎༻ɺΦϑΟεωοτϫʔΫ୲౰ l γεςϜࣨɿαʔό୲౰ l Verdaࣨɿࣾ಺Private

    CloudɺSoftware LB୲౰ l Infra Protection 2νʔϜɿωοτϫʔΫؔ࿈ͷηΩϡϦςΟ୲౰ l Reliability Engineeringɿ֤ίϯϙʔωϯτͷ։ൃऀͱͷௐ੔ɺCDN୲౰
  6. Backbone Network IPv4 Internet Πϯλʔωοτ ઀ଓϧʔλ IPv4 ڥքεΠον IPv4 IPv4

    αʔό1 IP Clos ঎༻ SW SW ToR ToR αʔό2 w/Public IPv4 wo/Public IPv4 NATϧʔλ w/Public IPv4 Verda Internet Gateway w/Public IPv4 ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4ͷϙϦγʔ αʔϏε༻్Ͱͳ͍αʔό͸ Public IPv4ΞυϨεΛ࣋ͨͳ͍ ͷͰɺΠϯλʔωοτʹ઀ଓ͢ Δ৔߹͸NATϧʔλ΍Verdaͷ Internet Gatewayܦ༝ͱͳΔ αʔϏε༻్ͷαʔό͸ Public IPv4ΞυϨεΛ࣋ͭ
  7. ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4ͷϙϦγʔ Backbone Network IPv4 Internet Πϯλʔωοτ ઀ଓϧʔλ IPv4

    ڥքεΠον IPv4 IPv4 αʔό1 IP Clos ঎༻ SW SW ToR ToR αʔό2 w/Public IPv4 wo/Public IPv4 NATϧʔλ w/Public IPv4 Verda Internet Gateway w/Public IPv4 ͜ͷϙϦγʔΛมߋ͢Δ৔߹ɺωοτϫʔΫػثҎ֎ ʹ΋ඇৗʹଟ͘ͷηΩϡϦςΟػث΍αʔόɺ Verdaͷมߋ͕ඞཁͱͳΔɻɻɻ
  8. ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4 ͱ IPv6 ΞυϨεͷൺֱ ߲໨ IPv4 IPv6 IP؅ཧπʔϧ

    ˓ ˓ ϏοτʹؚΜͰ͍ Δ৘ใ ಛʹͳ͠ l ಺෦༻͔֎෦༻͔ l ωοτϫʔΫػثʹ࢖ΘΕΔ΋ ͷ͔ɺαʔόʹ࢖ΘΕΔ΋ͷ͔ɺ ΦϑΟεωοτϫʔΫʹ࢖ΘΕ Δ΋ͷ͔(෼ذ৚݅) l ͲͷDCͰ࢖ΘΕ͍ͯΔ͔ l ͲͷαʔόϧʔϜͰ࢖ΘΕΔ͔ l Ͳͷू໿ػثʹ࢖ΘΕΔ͔ l Ͳ͏͍͏໨తͰ࢖ΘΕΔΞυϨ ε͔
  9. ՝୊5 Dual Stack環境をどう安定運用させるか n՝୊ ঎༻؀ڥ͸ෳ਺ϕϯμʔͷωοτϫʔΫػثͰӡ༻͞Ε͓ͯΓɺ҆ఆӡ༻͢Δ ඞཁ͕͋ͬͨ →େن໛ʹIPv6ରԠΛߦ͏ඞཁ͕͋ͬͨͷͰɺ҆ఆͨ͠ӡ༻΋ηοτͰ ݕ౼͕ඞཁ nରԠํ๏ l2019೥࣌఺ͰIPv6ʹؔ࿈͢ΔBugௐࠪΛ࣮ࢪ͠ɺλʔήοτͷVersionΛ

    બఆɺBackbone NWͷػثʹ͍ͭͯ͸2019〜2020೥ʹ͔͚ͯେن໛ͳ Version UpΛ࣮ࢪ l৽͘͠ग़དྷͨαʔόϧʔϜ͸Multi AZؔ࿈ͷ΋ͷ͸࠷ॳ͔ΒDual StackԽ ग़དྷͨঢ়ଶͰߏங lӡ༻தʹൃੜͨ͠CriticalͳBug͸ൺֱతૣظʹରԠ lIP Closʹؔͯ͠͸ෳ਺ϕϯμʔɺෳ਺VersionΛར༻͍ͯ͠Δ͕ ౷Ұ͍ͯ͜͠͏ͱ͍ͯ͠Δ(ग़དྷΔ͚ͩ҆ఆͨ͠VersionΛ࠾༻)
  10. ՝୊6 いかに影響を少なく移行できるか -*/& "4 5SBOTJU Πϯλʔωοτ ઀ଓϧʔλ *1W*1W 5SBOTJU 5SBOTJU

    5SBOTJU ੾ସର৅ "4 n୆࿷ͰͷҠߦલʹΠϯλʔωοτ઀ଓϧʔλͰ্ख͘ECMPग़དྷͯ ͍ͳ͍Bug͕͋Γɺ੾ସର৅ͱͷASͷؒͰ᫔᫓͕ൃੜ͢ΔՄೳੑ ͕͋ͬͨͨΊΠϯλʔωοτ઀ଓϧʔλͷVersion UpΛ࣮ࢪͨ͠ ੾ସର৅"4ͱ͸ෳ਺ͷ5SBOTJUܦ༝ Ͱ઀ଓ͍ͯͯ͠ɺ্ख͘&$.1Ͱ͖ Ε͹໰୊ͳ͘੾Γସ͑ΒΕΔ༧ఆ ͕ͩͬͨɺग़དྷͳͦ͞͏ͳ͜ͱ͕ࣄ લʹΘ͔ͬͨ ˠΠϯλʔωοτ ઀ଓϧʔλͷ7FSTJPO61Λ࣮ࢪ
  11. ՝୊6 いかに影響を少なく移行できるか nOBS΍LEGYͰ͸ԼهͷΑ͏ͳ෼ذΛ࡞੒͠ɺAS୯ҐͰ੾Γସ͑Λ ࣮ࢪ OBS/LEGY΁ͷ ໊લղܾ͕੾Γସ͍͑ͨ AS͔Βͷ΋ͷ͔ DNSΫΤϦʹ AAAAϨίʔυΛ Ԡ౴͢Δ͔

    IPv6 VIP IPv4 VIP IPv4 VIP Yes No Yes No AAAAϨίʔυΛ Ԡ౴͢Δ͔ɺAϨίʔ υΛԠ౴͢Δ͔͸ׂ߹ ΛมߋՄೳ OBSͷ࠷ॳͷ੾ସͷࡍʹݹ͍ ෼ذઃఆ͕࢒͍ͬͯͯҰ࣌తʹ શͯͷ௨৴͕IPv6Խ͞ΕΔӋ໨ ʹɻɻɻ →େ͖ͳ໰୊͸ͳ͠
  12. ՝୊6 いかに影響を少なく移行できるか nVoIP͸ಠࣗͷ࣮૷ʹΑΓҠߦΛ࣮ࢪ ి࿩Λֻ͚Δਓ IPv4/IPv6 ి࿩Λड͚Δਓ IPv4/IPv6 IPv6 ి࿩Λֻ͚Δਓ IPv4/IPv6

    ి࿩Λड͚Δਓ IPv4 IPv4 ి࿩Λֻ͚Δਓ IPv4 ి࿩Λड͚Δਓ IPv4/IPv6 IPv4 ి࿩Λֻ͚Δਓ IPv4 ి࿩Λड͚Δਓ IPv4 IPv4 Ұఆͷׂ߹Ͱ͜ͷ෦෼Λ૿΍͍ͯ͘͠
  13. ՝୊6 いかに影響を少なく移行できるか nVoIPͰ௨ৗ͸P2PͰߦΘΕΔ͸ͣͷ௨৴(1ର1ͷ΋ͷ)͕αʔόܦ༝ ͱͳΓΠϯλʔωοτͱͷτϥϑΟοΫ͕૿ՃɺݪҼௐࠪத ి࿩Λֻ͚Δਓ ి࿩Λड͚Δਓ IPv6 VoIP サーバ ి࿩Λֻ͚Δਓ

    ి࿩Λड͚Δਓ IPv6 VoIP サーバ 1ର1ͷ௨৴΋IPv6ͷ৔ ߹αʔόܦ༝ͱͳͬͯ ͠·͍ɺΠϯλʔωο τͱͷ௨৴͕૿Ճɻɻɻ ݱࡏͷঢ়گ ૝ఆ 1ର1ͷ௨৴͸૝ఆͰ͸ P2PͰߦΘΕΔ(IPv4 Ͱ͸ͦΕͳΓͷׂ߹ ͰP2PͰ௨৴Ͱ͖͍ͯ Δ)
  14. ൃੜͨ͠՝୊ͱରԠํ๏·ͱΊ No ՝୊ ରԠํ๏ 1 ϓϩδΣΫτείʔϓ͕ᐆດ l εςʔΫϗϧμʔΛఆٛ͠ɺؔ܎ऀΛר͖ࠐΉ l Կ͕ग़དྷͯɺԿ͕ग़དྷͳ͍͔࿩͠߹͏

    l ໌จԽ͢Δ 2 ϓϩδΣΫτͷਐ௙͕஗͍ l ՝୊؅ཧΛͯ͠ɺఆظతʹ֬ೝ͢Δ l PMࣗ਎΋ग़དྷΔ͜ͱΛ͢Δ 3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ l طଘͷϙϦγʔΛग़དྷΔݶΓ౿ऻ͢Δ l IPv6ΞυϨεʹؚΊ͍ͨ಺༰Λߟ͑Δ l কདྷ༧૝Ͱ͖ͳ͍෦෼͸Ͱ͖Δ͚ͩޙ͔ΒܾΊΒΕΔΑ͏ͳ ΞυϨεମܥʹ͢Δ 4 ΦϑΟεΛͲ͏IPv6ରԠ͢Δ͔ l ຊ౰ʹIPv6ͰͷΞΫηε͕ඞཁͳਓΛఆٛ͢Δ l ࣗࣾͷಛ௃(ΦϑΟεͷ૿ݮ͕සൟ͔ etc)Λ೺Ѳ͠ɺ࠷దͳ ࣮૷ํ๏Λ࠾༻͢Δ 5 Dual Stack؀ڥΛͲ͏҆ఆӡ༻ͤ͞ Δ͔ l BugௐࠪΛపఈతʹߦ͏ l େن໛ͳOS Version UpΛߦ͏ l ೔ৗͷதͰग़͖ͯͨBug΋Criticalͳ΋ͷ͸ૣΊʹରॲ 6 ͍͔ʹӨڹΛগͳ͘ҠߦͰ͖Δ͔ l ࣄલʹҠߦର৅ͷτϥϑΟοΫΛ༧ଌ l ग़དྷΔݶΓ্ͷϨΠϠʔͰมߋͰ͖Δํ๏Ͱɺ͔ͭখن໛ͳ ୯ҐͰมߋͰ͖Δํ๏Λ࠾༻͢Δ 7 ରԠ࿙ΕɺҠߦ࿙Ε͸ͳ͍͔ l IPv6ରԠ׬ྃ·Ͱʹ΍Βͳ͍ͱ͍͚ͳ͍͜ͱΛϦετԽ͢Δ (πʔϧʹ͸ಛʹ஫ҙ) l ఆظతʹ֬ೝ͢Δ
  15. ࠓޙൃੜͦ͠͏ͳ໰୊2 IPv4ΞυϨεͷރׇ n໰୊ LINEͰ͸૬౰ͳεϐʔυͰαʔό͕૿Ճ͓ͯ͠ΓɺϓϥΠϕʔτIPv4ɺ ύϒϦοΫIPv4ΞυϨεͷফඅ͕ܹ͘͠ɺIPv4ΞυϨε͕ރׇ͢Δʁ nରॲํ਑ l 2020೥〜2021೥ʹύϒϦοΫIPv4ΞυϨεΛ௥Ճߪೖ l IP

    Clos͚ͩͰͳ͘ɺରԠ͍ͯ͠Δػث͸όοΫϘʔϯଆͰ΋ϦϯΫؒͷ IPv4ΞυϨεΛՄೳͳݶΓফඅ͠ͳ͍ߏ੒(RFC5549)ʹมߋத l ISP SharedΞυϨεΛ࢖͑ΔΑ͏ʹγεςϜଆͰରԠ༧ఆ l IPv6ͷΈͰαʔόϧʔϜΛߏங͢Δ༧ఆ·Ͱ͸ݱঢ়ͳ͠ɺݕ౼΋·ͩ
  16. ࠓޙൃੜͦ͠͏ͳ໰୊ͱରॲํ਑·ͱΊ No ໰୊ ରॲํ਑ 1 TCAMͷރׇ l ܦ࿏਺ΛSNMPͰ؂ࢹ l ڥքεΠον͸TCAM͕े෼ͳ΋ͷʹϦϓϨʔε

    2 IPv4ΞυϨεͷރׇ l ύϒϦοΫIPv4ΞυϨεΛ௥Ճߪೖ l RFC5549ͷ༗ޮ׆༻ l ISP SharedΞυϨεΛར༻Ͱ͖ΔΑ͏γεςϜͷ४උ l IPv6ΦϯϦʔͷωοτϫʔΫͷݕ౼͸·ͩʂ
  17. ࢀߟࢿྉ l IPv6ΞυϨεͷ൪߸ܭըɺͲ͏ͯ͠·͔͢ɻ〜 ͋ΕͬɺΈΜͳͲ͏ͯ͠ΜͩΖ͏ 〜 https://www.janog.gr.jp/meeting/janog42/program/V6PL l LINEͷωοτϫʔΫΛθϩ͔Β࠶ઃܭͨ͠࿩ https://www.janog.gr.jp/meeting/janog43/program/line/ l

    ίϯςϯπͷIPv6ԽͷͨΊʹ͸Կ͕଍Γͳ͍ͷ͔ʁ https://www.janog.gr.jp/meeting/janog47/ipv6/ l LINEͷωοτϫʔΫΦʔέετϨʔλΛϦχϡʔΞϧͨ͠࿩ https://www.janog.gr.jp/meeting/janog49/lineorc/ l “Change” our private cloud infrastructures from single-AZ to multi-AZs https://www.janog.gr.jp/meeting/janog50/multiaz/