Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINEをdual stack環境に変更した話

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →

LINEをdual stack環境に変更した話

JANOG50での登壇資料です

LINE Developers

July 14, 2022

Transcript

49 slides
  1. n໊લ தߔ ࢲՎ(ͳ͔Έͧ ͻ͔ͦ) nॴଐ LINEגࣜձࣾ ITSC ωοτϫʔΫࣨ αʔϏεωοτϫʔΫ2νʔϜ n࢓ࣄ

    2019೥ΑΓLINEʹೖࣾɺLINEͷIPv6ରԠͷPM΍Fintech શൠʹؔ࿈͢ΔωοτϫʔΫҊ݅Λ୲౰த ࣗݾ঺հ
  2. ϓϩδΣΫτͷλʔήοτ2 nLINEͷϑϩϯτΤϯυΛIPv4/IPv6 Dual StackԽ͢Δ͜ͱͰIPv6ରԠ͠ɺ όοΫΤϯυ͸Ҿ͖ଓ͖IPv4ͷΈͰ௨৴͢Δߏ੒Λ࠾༻ Collaboration Services LINE Client Software

    LB LEGY LINE event delivery gateway Talk Server IPv4 IPv4 Storage OBS Object storage IPv4/IPv6 VoIP Voice over IP IPv4 IPv4/IPv6 IPv4/ IPv6 IPv4/ IPv6 n ֤ίϯϙʔωϯτͷ໾ׂ l LEGY(LINE Event delivery GatewaY): LINE Client͕֤छ ίϯϙʔωϯτͱ઀ଓ͢ΔͨΊʹඞཁͳήʔτ΢ΣΠɺϦόʔεϓϩΩγ l OBS(OBject Storage): ը૾ɺಈըͳͲͷσʔλอ؅ઌ l VoIP(Voice over IP): LINEͰͷԻ੠௨৴ɺϏσΦ௨৴
  3. Dual Stack؀ڥ΁ͷҠߦ·ͰͷಓͷΓ ཁ݅ఆٛɺ ઃܭɺݕূ Transit/IX޲͚ͷ઀ଓΛDual StackԽ Backbone NWɺ ։ൃ؀ڥ޲͚NWɺ Verda(։ൃ؀ڥ)Λ

    Dual StackԽ ։ൃ؀ڥͰ ΞϓϦଆͷ ಈ࡞νΣοΫ ঎༻؀ڥ޲͚NWɺ Verda(঎༻؀ڥ)ΛDual StackԽ ΞϓϦΛ Dual StackԽ 1 2 3 4 5 7 nԼهͷΑ͏ͳϓϩηεΛܦͯɺҠߦ ӡ༻πʔϧͳͲͷIPv6ରԠ 6
  4. ϓϩδΣΫτͷي੻ 2019 2020 2021 2022 10݄ Kickoff 11݄ ݕ౼։࢝ 2݄

    IPv6ΞυϨεମܥܾఆ 6݄ ։ൃ؀ڥ޲͚Dual Stack NWఏڙ׬ྃ 12݄ ։ൃ؀ڥͰͷΞϓϦ ςετ׬ྃ 10݄ ঎༻؀ڥ޲͚Dual Stack NWఏڙ׬ྃ 11݄ OBS JPରԠ׬ྃ 12݄ Transit/IX޲͚ͷ Dual StackԽରԠ׬ྃ 5݄-?݄ VoIP JPରԠ׬ྃ༧ఆ 6݄ OBS TWରԠ׬ྃ 7݄ LEGY JPରԠ׬ྃ 7݄-8݄ LEGY TWରԠ׬ྃ༧ఆ 9݄-12݄ VoIP TWରԠ׬ྃ༧ఆ n2022೥౓ͷ׬ྃΛ໨ࢦͯ͠ϓϩδΣΫτਪਐதɺVoIPͰҰ෦૝ఆ֎ ͷಈ࡞Λ͍ͯ͠Δ͕ɺҠߦʹ൐͏େ͖ͳτϥϒϧ͸ͳ͠
  5. ՝୊1 ϓϩδΣΫτείʔϓ͕ᐆດ n ૊৫Λލ͍ͩෳ਺ͷϝϯόʔ͕ࢀը l NWࣨɿ঎༻ɺΦϑΟεωοτϫʔΫ୲౰ l γεςϜࣨɿαʔό୲౰ l Verdaࣨɿࣾ಺Private

    CloudɺSoftware LB୲౰ l Infra Protection 2νʔϜɿωοτϫʔΫؔ࿈ͷηΩϡϦςΟ୲౰ l Reliability Engineeringɿ֤ίϯϙʔωϯτͷ։ൃऀͱͷௐ੔ɺCDN୲౰
  6. Backbone Network IPv4 Internet Πϯλʔωοτ ઀ଓϧʔλ IPv4 ڥքεΠον IPv4 IPv4

    αʔό1 IP Clos ঎༻ SW SW ToR ToR αʔό2 w/Public IPv4 wo/Public IPv4 NATϧʔλ w/Public IPv4 Verda Internet Gateway w/Public IPv4 ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4ͷϙϦγʔ αʔϏε༻్Ͱͳ͍αʔό͸ Public IPv4ΞυϨεΛ࣋ͨͳ͍ ͷͰɺΠϯλʔωοτʹ઀ଓ͢ Δ৔߹͸NATϧʔλ΍Verdaͷ Internet Gatewayܦ༝ͱͳΔ αʔϏε༻్ͷαʔό͸ Public IPv4ΞυϨεΛ࣋ͭ
  7. ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4ͷϙϦγʔ Backbone Network IPv4 Internet Πϯλʔωοτ ઀ଓϧʔλ IPv4

    ڥքεΠον IPv4 IPv4 αʔό1 IP Clos ঎༻ SW SW ToR ToR αʔό2 w/Public IPv4 wo/Public IPv4 NATϧʔλ w/Public IPv4 Verda Internet Gateway w/Public IPv4 ͜ͷϙϦγʔΛมߋ͢Δ৔߹ɺωοτϫʔΫػثҎ֎ ʹ΋ඇৗʹଟ͘ͷηΩϡϦςΟػث΍αʔόɺ Verdaͷมߋ͕ඞཁͱͳΔɻɻɻ
  8. ՝୊3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ nIPv4 ͱ IPv6 ΞυϨεͷൺֱ ߲໨ IPv4 IPv6 IP؅ཧπʔϧ

    ˓ ˓ ϏοτʹؚΜͰ͍ Δ৘ใ ಛʹͳ͠ l ಺෦༻͔֎෦༻͔ l ωοτϫʔΫػثʹ࢖ΘΕΔ΋ ͷ͔ɺαʔόʹ࢖ΘΕΔ΋ͷ͔ɺ ΦϑΟεωοτϫʔΫʹ࢖ΘΕ Δ΋ͷ͔(෼ذ৚݅) l ͲͷDCͰ࢖ΘΕ͍ͯΔ͔ l ͲͷαʔόϧʔϜͰ࢖ΘΕΔ͔ l Ͳͷू໿ػثʹ࢖ΘΕΔ͔ l Ͳ͏͍͏໨తͰ࢖ΘΕΔΞυϨ ε͔
  9. ՝୊5 Dual Stack環境をどう安定運用させるか n՝୊ ঎༻؀ڥ͸ෳ਺ϕϯμʔͷωοτϫʔΫػثͰӡ༻͞Ε͓ͯΓɺ҆ఆӡ༻͢Δ ඞཁ͕͋ͬͨ →େن໛ʹIPv6ରԠΛߦ͏ඞཁ͕͋ͬͨͷͰɺ҆ఆͨ͠ӡ༻΋ηοτͰ ݕ౼͕ඞཁ nରԠํ๏ l2019೥࣌఺ͰIPv6ʹؔ࿈͢ΔBugௐࠪΛ࣮ࢪ͠ɺλʔήοτͷVersionΛ

    બఆɺBackbone NWͷػثʹ͍ͭͯ͸2019〜2020೥ʹ͔͚ͯେن໛ͳ Version UpΛ࣮ࢪ l৽͘͠ग़དྷͨαʔόϧʔϜ͸Multi AZؔ࿈ͷ΋ͷ͸࠷ॳ͔ΒDual StackԽ ग़དྷͨঢ়ଶͰߏங lӡ༻தʹൃੜͨ͠CriticalͳBug͸ൺֱతૣظʹରԠ lIP Closʹؔͯ͠͸ෳ਺ϕϯμʔɺෳ਺VersionΛར༻͍ͯ͠Δ͕ ౷Ұ͍ͯ͜͠͏ͱ͍ͯ͠Δ(ग़དྷΔ͚ͩ҆ఆͨ͠VersionΛ࠾༻)
  10. ՝୊6 いかに影響を少なく移行できるか -*/& "4 5SBOTJU Πϯλʔωοτ ઀ଓϧʔλ *1W*1W 5SBOTJU 5SBOTJU

    5SBOTJU ੾ସର৅ "4 n୆࿷ͰͷҠߦલʹΠϯλʔωοτ઀ଓϧʔλͰ্ख͘ECMPग़དྷͯ ͍ͳ͍Bug͕͋Γɺ੾ସର৅ͱͷASͷؒͰ᫔᫓͕ൃੜ͢ΔՄೳੑ ͕͋ͬͨͨΊΠϯλʔωοτ઀ଓϧʔλͷVersion UpΛ࣮ࢪͨ͠ ੾ସର৅"4ͱ͸ෳ਺ͷ5SBOTJUܦ༝ Ͱ઀ଓ͍ͯͯ͠ɺ্ख͘&$.1Ͱ͖ Ε͹໰୊ͳ͘੾Γସ͑ΒΕΔ༧ఆ ͕ͩͬͨɺग़དྷͳͦ͞͏ͳ͜ͱ͕ࣄ લʹΘ͔ͬͨ ˠΠϯλʔωοτ ઀ଓϧʔλͷ7FSTJPO61Λ࣮ࢪ
  11. ՝୊6 いかに影響を少なく移行できるか nOBS΍LEGYͰ͸ԼهͷΑ͏ͳ෼ذΛ࡞੒͠ɺAS୯ҐͰ੾Γସ͑Λ ࣮ࢪ OBS/LEGY΁ͷ ໊લղܾ͕੾Γସ͍͑ͨ AS͔Βͷ΋ͷ͔ DNSΫΤϦʹ AAAAϨίʔυΛ Ԡ౴͢Δ͔

    IPv6 VIP IPv4 VIP IPv4 VIP Yes No Yes No AAAAϨίʔυΛ Ԡ౴͢Δ͔ɺAϨίʔ υΛԠ౴͢Δ͔͸ׂ߹ ΛมߋՄೳ OBSͷ࠷ॳͷ੾ସͷࡍʹݹ͍ ෼ذઃఆ͕࢒͍ͬͯͯҰ࣌తʹ શͯͷ௨৴͕IPv6Խ͞ΕΔӋ໨ ʹɻɻɻ →େ͖ͳ໰୊͸ͳ͠
  12. ՝୊6 いかに影響を少なく移行できるか nVoIP͸ಠࣗͷ࣮૷ʹΑΓҠߦΛ࣮ࢪ ి࿩Λֻ͚Δਓ IPv4/IPv6 ి࿩Λड͚Δਓ IPv4/IPv6 IPv6 ి࿩Λֻ͚Δਓ IPv4/IPv6

    ి࿩Λड͚Δਓ IPv4 IPv4 ి࿩Λֻ͚Δਓ IPv4 ి࿩Λड͚Δਓ IPv4/IPv6 IPv4 ి࿩Λֻ͚Δਓ IPv4 ి࿩Λड͚Δਓ IPv4 IPv4 Ұఆͷׂ߹Ͱ͜ͷ෦෼Λ૿΍͍ͯ͘͠
  13. ՝୊6 いかに影響を少なく移行できるか nVoIPͰ௨ৗ͸P2PͰߦΘΕΔ͸ͣͷ௨৴(1ର1ͷ΋ͷ)͕αʔόܦ༝ ͱͳΓΠϯλʔωοτͱͷτϥϑΟοΫ͕૿ՃɺݪҼௐࠪத ి࿩Λֻ͚Δਓ ి࿩Λड͚Δਓ IPv6 VoIP サーバ ి࿩Λֻ͚Δਓ

    ి࿩Λड͚Δਓ IPv6 VoIP サーバ 1ର1ͷ௨৴΋IPv6ͷ৔ ߹αʔόܦ༝ͱͳͬͯ ͠·͍ɺΠϯλʔωο τͱͷ௨৴͕૿Ճɻɻɻ ݱࡏͷঢ়گ ૝ఆ 1ର1ͷ௨৴͸૝ఆͰ͸ P2PͰߦΘΕΔ(IPv4 Ͱ͸ͦΕͳΓͷׂ߹ ͰP2PͰ௨৴Ͱ͖͍ͯ Δ)
  14. ൃੜͨ͠՝୊ͱରԠํ๏·ͱΊ No ՝୊ ରԠํ๏ 1 ϓϩδΣΫτείʔϓ͕ᐆດ l εςʔΫϗϧμʔΛఆٛ͠ɺؔ܎ऀΛר͖ࠐΉ l Կ͕ग़དྷͯɺԿ͕ग़དྷͳ͍͔࿩͠߹͏

    l ໌จԽ͢Δ 2 ϓϩδΣΫτͷਐ௙͕஗͍ l ՝୊؅ཧΛͯ͠ɺఆظతʹ֬ೝ͢Δ l PMࣗ਎΋ग़དྷΔ͜ͱΛ͢Δ 3 IPv6ΞυϨεମܥ͕ܾ·Βͳ͍ l طଘͷϙϦγʔΛग़དྷΔݶΓ౿ऻ͢Δ l IPv6ΞυϨεʹؚΊ͍ͨ಺༰Λߟ͑Δ l কདྷ༧૝Ͱ͖ͳ͍෦෼͸Ͱ͖Δ͚ͩޙ͔ΒܾΊΒΕΔΑ͏ͳ ΞυϨεମܥʹ͢Δ 4 ΦϑΟεΛͲ͏IPv6ରԠ͢Δ͔ l ຊ౰ʹIPv6ͰͷΞΫηε͕ඞཁͳਓΛఆٛ͢Δ l ࣗࣾͷಛ௃(ΦϑΟεͷ૿ݮ͕සൟ͔ etc)Λ೺Ѳ͠ɺ࠷దͳ ࣮૷ํ๏Λ࠾༻͢Δ 5 Dual Stack؀ڥΛͲ͏҆ఆӡ༻ͤ͞ Δ͔ l BugௐࠪΛపఈతʹߦ͏ l େن໛ͳOS Version UpΛߦ͏ l ೔ৗͷதͰग़͖ͯͨBug΋Criticalͳ΋ͷ͸ૣΊʹରॲ 6 ͍͔ʹӨڹΛগͳ͘ҠߦͰ͖Δ͔ l ࣄલʹҠߦର৅ͷτϥϑΟοΫΛ༧ଌ l ग़དྷΔݶΓ্ͷϨΠϠʔͰมߋͰ͖Δํ๏Ͱɺ͔ͭখن໛ͳ ୯ҐͰมߋͰ͖Δํ๏Λ࠾༻͢Δ 7 ରԠ࿙ΕɺҠߦ࿙Ε͸ͳ͍͔ l IPv6ରԠ׬ྃ·Ͱʹ΍Βͳ͍ͱ͍͚ͳ͍͜ͱΛϦετԽ͢Δ (πʔϧʹ͸ಛʹ஫ҙ) l ఆظతʹ֬ೝ͢Δ
  15. ࠓޙൃੜͦ͠͏ͳ໰୊2 IPv4ΞυϨεͷރׇ n໰୊ LINEͰ͸૬౰ͳεϐʔυͰαʔό͕૿Ճ͓ͯ͠ΓɺϓϥΠϕʔτIPv4ɺ ύϒϦοΫIPv4ΞυϨεͷফඅ͕ܹ͘͠ɺIPv4ΞυϨε͕ރׇ͢Δʁ nରॲํ਑ l 2020೥〜2021೥ʹύϒϦοΫIPv4ΞυϨεΛ௥Ճߪೖ l IP

    Clos͚ͩͰͳ͘ɺରԠ͍ͯ͠Δػث͸όοΫϘʔϯଆͰ΋ϦϯΫؒͷ IPv4ΞυϨεΛՄೳͳݶΓফඅ͠ͳ͍ߏ੒(RFC5549)ʹมߋத l ISP SharedΞυϨεΛ࢖͑ΔΑ͏ʹγεςϜଆͰରԠ༧ఆ l IPv6ͷΈͰαʔόϧʔϜΛߏங͢Δ༧ఆ·Ͱ͸ݱঢ়ͳ͠ɺݕ౼΋·ͩ
  16. ࠓޙൃੜͦ͠͏ͳ໰୊ͱରॲํ਑·ͱΊ No ໰୊ ରॲํ਑ 1 TCAMͷރׇ l ܦ࿏਺ΛSNMPͰ؂ࢹ l ڥքεΠον͸TCAM͕े෼ͳ΋ͷʹϦϓϨʔε

    2 IPv4ΞυϨεͷރׇ l ύϒϦοΫIPv4ΞυϨεΛ௥Ճߪೖ l RFC5549ͷ༗ޮ׆༻ l ISP SharedΞυϨεΛར༻Ͱ͖ΔΑ͏γεςϜͷ४උ l IPv6ΦϯϦʔͷωοτϫʔΫͷݕ౼͸·ͩʂ
  17. ࢀߟࢿྉ l IPv6ΞυϨεͷ൪߸ܭըɺͲ͏ͯ͠·͔͢ɻ〜 ͋ΕͬɺΈΜͳͲ͏ͯ͠ΜͩΖ͏ 〜 https://www.janog.gr.jp/meeting/janog42/program/V6PL l LINEͷωοτϫʔΫΛθϩ͔Β࠶ઃܭͨ͠࿩ https://www.janog.gr.jp/meeting/janog43/program/line/ l

    ίϯςϯπͷIPv6ԽͷͨΊʹ͸Կ͕଍Γͳ͍ͷ͔ʁ https://www.janog.gr.jp/meeting/janog47/ipv6/ l LINEͷωοτϫʔΫΦʔέετϨʔλΛϦχϡʔΞϧͨ͠࿩ https://www.janog.gr.jp/meeting/janog49/lineorc/ l “Change” our private cloud infrastructures from single-AZ to multi-AZs https://www.janog.gr.jp/meeting/janog50/multiaz/

More Decks by LINE Developers

Other Decks in Technology