LINEのネットワーク / Network of LINE

LINEのネットワーク / Network of LINE

53850955f15249a1a9dc49df6113e400?s=128

LINE Developers

February 15, 2019
Tweet

Transcript

  1. -*/&ͷωοτϫʔΫ :PIFJ ,BOFNBSV -*/&$PSQPSBUJPO 

  2. ࣗݾ঺հ ؙۚ ༸ฏ ωοτϫʔΫνʔϜ d σʔληϯλωοτϫʔΫͷӡ༻ɺϐΞϦϯάௐ੔ ΦϑΟεແઢ-"/ͷߏஙɾӡ༻ 0QFO4UBDLϕʔεͷϓϥΠϕʔτΫϥ΢υ7FSEBͷ։ൃνʔϜ d -PBE#BMBODFSBTB4FSWJDF

    -#BB4 ͷઃܭɾ։ൃɾߏஙɾӡ༻ 2019-02-06 2
  3. "HFOEB -*/&ͷωοτϫʔΫͷ֓ཁ ࠷ۙͷऔΓ૊Έࣄྫ ϩʔυόϥϯαͷࣗࣾ։ൃ σʔληϯλωοτϫʔΫͷ࡮৽ ࠓޙͷऔΓ૊Έ༧ఆ 43WʹΑΔॊೈͳωοτϫʔΫͷ࣮ݱ 2019-02-06 3

  4. -*/&ͷωοτϫʔΫͷ֓ཁ 2019-02-06 4

  5. lωοτϫʔΫz 2019-02-06 https://syllabus.kyoto-su.ac.jp/syllabus/html/2018/3783.html / 5

  6. -*/&ʹ͓͚ΔlωοτϫʔΫz 2019-02-06 /$!D9 %2D9 <5B@ &+ -"%)/ NAT E6@ :H'+

    ?@LAN IP",C5 (#*/ NIC TCP TLS + /  + 7G14  =I3  F>; 0$!.A8 /!-) DNS 6
  7. Datacenter Network Datacenter Network యܕతͳΦϯϓϨϛεωοτϫʔΫ 2019-02-06 Datacenter Interconnect Internet Internet

    North-South traffic East-West traffic East-West traffic North-South traffic 7
  8. ωοτϫʔΫͷಛ৭ ಛ௃తͳ/PSUI4PVUIτϥϑΟοΫ ಛఆͷ஍Ҭ΍*41ʹภ͍ͬͯΔ ৗʹ5$1ίωΫγϣϯΛҡ࣋͢Δ Πϕϯτੑͷ͋ΔτϥϑΟοΫύλʔϯ ৗʹ߈ܸʹࡽ͞Ε͍ͯΔ ലେͳ&BTU8FTUτϥϑΟοΫ σʔλ෼ੳʹ൐͏αʔόؒτϥϑΟοΫ ϚΠΫϩαʔϏε 2019-02-06

    8
  9. ωοτϫʔΫʹٻΊΒΕΔ͜ͱ ๲େͳτϥϑΟοΫΛॲཧͰ͖Δ͜ͱ ਺ඦ(CQTن໛ͷ/PSUI4PVUIτϥϑΟοΫ ਺5CQTن໛ͷ&BTU8FTUτϥϑΟοΫ ؆୯͔ͭਝ଎ʹεέʔϧͰ͖Δ͜ͱ ͢͹΍͍ࣄۀల։ʹ଱͑͏Δߏ੒ ҆ఆͨ͠ӡ༻͕Ͱ͖Δ͜ͱ ͋ΒΏΔ΋ͷ͸յΕΔ 2019-02-06 9

  10. Πϯϑϥͷ՝୊ղܾʹର͢Δߟ͑ํ ࠜຊ͔Βղܾ͢Δ͜ͱΛࢼΈΔ ର঱ྍ๏తͳ΋ͷΑΓ͸ɺΞʔΩςΫνϟ͔Βݟ௚͢ ՄೳͳݶΓγϯϓϧʹอͭ ࠷খݶͷߏ੒ཁૉ΍Φʔϓϯͳϓϩτίϧ εςʔτϑϧΑΓ͸εςʔτϨεߏ੒ ద੾ͳλΠϛϯάͰ౤ೖ͢Δ Πϯϑϥߏ੒ཁૉͷϥΠϑαΠΫϧ͸ԟʑʹͯ͠௕͍ 2019-02-06 10

  11. ࠷ۙͷऔΓ૊Έࣄྫ ϩʔυόϥϯαͷࣗࣾ։ൃ ΞʔΩςΫνϟ͔Βࠜຊతʹมߋ ࠾༻ࣄྫͷແ͍ٕज़Ͱ͋ͬͯ΋ඞཁͰ͋Ε͹બ୒ ͚͓͋ΊτϥϑΟοΫʹ଱͑ΔύϑΥʔϚϯεΛ࣮ݱ σʔληϯλωοτϫʔΫͷ׬શ-Խ ΞʔΩςΫνϟ͔Βࠜຊతʹมߋ ωοτϫʔΫɾαʔόͷ૒ํΛแׅతʹઃܭ εέʔϥϏϦςΟͱӡ༻ίετܰݮΛཱ྆ 2019-02-06

    11
  12. ࣄྫ ϩʔυόϥϯα։ൃ 2019-02-06 12

  13. -#ͷ໾ׂ 7JSUVBM*1 7*1 Ѽͷ௨৴ΛԿΒ͔ͷϙϦγʔʹै࣮ͬͯαʔόʹసૹ ෛՙ෼ࢄ΍৑௕ੑ֬อ͕ओͳ༻్ *1ϒϥοΫϦετ΍5-4ऴ୺ͱ͍ͬͨ෇ՃػೳΛ࣋ͭ΋ͷ΋ 2019-02-06 Real Server (RS)

    LB Client VIP RSIP i • RS   • IP   • TLS (TLS Offload) • … 13
  14. ΦϯϓϨϛε؀ڥͰͷయܕతͳ-#αʔϏεߏ੒ 2019-02-06 OpenStack Octavia     HAProxy LVS

    (IPVS) LBaaS Frontend LBaaS Backend LB API / Web UI API API / Web UI / CLI 14
  15. -#ͷมભͱ௚໘ͨ͠՝୊ ॳظͷஈ֊ΑΓϋʔυ΢ΣΞΞϓϥΠΞϯε੡඼Λ࢖༻ εςʔτϑϧʹಈ࡞͢Δ୆ηοτߏ੒ εςʔτΛอଘ͢Δςʔϒϧ͕࣌ંރׇ͢ΔΑ͏ʹ ಉ࣌઀ଓ਺͕ଟ͍ͷʹՃ͑ɺಈ࡞ํࣜݻ༗ͷ໰୊΋ൃੜ -#୆਺͕૿͑ΔʹͭΕίετύϑΥʔϚϯεѱԽ ඞཁͳΩϟύγςΟͷഒΛ༻ҙ͢Δඞཁ͕͋Δ 2019-02-06 15

  16. /  .VMUJUJFSߏ੒  4UBUFMFTTԽ (PPHMF 'BDFCPPLͳͲͰ࠾༻͞Ε͍ͯΔΞʔΩςΫνϟɾํࣜ <> 2019-02-06 

     / .VMUJUJFS Client L4LB (Hardware) Real Server : In service : Not in service L4LB (Software) Real Server L7LB (Software) L3 Switch (Hardware) Client Stateful L3DSR TCP/HTTP(S) Proxy Stateless L3DSR ECMP [1]     http://yunazuno.hatenablog.com/entry/2016/02/29/090001 16
  17. /  .VMUJUJFS / ϝϯςφϯε༰қԽɺো֐υϝΠϯͷ࠷খԽ .VMUJUJFSಛఆͷॲཧʹूதͤ͞Δ͜ͱͰεέʔϧΞ΢τΛ༰қԽ 2019-02-06 L3 Switch L4LB

    L7LB Whitebox Switch 100G x 32 ports JPY 2,000,000 x86 Linux Server 25G x 1 ports JPY 800,000 x86 Linux Server 25G x 1 ports JPY 800,000          17
  18. L4LB Program -*/&ͷ-#։ൃͱύέοτॲཧͷؔ܎ --#ͷύέοτॲཧ෦෼͸ιϑτ΢ΣΞ 9%1 Ͱ࣮૷ 2019-02-06 Rx Tx Match

    Lookup Rewrite Table VIP-Backend table Statistics table From L3 Switch To L7 LB NIC 1. Match Dst. IP/Port  2. Lookup Hash  Real Server   3. Rewrite IP  Dst. IP, DSCP     18
  19. 9%1͸ͳͥߴ଎ͳύέοτॲཧ͕Մೳ͔ ௨ৗͷந৅Խ͞ΕͨωοτϫʔΫελοΫʹύέοτ͕౉Δલʹॲཧ %1%,΍OFUNBQͱ͍ͬͨྨࣅٕज़Ͱ͋ͬͯ΋جຊతͳߟ͑ํ͸ಉ͡ 2019-02-06  XDP NIC Kernel Kernel User

    NIC Driver Ethernet IPv4 TCP App Socket NIC Driver XDP App Ether net IPv4 TCP App Socket 19
  20. ࣄྫ σʔληϯλωοτϫʔΫ 2019-02-06 20

  21. σʔληϯλωοτϫʔΫͷมભͱ՝୊ ϨΠϠϕʔεͷωοτϫʔΫ ن໛֦େʹͭΕෆ҆ఆԽ  Λલఏͱͨ͠ߏ੒ཁૉ ΩϟύγςΟͷෆ଍ ߴ͍ӡ༻ෛՙ ಛఆϕϯμʹറΒΕΔ 2019-02-06 21

  22. ղܾࡦ -MFTT  #(1 $-04 ωοτϫʔΫػثɾαʔό૒ํ͕-ϧʔςΟϯά #(1 Λ࢖༻ / ߏ੒ʹΑΓεέʔϥϏϦςΟͱΩϟύγςΟΛ֬อ

    2019-02-06   7,200 servers / POD 7,200 servers / POD 100G x 720 links 10G x 14,400 links 100G x 80 links 100G x 1,440 links 72,000Gbps Capacity L3 22
  23. 2019-02-06 23

  24. ࠓޙͷऔΓ૊Έ 43W 2019-02-06 24

  25. γϯϓϧͳ-ωοτϫʔΫ্ͰॊೈͳॲཧΛ ৽ཁ݅ϚϧνςφϯγʔɺαʔϏενΣΠχϯά -ͷٕज़ 79-"/ Ͱ࣮ݱ͢Δͷ͕௨ྫ ʜ͕ɺ-MFTTͷԸܙΛாফ͠ʹ͢Δଆ໘΋ ొ৔ͨ͠͹͔Γͷ৽ͨͳٕज़Ͱ΋ݕ౼ 43W*1W4FHNFOU3PVUJOH ݱࡏݕূΛ࣮ࢪத 2019-02-06

    SRv6 Underlay Tenant A Tenant B Tenant C 25
  26. ·ͱΊ 2019-02-06 26

  27. جຊΛॏࢹͭͭ͠՝୊Λղܾ ͞·͟·ͳٕज़త՝୊Λͻͱͭͻͱͭղܾ σʔληϯλωοτϫʔΫɺϩʔυόϥϯαɺʜ ϕʔεͱͳΔߟ͑ํΛॏࢹ͠ͳ͕ΒखஈΛݕ౼ ৽͍͠ɾݹ͍ɺͰ͸ͳ͘ɺద੾͔Ͳ͏͔Λॏࢹ ࠓޙ΋৽ͨͳ෼໺΁ͷ௅ઓΛܧଓ ίϯςφωοτϫʔΩϯά΋ͦͷͻͱͭ 2019-02-06 27