-*/&ͷωοτϫʔΫ :PIFJ ,BOFNBSV -*/&
$PSQPSBUJPO  

ࣗݾ঺հ ؙۚ ༸ฏ ωοτϫʔΫνʔϜ 
d
 σʔληϯλωοτϫʔΫͷӡ༻ɺϐΞϦϯάௐ੔ ΦϑΟεແઢ-"/ͷߏஙɾӡ༻ 0QFO4UBDLϕʔεͷϓϥΠϕʔτΫϥ΢υ7FSEBͷ։ൃνʔϜ 
d -PBE
#BMBODFS
BT
B
4FSWJDF
-#BB4
ͷઃܭɾ։ൃɾߏஙɾӡ༻ 2019-02-06 2 

"HFOEB -*/&ͷωοτϫʔΫͷ֓ཁ ࠷ۙͷऔΓ૊Έࣄྫ ϩʔυόϥϯαͷࣗࣾ։ൃ σʔληϯλωοτϫʔΫͷ࡮৽ ࠓޙͷऔΓ૊Έ༧ఆ 43WʹΑΔॊೈͳωοτϫʔΫͷ࣮ݱ 2019-02-06 3 

-*/&ͷωοτϫʔΫͷ֓ཁ 2019-02-06 4 

lωοτϫʔΫz 2019-02-06 https://syllabus.kyoto-su.ac.jp/syllabus/html/2018/3783.html / 5 

-*/&ʹ͓͚ΔlωοτϫʔΫz 2019-02-06 /
$!D9 
%2D9 <5B@ &
+ -
"%)/ NAT E6@ :H
'+ ?@LAN IP",C5 (#*/ NIC TCP TLS +
/  +
 7G14 =I3  F>;0$!.
A8 /!-
) DNS 6 

Datacenter Network Datacenter Network యܕతͳΦϯϓϨϛεωοτϫʔΫ 2019-02-06 Datacenter Interconnect Internet Internet North-South traffic East-West traffic East-West traffic North-South traffic 7 

ωοτϫʔΫͷಛ৭ ಛ௃తͳ/PSUI4PVUIτϥϑΟοΫ ಛఆͷ஍Ҭ΍*41ʹภ͍ͬͯΔ ৗʹ5$1ίωΫγϣϯΛҡ࣋͢Δ Πϕϯτੑͷ͋ΔτϥϑΟοΫύλʔϯ ৗʹ߈ܸʹࡽ͞Ε͍ͯΔ ലେͳ&BTU8FTUτϥϑΟοΫ σʔλ෼ੳʹ൐͏αʔόؒτϥϑΟοΫ ϚΠΫϩαʔϏε 2019-02-06 8 

ωοτϫʔΫʹٻΊΒΕΔ͜ͱ ๲େͳτϥϑΟοΫΛॲཧͰ͖Δ͜ͱ ਺ඦ(CQTن໛ͷ/PSUI4PVUIτϥϑΟοΫ ਺5CQTن໛ͷ&BTU8FTUτϥϑΟοΫ ؆୯͔ͭਝ଎ʹεέʔϧͰ͖Δ͜ͱ ͢͹΍͍ࣄۀల։ʹ଱͑͏Δߏ੒ ҆ఆͨ͠ӡ༻͕Ͱ͖Δ͜ͱ ͋ΒΏΔ΋ͷ͸յΕΔ 2019-02-06 9 

Πϯϑϥͷ՝୊ղܾʹର͢Δߟ͑ํ ࠜຊ͔Βղܾ͢Δ͜ͱΛࢼΈΔ ର঱ྍ๏తͳ΋ͷΑΓ͸ɺΞʔΩςΫνϟ͔Βݟ௚͢ ՄೳͳݶΓγϯϓϧʹอͭ ࠷খݶͷߏ੒ཁૉ΍Φʔϓϯͳϓϩτίϧ εςʔτϑϧΑΓ͸εςʔτϨεߏ੒ ద੾ͳλΠϛϯάͰ౤ೖ͢Δ Πϯϑϥߏ੒ཁૉͷϥΠϑαΠΫϧ͸ԟʑʹͯ͠௕͍ 2019-02-06 10 

࠷ۙͷऔΓ૊Έࣄྫ ϩʔυόϥϯαͷࣗࣾ։ൃ ΞʔΩςΫνϟ͔Βࠜຊతʹมߋ ࠾༻ࣄྫͷແ͍ٕज़Ͱ͋ͬͯ΋ඞཁͰ͋Ε͹બ୒ ͚͓͋ΊτϥϑΟοΫʹ଱͑ΔύϑΥʔϚϯεΛ࣮ݱ σʔληϯλωοτϫʔΫͷ׬શ-Խ ΞʔΩςΫνϟ͔Βࠜຊతʹมߋ ωοτϫʔΫɾαʔόͷ૒ํΛแׅతʹઃܭ εέʔϥϏϦςΟͱӡ༻ίετܰݮΛཱ྆ 2019-02-06 11 

ࣄྫ ϩʔυόϥϯα։ൃ 2019-02-06 12 

-#ͷ໾ׂ 7JSUVBM
*1
7*1
Ѽͷ௨৴ΛԿΒ͔ͷϙϦγʔʹै࣮ͬͯαʔόʹసૹ ෛՙ෼ࢄ΍৑௕ੑ֬อ͕ओͳ༻్ *1ϒϥοΫϦετ΍5-4ऴ୺ͱ͍ͬͨ෇ՃػೳΛ࣋ͭ΋ͷ΋ 2019-02-06 Real Server (RS) LB Client VIP RSIP i • RS  
• IP  • TLS (TLS Offload) • … 13 

ΦϯϓϨϛε؀ڥͰͷయܕతͳ-#αʔϏεߏ੒ 2019-02-06 OpenStack Octavia 
   HAProxy LVS (IPVS) LBaaS Frontend LBaaS Backend  LB API / Web UI API API / Web UI / CLI 14 

-#ͷมભͱ௚໘ͨ͠՝୊ ॳظͷஈ֊ΑΓϋʔυ΢ΣΞΞϓϥΠΞϯε੡඼Λ࢖༻ εςʔτϑϧʹಈ࡞͢Δ୆ηοτߏ੒ εςʔτΛอଘ͢Δςʔϒϧ͕࣌ંރׇ͢ΔΑ͏ʹ ಉ࣌઀ଓ਺͕ଟ͍ͷʹՃ͑ɺಈ࡞ํࣜݻ༗ͷ໰୊΋ൃੜ -#୆਺͕૿͑ΔʹͭΕίετύϑΥʔϚϯεѱԽ ඞཁͳΩϟύγςΟͷഒΛ༻ҙ͢Δඞཁ͕͋Δ 2019-02-06 15 

/ .VMUJUJFSߏ੒  4UBUFMFTTԽ (PPHMF
'BDFCPPLͳͲͰ࠾༻͞Ε͍ͯΔΞʔΩςΫνϟɾํࣜ <> 2019-02-06 

 /


.VMUJUJFS Client L4LB (Hardware) Real Server : In service : Not in service L4LB (Software) Real Server L7LB (Software) L3 Switch (Hardware) Client Stateful L3DSR TCP/HTTP(S) Proxy Stateless L3DSR ECMP [1] 

   http://yunazuno.hatenablog.com/entry/2016/02/29/090001 16 

/ .VMUJUJFS /
ϝϯςφϯε༰қԽɺো֐υϝΠϯͷ࠷খԽ .VMUJUJFS
ಛఆͷॲཧʹूதͤ͞Δ͜ͱͰεέʔϧΞ΢τΛ༰қԽ 2019-02-06 L3 Switch L4LB L7LB Whitebox Switch 100G x 32 ports JPY 2,000,000 x86 Linux Server 25G x 1 ports JPY 800,000 x86 Linux Server 25G x 1 ports JPY 800,000

 
      17 

L4LB Program -*/&ͷ-#։ൃͱύέοτॲཧͷؔ܎ --#ͷύέοτॲཧ෦෼͸ιϑτ΢ΣΞ 9%1
Ͱ࣮૷ 2019-02-06 Rx Tx Match Lookup Rewrite Table VIP-Backend table Statistics table From L3 Switch To L7 LB NIC 1. Match Dst. IP/Port  2. Lookup Hash  Real Server   3. Rewrite IP   Dst. IP, DSCP 
   18 

9%1͸ͳͥߴ଎ͳύέοτॲཧ͕Մೳ͔ ௨ৗͷந৅Խ͞ΕͨωοτϫʔΫελοΫʹύέοτ͕౉Δલʹॲཧ %1%,΍OFUNBQͱ͍ͬͨྨࣅٕज़Ͱ͋ͬͯ΋جຊతͳߟ͑ํ͸ಉ͡ 2019-02-06
 XDP NIC Kernel Kernel User NIC Driver Ethernet IPv4 TCP App Socket NIC Driver XDP App Ether net IPv4 TCP App Socket 19 

ࣄྫ σʔληϯλωοτϫʔΫ 2019-02-06 20 

σʔληϯλωοτϫʔΫͷมભͱ՝୊ ϨΠϠϕʔεͷωοτϫʔΫ ن໛֦େʹͭΕෆ҆ఆԽ Λલఏͱͨ͠ߏ੒ཁૉ ΩϟύγςΟͷෆ଍ ߴ͍ӡ༻ෛՙ ಛఆϕϯμʹറΒΕΔ 2019-02-06 21 

ղܾࡦ -MFTT  #(1 $-04 ωοτϫʔΫػثɾαʔό૒ํ͕-ϧʔςΟϯά #(1 Λ࢖༻ /ߏ੒ʹΑΓεέʔϥϏϦςΟͱΩϟύγςΟΛ֬อ 2019-02-06







7,200 servers / POD 7,200 servers / POD 100G x 720 links 10G x 14,400 links 100G x 80 links 100G x 1,440 links 72,000Gbps Capacity L3 22 

2019-02-06 23 

ࠓޙͷऔΓ૊Έ 43W 2019-02-06 24 

γϯϓϧͳ-ωοτϫʔΫ্ͰॊೈͳॲཧΛ ৽ཁ݅
ϚϧνςφϯγʔɺαʔϏενΣΠχϯά -ͷٕज़ 79-"/
Ͱ࣮ݱ͢Δͷ͕௨ྫ ʜ͕ɺ-MFTTͷԸܙΛாফ͠ʹ͢Δଆ໘΋ ొ৔ͨ͠͹͔Γͷ৽ͨͳٕज़Ͱ΋ݕ౼ 43W
*1W
4FHNFOU
3PVUJOH ݱࡏݕূΛ࣮ࢪத 2019-02-06 SRv6 Underlay Tenant A Tenant B Tenant C 25 

·ͱΊ 2019-02-06 26 

جຊΛॏࢹͭͭ͠՝୊Λղܾ ͞·͟·ͳٕज़త՝୊Λͻͱͭͻͱͭղܾ σʔληϯλωοτϫʔΫɺϩʔυόϥϯαɺʜ ϕʔεͱͳΔߟ͑ํΛॏࢹ͠ͳ͕ΒखஈΛݕ౼ ৽͍͠ɾݹ͍ɺͰ͸ͳ͘ɺద੾͔Ͳ͏͔Λॏࢹ ࠓޙ΋৽ͨͳ෼໺΁ͷ௅ઓΛܧଓ ίϯςφωοτϫʔΩϯά΋ͦͷͻͱͭ 2019-02-06 27