Share
Embed
#23 “VFP: A Virtual Switch Platform for Host SDN in the Public Cloud”
cafenero_777
June 19, 2023
Transcript
19 slides
-
Research Paper Introduction #23 “VFP: A Virtual Switch Platform for
Host SDN in the Public Cloud” ௨ࢉ#74 @cafenero_777 2021/06/10 1 -
Agenda • ରจ • ֓ཁͱಡ͏ͱͨ͠ཧ༝ 1. Introduction 2. Design Goals
and Rationale 3. Overview and Comparison 4. Filtering Model 5. Programming Model 6. Packet Processor and Flow Compiler 7. Switching Model 8. Operational Considerations 9. Hardware O ffl oads and Performance 10.Experiences 11.Conclusions and Future Work 2 -
ରจ • VFP: A Virtual Switch Platform for Host SDN
in the Public Cloud • Daniel Firestone, Microsoft • NSDI ’17 • https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/ fi restone 3 -
֓ཁͱಡ͏ͱͨ͠ཧ༝ • ֓ཁ • VFP: AzureͷNWΛࢧ͑ΔԾεΠονʢTunnel/NAT/Stateful-ACL/QoSʣ • 100ສҎ্ͷIaaS/PaaSϗετͰར༻த • ̐Ҏ্ͷܦݧΛڞ༗
• ಡ͏ͱͨ͠ཧ༝ͱײ • AzureͷԾNWߏ͕ؾʹͳͬͨ • LinuxBridge/OvSͱͷઃܭɾ࣮ͷҧ͍ • τϯωϧͷுΓํ 4 -
1. Introduction • Public Cloud • ސ٬ຖʹϓϥΠϕʔτͳNWۭؒΛఏڙ • ैདྷͷHW switchͰແཧ
-> ϗετ্(HV্)ͷԾSW • SDNͰεέʔϧ͢ΔC-plane + D-planeͷ྆ํ͕ॏཁ • VFP: Virtual Filtering Platform • શHV্Ͱಈ࡞͢ΔԾSW, ԾNICͷ”ϑΟϧλ”ͱͯ͠ಈ࡞ • SDNϙϦγʔͱ࿈ಈͯ͠ಈ࡞ • stateless-tunnel (VNET), Ananta (L4LB=LB-NAT), etc 5 -
2. Design Goals and Rationale • ݩʑHyper-V্ͷNWϑΟϧλυϥΠό • ACL/Tunnel/B/QoSͷػೳ͕”ݻఆ”͞Ε͍ͯͨɻ2011ʹ͜ΕΛಠཱͤ͞APIΛੜͨ͠ɺMAT (Match/Action
Table) baseʹมߋɻVL2ͱҧ͏ɻ • ΰʔϧ 1. ୯ҰͷSDN-ControllerͰͳ͘ɺෳͷControllerΛಉ࣌ʹ͑ΔʢׂΛׂͰ͖ΔɻϚϧνίϯτϩʔϥϞσϧʣ 2. ύέοτ୯ҐͰͳ͘ɺίωΫγϣϯ୯ҐͰॲཧΛ͢Δɻstateful connection͕ fi st class object. (ASICॲཧͱͷҧ͍) i.e. NAT/ACL 3. Encap/DecapͷϩδοΫΛίϯτϩʔϥʹԡ͠ࠐΊͳ͍ɻ͍͍ײ͡ͷεΩʔϚΛD-planeʹಥͬࠐΊΔΑ͏ʹ͓ͯ͘͠ͱɺߋ৽͕͍ʢP4·ͰཁΒͳ͍ʣ 4. VFP updateʹΑΔ௨৴அڐ༰͞Εͳ͍ -> ෳࡶͳstateful fl owΛ͏ͷݫ͍͠ 5. ڊେͳϧʔϧ܈ͰߴૹΛ࣮ݱɿ10+ tables, 40Gbps+, ΩϟογϡͷΈɺޮతͳclassi fi cationͷ 6. SRIOV/NIC-o ffl oad: NICΩϟύʹऩ·ΔΑ͏ʹࣄલίϯύΠϧ+શҰகϑϩʔϞσϧͰରԠ • ΰʔϧ͡Όͳ͍ͷ • ΫϩεϓϥοτϑΥʔϜɿWindows only. ϑΝετύεҠ২͕ࠔ • ίϯτϩʔϥʔ༻ϓϩτίϧɿOVS/OVSDBϓϩτίϧతͳͷ • con fi gͷিಥɾෆਖ਼ݕɿ ඞཁͰՄೳͰແ͍ʂͦͷΘΓʹ։ൃऀ༻ϙϦγʔݕূπʔϧ࡞ͬͨ 6 Ref: https://dl.acm.org/doi/10.1145/1594977.1592576 VFPv1 VFPv2 -
3. Overview and Comparison • O(1M) VMsΛఆͨ͠γφϦΦΛఆ͠ൺֱ • OvSͱͷൺֱɺOvSૉΒ͍͠ɺ͕ɺ •
ϚϧνίϯτϩʔϥϞσϧʹ߹Θͳ͍ɻtable͕ॱํ͔͠μϝ -> stateful layerԽ • MATͰεςʔτϑϧαϙʔτ͞Εͳ͍ -> stateful-layerԽ • MATͰEncap/DecapΛϞσϧԽͰ͖ΔΑ͏ʹमਖ਼ • virtual/physical address mappingΛ୳ͤΔΈ • ൚༻తͳΦϑϩʔυΞΫγϣϯAPI͕ͳ͍ -> ࡞ͬͨɻಛʹVTEP schema!! 7 -
5. Programming Model • ϙʔτɿVM/NICͷingress/egress͕ϕʔε • ϨΠϠʔ • ֤ػೳ͝ͱͷ·ͱ·ΓɻΓٯʹḷΔʹεςʔτϑϧ, Γ5tuple͕ٯ
• LBྫɿߦ͖: VIP -> DIP, ؼΓ: DIP-> VIPɻVIPۭؒ, DIPۭؒΛલޙͷϨΠϠʔͰಠཱͯ͠ϧʔϧ͕ॻ͚Δ • 1ϨΠϠʔͰmatch/action͞ΕΔϧʔϧ1͚ͭͩɺॲཧॱ༗Γ • ϧʔϧɿmatch/actionΤϯςΟςΟ • Encap/Decap actionͷ߹ඞཁͳσʔλɾΩʔΛड͚औΔ • άϧʔϓ • ϧʔϧ܈ʢάϧʔϓ͕࠷খ୯Ґʣ • ྫɿDockerίϯςφͰಠࣗIPΛ͍͍ͨ߹ɻACL (ϢʔβఆٛͱΠϯϑϥఆٛͷ྆ํΛ͍͍ͨ߹) • ͦͷଞ • ͜ΕΒΛϙϦγʔΦϒδΣΫτͱͯ͠ѻ͑Δɻaddress mapping (hash), ಈతNAT-key • Fast eventing APIͱඇಉظI/O 9 -
6. Packet Processor and Flow Compiler (1/2) • ύϑΥʔϚϯεΛ্ͤ͞ΔΈ •
VFPv1 ଟஈͷϨΠϠʔͩͱੑೳͰͳ͍ -> ύέοτͦͷͷͰͳ͘ϝλσʔλԽͯ͠matchͤ͞Δ • UFID: fl ow IDͷ౷Ұ • Header Transpositions (HTs) ~= OvS actionͷϔομૢ࡞ // ද2, ۩ମྫද3 • v2͔Βϝλσʔλૢ࡞ɻ • UFID/HTsͷΩϟογϡԽ (UF Table = fast-path = OvS micro fl ow cache) • fi rst packetͷΈslow-path (HTΤϯδϯ)Λ௨Δ • τϯωϧαϙʔτͰ͖ΔΑ͏ʹ • ύέοτमਖ਼Ҏ֎ͷΞΫγϣϯ • HTsͷcallbackͱ࣮ͯ͠ɿϝʔλϦϯάɺ҉߸Խ 10 -
6. Packet Processor and Flow Compiler (2/2) • Flow Reconciliation
• ϧʔϧΛมߋޙɺUF͕͋ͬͯ৽͍͠ϧʔϧΛద༻͍ͤͨ͞ • portຖʹੈ൪߸͕༩ɺཧ͞ΕΔ • TCPϑϩʔঢ়ଶͷɿٯํͷΧϯλΛݟͯஅ • ϑϩʔͷϖΞϦϯά • DSRͷ߹”୯७ͳٯ”Ͱͳ͍ • inbound UF -> iUFID … ٯʹ͢Δ -> outbound UFΛੜ • TCPτϥοΩϯά • TCPεςʔτϚγϯΛ͏ɻsyn- fl oodରࡦͰ͖Δɻ֤छstatsɾஅͰ͖Δ • ύέοτྨ • ෳࡶͳACLϧʔϧɾԿઍϧʔϧ͋ΔͱύϑΥʔϚϯεӨڹ • ѹॖτϥΠ (compressed trie), ۠ؒ (interval tree), hash-table, listͰ࠷దԽ 11 -
7. Switching Model • FilterҎ֎ɺͭ·Γ௨ৗͷύέοτసૹ • BridgeʹΑΔL2సૹ (outer or inner
MAC addressʹΑΔ) • ϔΞϐϯɺϛϥʔϦϯάVFPͰॲཧɺಛʹgateway VMͩͱ༗ޮɻ • QoSαϙʔτɿଳҬ༧ 12 -
8. Operational Considerations • อकੑ (serviceability)ɺϞχλϦϯάɺ diagnosticsඞਢʂ • Rebootless update:
1ඵະຬͰྃͰ͖ΔɻVM͔Β͔Δ • State Save/Restore: VFPͷACL/NATͷঢ়ଶҡ͕࣋త • VMϚΠάϨʔγϣϯɿStateҡ࣋͞ΕΔ (্هSSRར༻) • ϙʔτɾϨΠϠɾϧʔϧຖʹ300Ҏ্ͷΧϯλɾ౷ܭใ //ද8 • அػೳ • VFP/VFPAPIΫϥΠΞϯτ྆ํͰஅՄೳ • ྫɿҙͷUFIDΛͬͨΫΤϦ • VFPͷτϨʔε༗ޮʹ͢Ε࣮ࡍͷΞΫγϣϯϩά͕ݟΕΔ • snapshotΛऔಘ͠ɺϩʔΧϧͰ෮ݩɾղੳ 13 -
9. Hardware Of fl oads and Performance • NIC (HW)ͰΦϑϩʔυ
• NVGRE/VXLANΦϑϩʔυ: • 40GbpsϥΠϯϨʔτEncapୡ • QoSΦϑϩʔυ • VFPϙϦγʔΦϑϩʔυ • ଟஈtableΛྻlookup͢ΔͱTCAM/CPU͕ඞཁ • Uni fi ed FlowsͰhash͢Δ͜ͱͰղফ • VMͰ25GbpsϥΠϯϨʔτ, VNETE2EͰ<25us 14 TCP synͰ1-2ܻੑೳྑ͍ ϨΠϠ૿͑ͯੑೳҡ࣋ Q (CPU)Ͱੑೳ্ ཧ(PacketDirect)ͷ߹ ΑΓྑ͍ -
10. Experiences (1/3) • ΰʔϧ ʢP.6 ࠶ܝʣ 1. ୯ҰͷSDN-ControllerͰͳ͘ɺෳͷControllerΛಉ࣌ʹ͑ΔʢׂΛׂͰ͖ΔɻϚϧνίϯτϩʔϥϞσϧʣ •
ϨΠϠʔಠཱͤ͞ɺଞͷίϯτϩʔϥʹӨڹͳ͘σϓϩΠՄೳ 2. ύέοτ୯ҐͰͳ͘ɺίωΫγϣϯ୯ҐͰॲཧΛ͢Δɻstateful connection͕ fi st class object. (ASICॲཧͱͷҧ͍) i.e. NAT/ACL • VFPͰશͯͷଓεςʔτϑϧʹѻΘΕΔ 3. Encap/DecapͷϩδοΫΛίϯτϩʔϥʹԡ͠ࠐΊͳ͍ɻ͍͍ײ͡ͷεΩʔϚΛD-planeʹಥͬࠐΊΔΑ͏ʹ͓ͯ͘͠ͱɺߋ৽͕͍ʢP4·ͰཁΒͳ͍ʣ • VNET, LB, ACL͕VFPͷมߋͳ͠Ͱ͍ଓ͚ΒΕΔɻ 4. VFP updateʹΑΔ௨৴அڐ༰͞Εͳ͍ -> ෳࡶͳstateful fl owΛ͏ͷݫ͍͠ • RebootlessͰԿupdate͖ͯͨ͠ 5. ڊେͳϧʔϧ܈ͰߴૹΛ࣮ݱɿ10+ tables, 40Gbps+, ΩϟογϡͷΈɺޮతͳclassi fi cationͷ • UFTͰVFPੑೳ্ɺಛʹϨΠϠʔ͕ଟஈͷ߹ɻ 6. SRIOV/NIC-o ff l oad: NICΩϟύʹऩ·ΔΑ͏ʹࣄલίϯύΠϧ+શҰகϑϩʔϞσϧͰରԠ • UFTͰͷ୳ࡧʹͯ1-2ܻੑೳ্ɺ͞ΒʹHWΦϑϩʔυ (SRIOV)͠ੑೳ্ 15 -
10. Experiences (2/3) • 2012Ҏ߱ɺVFP21ճϝδϟʔϦϦʔεɺશͯͷAzureαʔόɺ30regions • L4 fl ow cacheͰेɺOvSͰݴ͏mega
fl ow (wildcard)ඞཁͳ͍ɺͱ͍͏ओு • Statefulʹ͢ΔͳΒ࠷ॳ͔ΒΖ͏ɻޙ͔ΒMatch/action tableͷมߋແཧʂ • ϨΠϠϦϯάॏཁɿίϯτϩʔϥʔؒͰͷϨΠϠϦϯάηϚϯςΟΫεͰ֤ίϯτϩʔϥΛਖ਼͘͠ಠཱͤ͞Δ • goto༗ɿ”ؾΛ͚ͭΔ”ແཧɻϨΠϠϦϯάηϚϯςΟΫεͰΒͤΔ • IaaSμϯλΠϜʹහײɿupdateํ๏ΛͪΌΜͱઃܭͯ͠அ࣌ؒΛอূ͢ΔɻͦͷͨΊʹඞཁͳػೳΛՃ, StateSaveRestoreͳͲɻ • D-planeͱwire protocolΓͤɿO(1M)εέʔϧͷcontroller/agentϞσϧͰͷ࣮ܦݧɻVFPAPI (southbound API)ͷಠཱ 16 ղઆऀͷ ؾʹͳΔ -
10. Experiences (3/3) • ίϯϑϦΫτݕ݁ہཁΒͳ͔ͬͨɻಠཱίϯτϩʔϥ͕͏·͘ಈ࡞ɻίϯτϩʔϥσόοάπʔϧͷํ͕େ • શͯ”ΞΫγϣϯ”: VL2ͷτϯωϧτϯωϧI/Fͩͬͨɻ͜ΕΛΞΫγϣϯʹͯ͠MA͕γϯϓϧʹͳΓΦϑϩʔυͰ͖ͨ • MTUݒ೦͕ͩͬͨɺΞϯμʔϨΠଆΛେ͖͘औΔ͜ͱͰͳ͠
• MATεέʔϧɿ࠷େ10-20ϨΠϠɺ1ϨΠϠ࠷େඦάϧʔϓɺ1άϧʔϓลΓO(50k)ϧʔϧɺ1ϙʔτ͋ͨΓ50ສTCP current-conn • γϯϓϧͳMACΞυϨεϕʔεͷϑΥϫʔσΟϯάҎ֎ͷํ๏ݟ͔ͭΒͣ • E2EϞχλϦϯάɿVMʹೖΕͳ͍ͷͰɺVFPϧʔϧͱͯ͠inject/responseΛ࡞Γɺ͜ΕͰϞχλϦϯάɻVM/ϗετڥքΛࢹ • ༻NICཧతͰͳ͍ɿAzureͷϙϦγʔSR-IOVͰΦϑϩʔυͰ͖ͳ͍ɻFPGAϕʔεͳNICͰ࣮ 17 ղઆऀͷ ؾʹͳΔ -
11. Conclusion and Future Work • VFP (Virtual Filtering Platform)Microsoft
AzureͷԾεΠον • ϓϩάϥϚϏϦςΟͱεέʔϥϏϦςΟΛհ • ຊ൪ڥͰͷอकੑɾϞχλϦϯάɾdiagnosticsͷݒ೦ͱܦݧ • ࠓޙΑΓ৽͍͠HWϞσϧVFPΦϑϩʔυݴޠͷ֦ு 18