Research Paper Introduction #19 Sirius: A Flat Datacenter Network with Nanosecond Optical Switching

https://www.microsoft.com/en-us/research/publication/sirius-a-flat-datacenter-network-with-nanosecond-optical-switching/

Transcript

1. Research Paper Introduction #19 “Sirius: A Flat Datacenter Network with Nanosecond Optical Switching” ௨ࢉ#69 @cafenero_777 2021/03/25
2. Agenda • ର৅࿦จ • ֓ཁͱಡ΋͏ͱͨ͠ཧ༝ 1. Introduction 2. Motivation 3. Building Block Technologies 4. Sirius Architecture 5. Cost and Power Analysis 6. Prototype Implementation 7. Simulation Results 8. Conclusion
3. ର৅࿦จ • Sirius: A Flat Datacenter Network with Nanosecond Optical Switching • Hitesh Ballani Paolo Costa Raphael Behrendt Daniel Cletheroe Istvan Haller Krzysztof Jozwik Fotini Karinou Sophie Lange Kai Shi Benn Thomsen Hugh Williams • Microsoft Research • SIGCOMM 2020 • https://www.microsoft.com/en-us/research/project/sirius/ • MSͷؔ࿈Project 4ܑఋʁ • http://opticsforthecloud.com/
4. ֓ཁͱಡ΋͏ͱͨ͠ཧ༝ • ֓ཁ • ैདྷͷʢిؾʣεΠονͰ͸cloud workloadͷτϥϑΟοΫΛࡹ͖͖Εͳ͍༧૝ • Sirius: DC޲͚all-optical datacenter network (ޫεΠονNW) • 50Gbps/channel, E2E latency<3.84ns • ಡ΋͏ͱͨ͠ཧ༝ • ๭ॴͰ࿩୊ʹͳͬͯͨ • Flat network (non ଟ૚Clos NW)Ͱ750k nodeΛऩ༰?ͲΜͳຐ๏ɺɺ • Optical switching? bu ff er͕ແ͍ͷʹͲ͏΍ͬͯɻɻʁ
5. 1. Introduction • ϜʔΞͷ๏ଇ V.S. DC಺τϥϑΟοΫ • ൜ਓ͸HW acceleration: GPU/TPU, FPGA, non-volatile memory • SW૿΍͢ -> DCશମͰ࢖͑ΔిྗΛ௒͑ͯ͠·͏ • Sirius: all-optical datacenter network • ESWʢిؾεΠονʣΛ࢖͏ͷΛࢭΊͯɺoptical switchΛ࢖͏ • core nw͸grating ʢճં֨ࢠʣΛ࢖͏ͨΊɺిྗ΍Քಇ෦඼͸࢖Θͳ͍ • ೾௕ʹԠͯ͡ग़ྗઌϙʔτ͕มΘΔʢ೾௕=next hopࣝผࢠͬΆ͍΋ͷʣ • ߴ଎ʹ೾௕ΛมߋͰ͖ΔϨʔβ (tunable laser)Λ։ൃ (~ms -> 912ps), E2EͰnsΦʔμʔ • WDMͰ͸ͳ͘TDMʢ࣌ؒಉظͯ͠ૹ৴͢Δʣ • τϙϩδͷϑϥοτԽͰεΠονɾτϥϯγʔόͷ࡟ݮͰফඅిྗ࡟ݮɺ޿ଳҬԽɺ௿ϨΠςϯγʔɺݎ࿚ੑʢnon CMOSʣ ΪϟοϓͱಷԽ https://www.jpu.or.jp/useful/spectrometer/ https://ja.wikipedia.org/wiki/ %E3%82%B3%E3%83%B3%E3%83%91 %E3%82%AF%E3%83%88%E3%83%87 %E3%82%A3%E3%82%B9%E3%82%AF
6. 2. Motivation • ࠷ۙͷCMOS܏޲ • Clos NWͰ֊૚Λ૿΍͢ͱ”੫ۚ”͕͔͔Δ • ੫ۚྫɿফඅిྗ(a)ɺ୆਺ʹΑΔίετɺ஗Ԇ • CMOSεέʔϦϯάݶք(b) 3Dੵ૚Խ…? • ߴ଎ͳεΠονϯά͕ඞཁ • KVS, memory disaggregationͷΑ͏ͳόʔετɾϑΝϯΞ΢τ • 576Bͷ৔߹ɺ92ns͝ͱʹ೾௕ʢѼઌʣΛม͑Δඞཁ͋Γ • ։ൃ໨ඪɿมߋ࣌ؒ͸10%ҎԼ͕͏Ε͍͠ Cloud workload in 2019/03 Sub-Nanosecond Clock and Data Recovery in an 
 Optically-Switched Data Centre Network. ECOC High-Resolution Measurement of Data Center Microbursts. In IMC.
7. 3. Building Block Technologies • Arrayed Wavelength grating Routers (AWGR) • ೖग़ྗΛपظతʹճંʢ=ϧʔςΟϯάʣͤ͞Δ • ࢢൢ඼Ͱ100ϙʔτɺϓϩτλΠϓͰ512ϙʔτ • Tunable lasers • tuningిѹͰग़ྗϨʔβ೾௕(1550nmลΓ)Λ100೾ௐ੔Ͱ͖Δ • switch recon fi guration=೾௕tuning࣌ؒ • ࢢൢ඼͸10ms -> 14ns~92nsΛ։ൃ • ੜ੒෦ɾग़ྗ෦Λ෼཭ • ૉࢠ࠷దԽ • SOAͰ࣍ʹग़͢೾௕Λ”Ӆͯ͠”͓͚Δ->࣌ؒ୹ॖ • staticͳϨʔβ -> tunableϨʔβ -> ޫίϜར༻ • νϡʔχϯάϨΠςϯγ912psΛ࣮ݱ ੜ੒ ग़ྗ
8. 4. Sirius Architecture (1/2) • Physical topology: nodeؒ͸1ର(50Gbps)ͷܦ࿏ͷΈ • 2port, 2೾Ͱ4node (2*2) • ֤αʔόͰ100port, 48೾Λ࢖͑͹4.8kαʔό • ֤ToRͰ256port, 100೾࢖͑͹25.6kϥοΫ, 768kαʔό • ࣌ؒ෼ׂͰૹΔ • WDMతʹଋͶͯૹ৴͸͠ͳ͍ • Routing and scheduling • όοϑΝͳ͠ɻ೾௕͕”িಥ”͢ΔͱࠔΔ • ࣌ࠁಉظ͓͖ͯ͠ɺtime slotΛܾΊ͓ͯ͘ɻscheduler-less • 50GҎ্͸indirect routing (2hop)Λ࢖͏ɻ • 2hopͷ৔߹͸఻ൖ஗Ԇ͸2ഒ + time slotΛ଴ͭ࣌ؒ = ਺us • ύέοτ͕ॱ൪ʹདྷͳ͍৔߹͸᫔᫓੍ޚͰΧόʔ
9. 4. Sirius Architecture (2/2) • Time synchronization • 100psҎԼͰಉظ͍ͨ͠ (i.e. NTP<10ms, PTP<1us) • ֎෦ͷλΠϜεέδϡʔϥ͸࢖Θͳ͍ • ϦʔμʔnodeͷϨʔβʔclockʹ߹ΘͤΔ͚ͩ
 ʢcore͕passiveͰ͋ΔϝϦοτʣ • clock͕υϦϑτͯ͠΋ߏΘͳ͍ • topology಺Ͱಉظ͍ͯ͠Ε͹ྑ͍ • ఻ૹ஗Ԇ͸఻ૹڑ཭͔Βิਖ਼͢Δ • Design discussion • ൒ࢮͷ৔߹͸ɻɻʁ https://www.microsoft.com/en-us/research/publication/sirius-a- fl at-datacenter-network-with-nanosecond-optical-switching/
10. 5. Cost and power analysis • 4000ϥοΫΛ૝ఆͯ͠ࢼࢉ • ESN (Electrically switched network) • 500Wͳ25.6TbpsεΠον(5k$) • 10Wͷ400Gbpsτϥϯγʔό(400$) • 6port SW * 4૚ • Sirius: 1૚ͷESN • 1૚ͷAWGRs • tunable xceiver * 2 • ίετൺֱ • ిྗൺͰ~26%·ͰݮΒͤΔ • Ձ֨ൺͰSWػث୯ମͰਪఆ25%ɺNWશମͰ28-52% 3:1 ిྗ ిྗ ిྗ ిྗͳ͠ʂ ଟ͍ʂʂ গͳ͍ʂ
11. 6. Prototype implementation • v1. ࢢൢϨʔβͰ4nodeߏ੒ • FPGAͰ4೾௕ͷૢ࡞Λ֬ೝ • v2. 912psϨʔβͰτϥϑΟοΫεέδϡʔϦϯά΍ ࣌ࠁಉظΛ֬ೝ • ύοέʔδԽ, 19೾௕, ࠷ѱ஋Ͱ΋912ps • ຊ൪؀ڥ΁ͷಋೖ • ΫϩοΫಉظػೳͷϚΠίϯԽ • ϧʔςΟϯάʢͲͷ೾௕Λ࢖͏͔ʣͱ᫔᫓੍ޚ͸ P4ར༻͕ྑ͍͔΋?
12. 7. Simulation results • 200k fl ows (ฏۉ͸100KB͕ͩɺϩϯάςʔϧ) • 24server * 128rackΛ૝ఆ • ESN (3૚Clos NW) V.S. Sirius (1૚) • ߴෛՙͰ΋FCT௿஗ԆɻNWશମͷଳҬར༻Մೳɻ • fl ow sizeΛখ͘͢͞Δੑೳ͕ྼԽؾຯ • 16KiB෇ۙͰ͸͕ࠩ1.2ഒఔ౓ • ݻఆαΠζͷ”ηϧ”Λ༻͍ΔͨΊͷΦʔόʔϔου༗Γ fl ow size<100KBʹͯ͠ɺ fl ow਺Λ૿΍ͨ͠৔߹ɻ fl ow sizeΛม͑ͨ৔߹ɻ (͜ͷςετͰ͸avg 16KiB -> தԝ஍1500byte)
13. Conclusion • ϙετϜʔΞͷ๏ଇͷ࣌୅ͷDC NW՝୊ • Sirius • E2EͰnsΛ࣮ݱ • tunableϨʔβɺτϙϩδɺεέδϡʔϥϨεɺ࣌ࠁಉظɺ᫔᫓੍ޚ • passive coreͷಛ௃Λ׆͔͢ • NW͝ͱ࡞Γม͑Δ͜ͱ͕Ͱ͖Δɺͱ͍͏Ϋϥ΢υ؀ڥͰ׆༻Ͱ͖Δ • ޫεΠονɺເ͕͋Δ
14. EoP