#5 “Experiences Evaluating DCTCP”

Experiences Evaluating DCTCP
Facebook
Linux Plumbers Conference 2018
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=Zupe1MUAAAAJ&citation_for_view=Zupe1MUAAAAJ:u-x6o8ySG0sC

Transcript

1. Research Paper Introduction #5 “Experiences Evaluating DCTCP” @cafenero_777 2019/09/10
2. $ which • Experiences Evaluating DCTCP • Lawrence Brakmo, Boris Burkov, Greg Leclercq, Murat Mugan • Facebook • Linux Plumbers Conference 2018
3. Agenda • ֓ཁͱಡ΋͏ͱͨ͠ཧ༝ • Abstract • Introduction • Overview • Intra-Rack Test Results • Inter-Rack Test Results • Conclusions • References
4. ֓ཁͱಡ΋͏ͱͨ͠ཧ༝ • ֓ཁ • େن໛DC Closӡ༻ͰͷDCTCPύϑΥʔϚϯεൺֱ݁Ռ • Linux kernel΍NIC’s fi rmware, CPU utilizationͷൺֱ • ಡ΋͏ͱͨ͠ཧ༝ • ࣮ࡍͷେن໛Closߏ੒Ͱͷੑೳൺֱ΍ϋϚΔϙΠϯτ͕஌Γ͍ͨͨΊ
5. Introduction • TCP30೥ͷྺ࢙ • fully, fairly, utilize the available bandwidth • ᫔᫓੍ޚ,Van Jacobson 1988 • ଟ͘͸loss-based algorithm • ύέϩε͢Δ·ͰΩϡʔʹஷΊΔ->(tail)ϨΠςϯγ૿Ճʂ • ύέϩεΛݕ஌͢ΔͨΊʹύέϩεΛ”଴ͭ” • ຊ຤స౗ʁ
6. Introduction (Cont.) • Congestion avoidance algorithm • ϩε͢Δ”લ”ʹ᫔᫓ݕ஌ • Ωϡʔͷ੒௕Λ᫔᫓લஹͱͯ͠ݕ஌ • TCP-Vegas, BBR: RTTΛར༻ • DCTCP: ECNΛར༻ • Reno: • ᫔᫓ݕ஌: cwnd 50%ݮʂ • RTTΑΓ΋୹͍᫔᫓΋ݕ஌ͯ͠͠·͏ • DCTCP: • cwndͷݮΒ͠ํΛ޻෉ • RTTຖʹbyteׂ߹ΛτϥοΩϯά • 100%᫔᫓->cwnd 50%ݮ • 50%᫔᫓->cwnd25%ݮ • ࣮ࡍ͸ҠಈฏۉΛ࠾༻ • 100%᫔᫓͔ͭҎલʹ᫔᫓͕ແ͚Ε͹ • -> cwnd 1/32ݮ
7. Overview • Intra-Rack Test • 3 Sender -> 1 Receiver • 1MB & 10KB RPC • DCTCPಈ࡞֬ೝ • ύέϩεݮ, tail latencyݮ, fairness • Inter-Rack Tests • 3 Worker Racks -> 3 Storage Racks • workers read storage • netestoΛ༻͍ͯղੳ • https://github.com/facebook/fbkutils/tree/master/netesto
8. Intra-Rack Tests Results • 1. ᫔᫓੍ޚʹدΒͳ͍ෆެฏੑͷ໰୊ • Server 1: 25%, Server 2: 25%, Server 3: 50% • εΠονόοϑΝઃܭʹґଘ • Work-around: Bu ff er AͷΈ࢖͏͜ͱͰެฏੑ୲อ • 2. ECNΛ࢖ͬͨ৔߹ͷϑϩʔͷެฏੑͷภΓ໰୊ • 25GbpsΛֻ͚Δɻ΋͏Ұͭͷϑϩʔ͸0.5Gbps͔͠ग़ͳ͍ • 60usͱ1.2msͷόΠϞʔμϧ෼෍ʹͳͬͯ͠·ͬͨ • NICϑΝʔϜ΢ΣΞͷ৽ػೳ͕ݪҼ -> ແޮԽ fl ow# RTT։࢝࣌ؒ (ܦա࣌ؒ) ૹ৴όΠτ਺ Ϩʔτ
9. Intra-Rack Tests Results (Cont.) • 3. DCTCPͷʹΑΔߴtail-latency໰୊ • CubicΑΓDCTCPͷํ͕஗͍ɻɻɻ • 2015೥ͷpatchʹىҼ • όάमਖ਼ޙ • 10KBͷੑೳ͕େ෯޲্ • 1MBͷํ͸૬ରతʹੑೳΛऔΒΕͨͨΊɺ
 Cubicͱൺ΂ͯେ͖͍ • RPCαΠζؒͷෆެฏੑ΋ղܾ
10. Inter-Rack Test Results • Worker͕storageΛಡΉϕϯνϚʔΫ • 3rack, 70% of FSW links • DCTCP͕ͱͯ΋ྑ͍݁Ռ • +1% CPUෛՙఔ౓ • 2rack, 99% of FSW links • DCTCPͷdiscardߴ͍ɺCPUෛՙ • ECNϚʔΫ͕63.7%
11. Inter-Rack Test Results (Cont.) • 1rack, 99.9% of FSW links • discards, ࠶ૹଟ͍ɻɻ • ߴෛՙ࣌ͷDCTCPͷޮՌ͸ݶఆతʁ • ਓ޻తͳϕϯνϚʔΫա͗ͨʁ • prodͰಉ͜͡ͱ͕ى͖Δ͔ෆ໌ • CPUෛՙݮ • ͓ͦΒ͘᫔᫓ͷͨΊcwnd͕ݮ -> ECNϚʔΩϯάݮͷͨΊɻ
12. Conclusions • খ͞Ίͳintra-rackͰ΋DCTCPͷॏཁͳൃݟɺमਖ਼Λݟ͔ͭΒΕͨ • ̏೥ӽ͠ͷόά • inter-rackͰ͸แׅతʹύέϩεݮɺlatencyվળΛ؍ଌͰ͖ͨ • DCTCPͰRPCͷlatencyվળ • ࣮ϫʔΫϩʔυͰCPUෛՙ͕໰୊ʹͳΔ͔ߋʹݕূ͕ඞཁ
13. EoP