What Happened in Data Center Networking?

#OpenCORD #ONF #Edgecore #Trellis #BSN #BCF #ProjectOlympus

Transcript

1. What Happened in Data Center Networking? Phil Huang <phil_huang@edge-core.com> Open Networking Solution Engineer, Edgecore Networks Digital Ocean HsinChu, Taiwan, Aug 25, 2017
2. Phil Huang 黃秉鈞 (小飛機) • Edgecore Networks Solution Engineer • ONF CORD / Atrium • BigSwitch / Pica8 / CumulusLinux • Open Source SI • ONF CORD Ambassador • SDNDS-TW Co-Founder 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 2
3. Why Edgecore Networks? 8/26/17 3 Delivering at Scale Cumulus® Linux® Network OS ICOS Software Network OS Partner SONiC Open Hardware / Open Source Software Delivery & Support
4. Network Evolution
5. Facebook Datacenter © 2017 Edgecore Networks. All rights reserved | www.edge-core.com Ref: http://www.zdnet.com/pictures/facebooks-data-centers-worldwide-by-the-numbers-and-in-pictures/
6. Open Networking Evolution © 2017 Edgecore Networks. All rights reserved | www.edge-core.com DC Core Data Center Clos Fabric Cloud Service Providers Telecom Service Providers Enterprise & Campus IXP
7. Underlay Network Evolution for Data Center © 2017 Edgecore Networks. All rights reserved | www.edge-core.com Three-Tier Architecture Ref: https://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the-next-generation-facebook-data-center-network/ Leaf-Spine Architecture 1 3 2 4 Facebook Fabric
8. Overlay Network Evolution for Data Center © 2017 Edgecore Networks. All rights reserved | www.edge-core.com Tenant A Tenant B Tenant C Physical Network Infrastructure Abstract network view for an tenant • Decoupled from physical infra • Composed as a set of logical network resources
9. Next Gen Data Center Networking • Trend 1 • Disaggregation and White box • Trend 2 • Virtualization, Overlays, and OpenStack • Trend 3 • Two-stage Leaf-spine Clos- Fabrics with ECMP and Pods • Trend 4 • SDN, Policy, and Intent • Trend 5 • Big Data and Analytics 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 9 Ref: https://www.linux.com/blog/event/open-networking-summit/2017/3/linux-foundation-highly-relevant-data-center-networking-evolution-says-sdxcentral-report
11. Open Compute Project, OCP • Founded in 2011 • Global community for Open IT hardware • Increased Flexibility • Push for standard HW and Reduced Cost • Initial Data Center focus • Now broadening to telecom and Enterprise • Disaggregated • Fully open hardware with enabling software 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 11
12. What’s Inside Switch Box? 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 12 Silicon Hardware Driver Control / Management Software Network OS Mechanical Box
13. OCP, Networking • Fully disaggregated and open networking HW & SW • Operating System - Linux based operating systems & developer tools, and REST API’s • Fully automated configuration management & bare metal provisioning • Universal & Multi-Form Factor Switch motherboard hardware • Fully open integration & connectivity • Energy efficient power & cooling designs • Software Defined Networking (SDN) Ref: http://www.opencompute.org/wiki/Networking 13
14. Example: Wedge 100 Ref: https://code.facebook.com/posts/681382905244727/introducing-wedge-and-fboss-the-next-steps-toward-a-disaggregated-network/ Facebook Design CPU: Intel/ARM/… ASIC: Broadcom/Mellanox/… - NOS: Open Network Linux - Forwarding Agent: FBOSS - BMC: OpenBMC Hardware Software ”Switch as a Server” 14
15. OCP Networking - Software Switch Abstraction Interface, SAI − Defines API to provide a vendor-independent way of controlling forwarding elements, such as a switching ASIC, an NPU or a software switch in a uniform manner. Open Network Linux, ONL − Linux distribution (Debian) with added driver and configuration for running bare metal switches Open Optical Monitoring, OOM − Contents of optical module EEPROM accessible to python programmers. Open Network Install Environment, ONIE − Open “install environment” for bare metal network switches − ONIE enables a bare metal network switch ecosystem where end users have a choice among different network operating systems Ref: https://github.com/opencomputeproject 15
16. SDN Based Switch Models 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 16 Traditional Switch Data Plane Control Plane Applications SDN-based System Data Plane Control Plane Applications Control Protocol Open Networking Switch Data Plane Linux OS Applications
17. Big Switch Networks Big Cloud Fabric
18. Shared ”One Big Switch” Architecture § Traditional Netframe Design § Single point of management § Proprietary, Vendor Lock-in, Fixed slots Big Cloud Fabric Controller Hierarchical Control Plane 1 3 Spine Switches 2 10G/40G Backplane 4 1 3 2 4 Leaf Switches 1G/ 10G/40G Workloads § Disaggregate Netframe to one “Big Switch” § Open, Centralized management Easy to scale-out your network
19. Overview of Big Cloud Fabric Open, Economical Solution for Existing Enterprise & Service Provider Data Center Multi-Orchestrated VM/Container Single Programmatic Interface for up to 64 Rack Fabric Big Switch Controller Full Automation for Provisioning, HA / Resiliency & Management Switch Light OS Open Network Linux (ONL) Based OS for Edgecore Networks switches OCP-enabled Switch High customizable & quality switch vendor Switch Light Virtual For OpenStack/OpenShift/ Kubernetes...etc deployments
20. Distributed Logical Routing Host 1 10.50.1.2 Host 2 10.50.1.3 10.50.1.0/24 Host 3 10.50.2.2 Host 4 10.50.2.3 10.50.2.0/24 Logical Network (TENANT T1) Rack 1 Rack 2 Spine Router IP 10.50.1.1 Router IP 10.50.2.1 Segment Green 10.50.2.0/24 Segment Orange 10.50.1.0/24 Host 1 10.50.1.2 Host 3 10.50.2.2 Host 2 10.50.1.3 Host 4 10.50.2.3 Physical Network Logical Tenant Router T1
21. Test Path Visibility Network Troubleshooting Spine 3 Spine 1 Spine 2 R1L2 R1L1 R3L2 R3L1 R2L2 R2L1 Mesos-agent-2 7Xkf8don6Y Spine Leaf Mesos-agent-3 nmDh0cpymd Ethernet 26 Ethernet 49 Ethernet 5 Ethernet 2 Ethernet 50 Ethernet 19 enp5s0f1 enp8s0f0 qvo7Xkf8don6Y qvonmDh0cpymd vSwitch Source Container BIG CLOUD FABRIC CONTROLLER (CLI, GUI, API) Destination Container One-click flow trace across the fabric No box-by-box hopping
22. ONF Trellis
23. Metro Router White Box White Box White Box White Box White Box White Box White Box White Box White Box White Box White Box White Box White Box White Box Open Source SDN-based Bare-metal White Box White Box R,E,M- Access CORD Architecture 23 ONOS Controller Cluster vRouter Control XOS (Orchestrator) vSG VNF VNF VNF VNF vSG VNF VNF VNF VNF vSG VNF VNF VNF VNF OVS OVS OVS OVS OVS Residential Mobile Enterprise Underlay Control Underlay Overlay Control Overlay vOLT Control
24. What is Trellis? 24 Datacenter Leaf-Spine Fabric Underlay Virtual Network Overlay Unified SDN Control Of Underlay & Overlay ONOS Controller Cluster & Apps Trellis is the enabling Network Infrastructure for CORD Trellis Provides Common control over underlay & overlay networks, including 1. Service Composition for Tenant Networks 2. Distributed Virtual Routing 3. Optimized Delivery of Multicast Traffic Streams
25. Underlay Fabric – Open Hardware 25 White Box SDN Switch Edgecore AS6712-32x Spine Switch 32 x 40G ports downlink to leaf switches 40G QSFP+/DAC GE mgmt. AS6712-32x White Box SDN Switch Edgecore AS6712-32x Leaf Switch 24 x 40G ports downlink to servers and vOLT 8 x 40G ports uplink to different spine switches ECMP across all uplink ports GE mgmt. AS6712-32x
26. 26 BRCM ASIC OF-DPA Indigo OF Agent OF-DPA API OpenFlow 1.3 Leaf/Spine Switch Software Stack OCP Software - ONL ONIE OCP Bare Metal Hardware BRCM SDK API ONOS OCP: Open Compute Project ONL: Open Network Linux ONIE: Open Network Install Environment BRCM: Broadcom Merchant Silicon ASICs OF-DPA: OpenFlow Datapath Abstraction ONL-2.0.0-ONL-OS-DEB8-2016-12-22 OF-DPA 3.0 EA4 ONOS 1.8.9 CORD-3.0 Underlay Fabric – Software Stacks
27. L2 Unicast 27 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS
28. L2 Broadcast 28 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS
29. L3 Unicast 29 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS
30. L3 Multicast 30 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS
31. vRouter Integration 31 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS BGP Data
32. vSG Integration 32 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS Q-in-Q
33. Put everything all together… 33 Leaf1 Leaf2 Spine1 Spine2 Host1 Host2 Host3 OLT Upstream Router Quagga ONOS
34. Virtual Network Overlay 34 OVS OVS OVS OVS OVS OVS OVS OVS OVS Service VNFs & vNets Non-overlapping addresses Service B Virtual Network Tenant Green Virtual Network Overlapping address space Connectivity isolation VMs/Containers Service Y Virtual Network Tenant Blue Virtual Network Services can dynamically grow or shrink VXLAN Overlay VXLAN Overlay VXLAN Overlay Single VXLAN port in OVS
35. Trellis Summary Underlay Fabric • L2/L3 spine-leaf fabric – Bare-metal hardware + open source software • SDN control plane – No distributed protocols • Modern ASIC data plane – 1.28 Tbps switching bandwidth for each switch Virtual Network Overlay • Designed for NFV – ChainedVNFs using with best principles of cloud • Overlay Control – XOS and VTN implement service graph • OVS + VXLAN Data Plane Unified SDN Control • Common Control – Opportunity for optimized service delivery 35
36. 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 36
37. FreeRangeRouting • IP routing protocol suite for Linux and Unix platforms • Includes protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, EIGRP and RIP • Seamless integration with native Linux/Unix IP networking stacks including connecting hosts / VMs / containers • Fork from Quagga • Community driven based on • Github • Mail List • Slack 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 37 Ref: https://frrouting.org/
38. Major Change 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 38 BGP EVPN Ref: https://github.com/FRRouting/frr/wiki/Major-Changes
39. Continuous Integration 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 39 Ref: https://ci1.netdef.org/browse/FRR-FRR-470/test
40. FRRouting Testing Report 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 40 Ref: https://frrouting.org/test-results/BGP4_extended_results.pdf
41. How to Install FRRouting? 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 41 Ref: https://github.com/FRRouting/frr/tree/master/doc
42. 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 42 Ref: https://twitter.com/menotyousharp/status/859802897722335236
43. Summary • Hardware and Software Disaggregation • Unified Centralized Control and Management • Flexible • Security • Visibility • Lower CAPEX and OPEX • Deliver new service quickly and efficiency 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 43 "You disaggregate to get choice; you aggregate to get efficiencies"
44. Join Us! 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 44
45. Appendix • Software for Open Networking in the Cloud SONiC • http://azure.github.io/SONiC/ • Ecosystem momentum positions Microsoft’s Project Olympus as de facto open compute standard • https://azure.microsoft.com/en-us/blog/ecosystem-momentum-positions-microsoft-s- project-olympus-as-de-facto-open-compute-standard/ • Channel 9 - Microsoft Project Olympus • https://channel9.msdn.com/Series/Microsoft-Global-Datacenters/Microsoft-Project- Olympus • GitHub – Project Olympus • https://github.com/opencomputeproject/Project_Olympus 8/26/17 © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 45
46. Open Networking from Freedom Control Innovation © 2017 Edgecore Networks. All rights reserved | www.edge-core.com 46