BMS Network Isolation, Policy & Services Tunnels Setup FW LB Virtual Network Containers CLOUD – TELCO OR PRIVATE/PUBLIC INFRASTRUCTURE AS CODE FOR DEVOPS AND AUTOMATION FOR INFRA OPS DevOps Ops Infrastructure Operations Cookie cutter approach to deploying Physical Hardware Automated Provisioning of Hardware - Gateway Routers, Spine Switches, TORs, Servers, JBODs, DAS .. Secure the Infrastructure, not the Apps Integrated Monitoring and Alerting of all hardware components Developer Operations (DevOps) Deploy Network & Network Services when you deploy Apps Tightly Integrated Orchestration of Compute, Storage, and the Network All Networking Services – VLANs, Routing, Policy, Load Balancing, IPAM/DHCP, DNS – is provided in software and virtualized No App specific knowledge in the underlying hardware App Build & Pkg App Test & Deploy Monitoring & Analytics Network Services Automated Provisioning Monitoring & Troubleshooting Underlay Set-up Orchestrator / Controller / Tools
SYSTEM(DC/HOST/WAN/PEERING) IN GOOGLE’S NETWORK 23 Cont rol Syst em s i n Googl e’ s Net work OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA OFA Ext er nal Ext er nal i BGP | I SI S eBGP OF BwE Cont rol l er Fabri c Cont rol l er(s) Androm eda Cont rol l er Bw E Bw E Fabri c Cont rol l er(s) Androm eda Cont rol l er TE Cont rol l er B4 Cont rol l er B4 Cont rol l er CONTRAIL NORTHSTAR Espresso BGP Peering Controller eBGP
OTHER SDN CONTROLLERS Cloud Focus WAN Focus DC Switch Focus CONTRAIL NORTHSTAR APIC • Virtual Router/vSwitch Focus • VPN provisioning by BGP/XMPP • NSX use OVSDB, L2 Focus • 10,000+ Servers VM VM IP CLOS vPE vPE VM VM IP CLOS VTEP VTEP • Switch Focus • Openflow/VTEP Tunnel • “Static Route” on every Hops • 1,000+ Switch • WAN Focus • Segment Routing/RSVP-TE Tunnels • PCEP, Openflow etc • 1,00+ Nodes
CONTROLLER ANDROMEDA: VIRTUALIZE PHYSICAL NETWORK ToR 10. 1. 1/24 10. 1. 2/24 ToR 10. 1. 3/24 ToR 10. 1. 4/24 ToR VNET: 5. 4/16 VNET: 192. 168. 32/24 VNET: 10. 1. 124 Load Bal anci ng DoS ACLs VPN N FV I nt er nal Netw ork Androm eda: Googl e’ s Network Vi rtual i zati on Control l er 13 H ost V M M Androm eda Control l er G oogle In frastru ctu re Servi ces V M V M Andromeda: Similar Contrail+ Openstack/K8S Tunnel: GRE/VXLAN
AS A ROUTER VM VM VM VM VM VM VM VM Physical underlay network Forward encapsulated packets from server to server Virtual overlay network Implemented using overlay tunnels Contrail Controller or NSX OpenStack or VMware Contrail vRouter or VMware vSwitch Neutron Plugin PHYSICAL GRAPH Control Plane DATA PATH GRAPH OVERLAY GRAPH Virtual Fabric CLOS FABRIC L3 IP FABRIC 3rd Switch Virtual Line Card Vmware vSwitch Contrail vRouter SDN Router
BASED ON MPLS EVPN TECHNOLOGY VM Hypervisor with vRouter Server Tenant VRF Encapsulation Tunnel XMPP (BGP) Datacenter Route Reflector BGP Provider Network L3 VPNs for Inter-Site Connectivity Traffic segmentation in the WAN MPLS over MPLS label encapsulation tunnels BGP route signaling Contrail Virtual Networks in Datacenters Traffic segmentation in the LAN MPLS over GRE or VXLAN label encapsulation tunnels XMPP (with BGP payload) route signaling Protocols, Architecture Customer Site CE Router PE Router Customer VRF Encapsulation Tunnel OpenStack Cloud Manager Contrail Controller
FOR HARDWARE I/O ACCELERATION Large R&D budgets, deep acceleration software expertise Proprietary silicon and hardware-based acceleration Rest of the market deploying cloud technologies need off-the-shelf solutions
& KUBERNETES Searching for a term on Google, calls out to ~70 microservices before it returns your results search Google starts 2 billion containers per week vs App1 App1 App2 App2 App2 program, libraries, config files Guest OS Hypervisor Host OS Host OS App1 App1 App2 App2 App2 program, libraries, config files Docker No Guest OS in a container
INDUSTRY 1ST https://www.youtube.com/watch?v=P3dpfvdkGJ0 • Kubernetes + Contrail Demo by Lithium (Customer) July 2015 Posted on Youtube WITH “we’ve got networking multi-tenancy provided by OpenContrail” OR Automation Abstraction High-Performance Micro-segmentation Interoperable Open standards Open source Fully automated virtual network & policy provisioning Developers provision network names with labels Remove the Kubernetes Proxy, docker0/OVS and use the vRouter Micro-segment network per micro-service within multi-tenant VPCs Kubernetes, Mesos, Swarm, Docker, VMware, OpenStack, Metal… Proven networking standards for seamless routing & hybrid cloud Open community of members, users and developers
CLOUD INFRASTRUCTURE = SHARED INFRASTRUCTURE Applications compete for shared resources resulting in unpredictable performance! The Hypervisor only virtualizes the CPU … 1 2 3 1 2 3 1 2 3 1 2 3 No Control over where applications execute!
MONITORING TECHNOLOGIES ARE SLOW! INEFFICIENT! INEFFECTIVE! The Monitoring infrastructure is more complex than the infrastructure that is being monitored Hadoop Cluster for storing & analyzing metrics Compute & Storage Infrastructure Metrics Signals 6 minutes Consumed by humans for after-the-fact analysis Too slow to influence orchestration Your OpenStack/Kubernetes Management is running blind Inefficient request-response
DISTRIBUTED STREAM ANALYSIS & OPTIMIZATION FAST! EFFICIENT! RESPONSIVE! 9 Actionable : Real-time optimizations 9 Local Optimizations for shared resources 9 AppFormix Orchestration delivers the the best workload placement to ensure high performance & high reliability! 9 Analyze more metrics for better prediction of failures 9 More metrics than any other solution 9 Solution scales with your Infrastructure 9 No central choke-point! 9 Extensible 9 Use Nagios style plugins to add your own metrics Compute & Storage Infrastructure Signals policy policy policy policy NoSQL Data Streams Message Bus Capacity Planning Reporting & Alarming Resource Orchestration Distributed Data Platform
OF CLOUD DEPLOYMENT MANY OTT CHOOSE CONTROLLER LESS Controller-less Data Centers Controller enabled Data Centers VLANS VLANS FINANCE HR MARKETIN G Firewalls Load-Balancer Physical Servers Local Hard Drives API Network Director VIRTUALIZED HR MARKETING FINANCE API BGP/ NETCONF Netruon Plugin API API Network Director
PLUGIN 1. Config Switch settings – username / password / ip 2. Define OVSwitch NIC port 3. Define Server –Switchport mapping. Can be done automatically via LLDP Netruon Plugin
PLUGIN Netruon Plugin • Current virtual network topology 1. Routing Instance are created on the VCF by neutron plugin 2. IP address are created on the irb vlan interface by neutron plugin
IP FABRIC USING BGP http://www.juniper.net/us/en/local/pdf/whitepapers/2000565-en.pdf draft-ietf-rtgwg-bgp-routing-large-dc AS 65001 AS 65100 EBGP • BFD • Multipath • Fast External Failover • Export servers • Export loopbacks AS per Cluster /31 per link Massive Data Center /21 per Cluster And /32 TOR system IP AS 65100 AS 64100 AS 64101 AS 64102 AS 64103 AS 64200 AS 64201 AS 64202 AS 64203 /32 per TOR AS per TOR Automated configuration
PROBLEM S S S S L L L L L L L L L L L L L L L L 3-Stage Clos Spine and Leaf L L L L L L L L A A A A A A A A A A A A A A A A S S S S EVPN-VXLAN Fabric PODs + Interface Assignments IP addressing Loopback addressing Subnet masks PTP Links Server VLAN RVI assignment Control Plane BGP ASN assignments BGP import policy BGP export policy BGP peer group design BGP next-hop self VXLAN Configuration VTEPs VNIDs Routing instances BUM traffic
SPACE NETWORK DIRECTOR https://github.com/Juniper/OpenClos Build • 3-stage Clos topology • 5-stage Clos topolog Monitor • Optics • BGP sessions • RIB and FIB • Queues Maintain • Add switches • Replace switches • Remove switches + S S S S L L L L L L L L OPENCONFIG
N PFE PFE uKernel Routing Engine Line Card 1 Application Network Element Configuration: NETCONF, CLI Provision Sensors In-band telemetry information Queries Data Collector Query Engine Database Forwarding Engine Forwarding Engine Juniper or a 3rd party collector uKernel RPD, Other daemons Telemetry manager JUNOS TELEMETRY INTERFACE INDUSTRY 1ST EDGE ROUTER INTEGRATED TELEMETRY Queue Depth Resource utilization Loss & Delay measurement Results can be visualized using customer applications or partner apps like SevOne and IBM’s The Now Factory NORTHSTAR CONTRAIL Controller New Applications Distributed design for Internet scale Push versus Pull model Thousands of monitoring objects Sub-second resolution Google Protocol buffer for export Provides real-time data to performance monitoring, analytics, and network optimization tools Dynamic Resource Provisioning Router Resource Pooling High Scale/Standardization Deep Analytic
INSIGHTS) ARCHITECT Pre-configured with all tools and with default dashboard .. Send it data, it will graph it Pre defined template for Cli commands Packaged with Docker, Easy to deploy, easy to maintain Can run on server, on laptop .. on the device itself Proof of concept Accept multiple sources of Data (Netconf/ JTI / Analyticsd / Network Agent) Can send to multiple collector / database Database Graphical Interface Data Streaming Collector MX Junos Telemetry (Jvision) Insight Analytics (Analyticsd) Data Collection Agent PTX QFX/EX SRX Netconf Shell PYEZ fluent-plugin-juniper-telemetry Talk to Juniper TME Damien Garros
– Architecture and Strategy All!of!the!above!are!properties!that!tend!to!simplify!the!design.!!What!makes!things!interesting!are!the!next!two! requirements:! · There!are!very!large!volumes!of!traffic!between!changing!pairs!of!end>points!(depending!on!the!requirements! of!the!applications!at!a!given!time),!necessitating!pro>active!bandwidth!provisioning!driven!by!the!requirements! of!the!applications.! · Bandwidth!must!be!utilized!as!close!to!100%!as!possible! The!figure!below!shows!the!footprint!of!the!Google!back>end!network,!based!on!some!public!announcements!they!have! made.!! Figure$ 3:$ The$ Google$ G<Scale$ (back<end)$ network$ Originally!Google’s!back>end!network!was!implemented!similarly!to!the!front!network!–!using!general>purpose!routers! GOOGLE B4 WAN NETWORK Site A Data Center OFA Switch OFA Switch OFA Switch OFA Switch Data Center Site B Servers RAP TE-AGENT OFC paxos Quagga Quagga Quagga Paxos Site B Controllers Servers Switch Hardware iBGP, ISIS eBGP Gateway Gateway Central TE Servers Central TE Servers Global TE 94.00% 95.00% 96.00% 97.00% 98.00% 99.00% 100.00% 101.00% Q1 Q2 Q3 Q4 Floor Avg Peak
WAN NETWORK DETAIL FORWARDING GRE tunnel use LPM and ACL(openflow) result to loadbalance(TE) Fi gure :M ul ti path W A N Forw ardi ng Exam pl e. (a) (b) Fi gure :Syst em transi ti on from one path assi gnm ent(a)to another(b Fi gure :M ul ti path W A N Forw ardi ng Exam pl e. Fi gure :Layeri ng tra c engi neeri ng on top ofshortestpath f or- w ardi ng i n an encap sw i tch. the packetbased on a tabl e entry pre-con gured by TE.A er de- capsul ati on,thesw i tch f orw ardsto thedesti nati on based on thei n- nerpacketheader,usi ng LongestPre x M at ch (LPM )entri es(from c c 5 O g p n i m G r o
WAN BANDWIDTH ENFORCER TRAFFIC ENGINEERING MOST SUCCESSFUL WAN SDN, NOT RSVP-TE B4 WAN SDN not for Carrier WAN Network only for DCI, Host/Switch/Cluster enforcer can control e- ng w - Fi gure :Bw E A rchi tect ure. chy (j ob-f gsand task-f gs)equal l y (no w ei ghts)based on thei r esti m at ed dem ands.
DCI NETWORK, 95% LINK UTILIZATION Inter-DC WAN: A critical, expensive resource Hong Kong Seoul Seattle Los Angeles New York Miami Dublin Barcelona BGP Conduit BGP Conduit Topology Service Existing SWAN Components TE Engine (SR-Aware) Service Agent - Traffic Demand - Type of Traffic - Rate Limiting Bandwidth Arbitrator - Demand Aggregation Centralized TE Engine Controller DC DC MPLS Prefix Map
morning the BGP Peer Link Controller/RR find a tunnel from Ingress to ASBR Controller/RR based on certain rules to select ASBR E2: BGP EPE/IPE DESIGN PHILOSOPHY How ASBR identify a Peer Per Peer /32 address per label Install the MPLS Label POP for every Peer When ASBR received different label and send traffic to specific Peer How Ingress mapping traffic to ASBR/Peer Ingress push tunnel label to ASBR Ingress push BGP-LU label
with Label Stacks PE1 1 Service Request 2 ASBR BGP BGP-LU Session between Controller/Router BGP LU carrier the label stack for SR/LSP BGP-LU carrier the Label stack for LSP + VPN Service BGP-LS get the network information TEDB information with label send back to Controller draft-gredler-idr-bgp-ls-segment-routing-ext-xx.txt BGP is the only protocol for Service and Tunnel QPPB/BGP FlowSpec With additional Openflow/PBR BGP-LU 101 103 114 80001 PayLoad Junos 15.1F5 bespalov@CentOS-1 ~/exabgp-3.4.16/sbin>cat ~bespalov/config/exabgp neighbor 192.168.255.12 { local-address 192.168.255.2; peer-as 65000; local-as 65000; family { ipv4 nlri-mpls; } static { route 10.255.255.8/32 { next-hop 10.0.0.2; label [ 800005 800007 800006 800008 ]; }} Example from ExaBGP
{ Jupiter, B4, Andromeda, Espresso}. · First, it allows us to dynamically choose from where to serve individual users based on measurements of how end-to-end network connections are performing in real time. · Second, we separate the logic and control of traffic management from the confines of individual router “boxes.” Rather than relying on thousands of individual routers to manage and learn from packet streams, we push the functionality to a distributed system that extracts the aggregate information
Office Home or SOHO HQ Service Edge Router Service Edge Router Agg Agg DC NETCONF for VPN vBNG vEPC leaf leaf leaf leaf BGP SR TE for Tunnel BGP-LS for Infor Peering Elastic Edge Controller MPLS/VXLAN Stitching VXLAN MPLS -> SPRING MPLS -> VXLAN/SPRING MPLS SPRING VXLAN EVPN Elastic Edge Controller
FABRIC FOR DIGITAL COHESION Mobile Edge Broadband Edge Business Edge Smart City Ubiquitous Connectivity Real-time and Low Latency Non Real-time DDC: Distributed Data Center CDC: Centralized Data Center Big Data Analysis Structured Data Unstructured Data Junos Fusion Virtual Chassis Virtual Chassis Fabric Qfabric Open Clos and L2/3 Fabrics
cloudbala
masatokinugawa
cuebic9bic
rinchsan
faithandbrave
koudaiii
watany
mineo_matsuya
sansan33
higaki_program
kakehashi
sugamasao
tsukuha
skipperchong
marcduiker
sonatard
tammyeverts
destraynor
chriscoyier
hannesfritz
eileencodes
deanohume
sergeychernyshev
lynnandtonic
jakevdp
