Data Center OCP/K8S Integration Don't stand in the way of the developer ! Provide solutions to map the Kubernetes constructs to enterprise networking constructs Secure Containers, VMs and any other endpoints with overarching Firewall Policies Provide visibility & troubleshooting tools to ease the container adoption in the enterprise
in a Cluster Network Automation for Kubernetes NSX / K8s topology 10.24.0.0/24 10.24.2.0/24 T1 admin@k8s-master:~$ oc new-project foo namespace ”foo" created admin@k8s-master:~$ oc new-project bar namespace ”bar" created admin@k8s-master:~$ oc run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ oc run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Active/StandBy T0 Active/Active T0 NAT boundary EBGP/Static Physical Router 1 Physical Router 2 SNAT IP per Project is plumbed here K8s nodes K8s masters
T1 Active/StandBy T0 Active/Active T0 EBGP/Static Physical Router 1 Physical Router 2 LB for Service of type LB OCP compute nodes OCP Control Plane Logical Segment and subnet per OC Project vSphere, NSX-T, Storage SNAT IP per NS is plumbed here T1 per OCP Cluster Openshift 4.4
NCP is a software component provided by VMware in form of a container image, e.g. to be run as a K8s/OCP Pod. NCP is build in a modular way, so that individual adapters can be added for different CaaS and PaaS systems at some point NSX Container Plugin (NCP) NCM Infra Openshift Adapter CloudFoundry Adapter NSX Container Plugin K8S Adapter NSX Manager API Client NSX Manager Project: foo Project: bar NSX/ OCP topology OCP/K8s master etcd API-Server Scheduler
for OCP Project With NSX-T each Tenant (OCP Project) either gets its own SNAT IP (NAT Mode), or is directly identifiable by its source subnet (No NAT Mode) Node VM OpenvSwitch 10.12.5.5/24 10.12.1.8/24 172.16.1.11/24 mgmt IP vnic Project. Foo T1 router PAS VMs T1 router VLAN Trunk NSX-T Logical Switch Project. Bar T1 router 172.16.1.1/24 10.12.1.1/24 10.12.5.1/24 Pods Database (VM based or Physical) Physical DC Firewall A new SNAT IP is allocated on the T1 GW for each Tenant for NAT Mode In NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source SNAT IP that is allocated to a specific Tenant. Tenant: foo Tenant: bar In No-NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source IP Subnet that is allocated to a specific Tenant.
in existing DC physical Firewalls to allow traffic from specific workloads in Openshift The OCP user / DevOps can deploy applications that are easily identifiable in the physical network With this feature a set of Kubernetes Workloads (Pods) can be assigned to use a specific IP or group of SNAT IPs to source their traffic from Before this feature only a SNAT IP to a OCP Project was assigned Feature Benefits Persistent SNAT IP per K8s/OCP Service Specifying the source IP Kubernetes Workloads using the K8s service Tier0 LR Corporate network DB allow – from: 134.247.100.10 (App) to: 134.247.200.9 (DB) Tier1 LR Openshift Project: Foo Web-Frontend Pods App Logic Pods K8S/OCP Svc for App K8S/OCP Svc for Web Namespace LS(s) SNAT App Svc Pods to: 134.247.100.10 For all other Pods use projects’s SNAT IP
pod nsx-demo-rc-c7x65 -o yaml apiVersion: v1 kind: Pod metadata: creationTimestamp: 2018-07-25T12:05:56Z generateName: nsx-demo-rc- labels: app: nsx-demo name: nsx-demo-rc-c7x65 namespace: nsx-ujo Metadata within Kubernetes like Namespace, Pod names, Labels all get copied to the NSX Logical Port as Port Tags
be configured to collect ports and switches in dynamic security groups based on Tags (Kubernetes Metadata) and apply Firewall rules on them Match on Port Tags Matching Pods are part of the Group Groups are used in Firewall sections as src and dst
Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server 10.114.209.209 HTTP and/or HTTPS traffic Server Pool 1 Server Pool 2 Rule 2 /bar/ Rule 1 /foo/ LB Service NCM Infra K8s / OS Adapter CloudFoundry Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server 10.114.209.212 TCP and/or UDP traffic Server Pool LB Service Offload the Openshift Router to the highly performant NSX-T LoadBalancer. It creates one single VIP for router and creates L7 rules for every Route. It also create L4 VIP for every Service of Type LoadBalancer.
Enhanced Operations Full Network Visibility Enterprise Support Unified VM-to- Container Networking Micro- Segmentation N S X - T Va l u e s f o r C o n t a i n e r s F e a t u r e s
double encapsulation and bypassing node TCP/IP stack ➢ Service type Load Balancer is realized automaticallyas NSX Virtual Server ➢ Admin Firewall policyenforced per service, per cluster, or across all clusters ➢ Distributed Firewall and Distributed Intrusion Detection Systemper Pod ➢ Reliableegress source IP address per OCP Project and per Service ➢ Mix of private and routed subnets per OpenShift Project ➢ Single pane of glass for OpenShift, Kubernetes, VM , and BM workload ➢ Network Qualityof Service, Multicast Routing, VRF ➢ Service Insertion to redirect traffic between Pods to third party security appliance ➢ Visibilityand Troubleshooting tools like NSX Traceflow, IPFIX, Port Mirroring, vRNI
NSX-NCP operator watches for Network CRD Triggers NCP deployment if networkType field in the CRD is ncp. Applies tags on NSX Segment Ports Once done the operator updates the network CRD status RedHat Universal Base Image (UBI) Streamlines Installation, Updates, and Management of NCP NSX-T Network Cluster Operator NSX-NCP operator OCP/K8s master etcd API-Server Controllers Network CRD cluster NCP NCP NCP bootstrap bootstrap bootstrap bootstrap bootstrap node-agent node-agent node-agent node-agent node-agent Deployment With replica of 3 Daemon-set All nodes Daemon-set All nodes nsx-system project/namespace Schedul er
It is recommended to set cluster name as it is in the configmap above Network name NCP will create IP Block fin NSX for Pod networking openshift-install create manifests --dir=<installation_directory> Configure NSX parameters Set NCP image name and location https://github.com/vmware/nsx-container-plugin-operator/tree/master/deploy Copy those files to manifest folder openshift-install create ignition-configs --dir=<installation_directory> https://docs.openshift.com/container-platform/4.4/installing/installing_vsphere/installing-vsphere.html
redhatlivestreaming
nihonbuson
line_developers_tw
hr01
kakehashi
skmkzyk
alchemy1115
oracle4engineer
kenichirokimura
musasabibyun
carta_engineering
lefred
destraynor
sergeychernyshev
chriscoyier
jmmastey
paulrobertlloyd
geoffreycrofte
kneath
gr2m
honnibal
addyosmani
zakiyama
jonrohan
