1 ©2018 VMware, Inc. NSX-T Data Center & Openshift Details of the NSX-T Data Center integration with Openshift

2 ©2018 VMware, Inc. Key design goals of the NSX-T Data Center OCP/K8S Integration Don't stand in the way of the developer ! Provide solutions to map the Kubernetes constructs to enterprise networking constructs Secure Containers, VMs and any other endpoints with overarching Firewall Policies Provide visibility & troubleshooting tools to ease the container adoption in the enterprise

3 ©2018 VMware, Inc. VMs Containers Bare Metal Servers Consistent Networking and Security Policy across all workloads NSX: Networking and Security for Any Workload NSX Data Center

4 ©2018 VMware, Inc. Shared T1 router for all Namespaces in a Cluster Network Automation for Kubernetes NSX / K8s topology 10.24.0.0/24 10.24.2.0/24 T1 admin@k8s-master:~$ oc new-project foo namespace ”foo" created admin@k8s-master:~$ oc new-project bar namespace ”bar" created admin@k8s-master:~$ oc run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ oc run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Active/StandBy T0 Active/Active T0 NAT boundary EBGP/Static Physical Router 1 Physical Router 2 SNAT IP per Project is plumbed here K8s nodes K8s masters

5 ©2018 VMware, Inc. NSX / OCP topology 10.24.0.0/24 10.24.2.0/24 T1 Active/StandBy T0 Active/Active T0 EBGP/Static Physical Router 1 Physical Router 2 LB for Service of type LB OCP compute nodes OCP Control Plane Logical Segment and subnet per OC Project vSphere, NSX-T, Storage SNAT IP per NS is plumbed here T1 per OCP Cluster Openshift 4.4

6 ©2018 VMware, Inc. OCP / NSX-T Data Center Components NCP is a software component provided by VMware in form of a container image, e.g. to be run as a K8s/OCP Pod. NCP is build in a modular way, so that individual adapters can be added for different CaaS and PaaS systems at some point NSX Container Plugin (NCP) NCM Infra Openshift Adapter CloudFoundry Adapter NSX Container Plugin K8S Adapter NSX Manager API Client NSX Manager Project: foo Project: bar NSX/ OCP topology OCP/K8s master etcd API-Server Scheduler

7 ©2018 VMware, Inc. Tenancy / Topology Mapping Persistent IPs for OCP Project With NSX-T each Tenant (OCP Project) either gets its own SNAT IP (NAT Mode), or is directly identifiable by its source subnet (No NAT Mode) Node VM OpenvSwitch 10.12.5.5/24 10.12.1.8/24 172.16.1.11/24 mgmt IP vnic Project. Foo T1 router PAS VMs T1 router VLAN Trunk NSX-T Logical Switch Project. Bar T1 router 172.16.1.1/24 10.12.1.1/24 10.12.5.1/24 Pods Database (VM based or Physical) Physical DC Firewall A new SNAT IP is allocated on the T1 GW for each Tenant for NAT Mode In NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source SNAT IP that is allocated to a specific Tenant. Tenant: foo Tenant: bar In No-NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source IP Subnet that is allocated to a specific Tenant.

8 ©2018 VMware, Inc. Infrastructure Teams can pre-create Firewall rules in existing DC physical Firewalls to allow traffic from specific workloads in Openshift The OCP user / DevOps can deploy applications that are easily identifiable in the physical network With this feature a set of Kubernetes Workloads (Pods) can be assigned to use a specific IP or group of SNAT IPs to source their traffic from Before this feature only a SNAT IP to a OCP Project was assigned Feature Benefits Persistent SNAT IP per K8s/OCP Service Specifying the source IP Kubernetes Workloads using the K8s service Tier0 LR Corporate network DB allow – from: 134.247.100.10 (App) to: 134.247.200.9 (DB) Tier1 LR Openshift Project: Foo Web-Frontend Pods App Logic Pods K8S/OCP Svc for App K8S/OCP Svc for Web Namespace LS(s) SNAT App Svc Pods to: 134.247.100.10 For all other Pods use projects’s SNAT IP

K8s/OCP Metadata / NSX Logical Port Mapping ▶ kubectl get pod nsx-demo-rc-c7x65 -o yaml apiVersion: v1 kind: Pod metadata: creationTimestamp: 2018-07-25T12:05:56Z generateName: nsx-demo-rc- labels: app: nsx-demo name: nsx-demo-rc-c7x65 namespace: nsx-ujo Metadata within Kubernetes like Namespace, Pod names, Labels all get copied to the NSX Logical Port as Port Tags

Pre-Created Security Groups / Firewall rules (admin rules) NSX can be configured to collect ports and switches in dynamic security groups based on Tags (Kubernetes Metadata) and apply Firewall rules on them Match on Port Tags Matching Pods are part of the Group Groups are used in Firewall sections as src and dst

Policy support – Security per Category Environment Health-checks Admin Rules Application Kubernetes Network Policy Default rule: 1. Allow Cluster 2. Allow Project 3. None apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: nsx-demo-policy spec: podSelector: matchLabels: app: nsx-demo policyTypes: - Ingress ingress: - from: - ipBlock: cidr: 100.64.160.11/32 ports: - port: 80 protocol: TCP

Built-in Load Balancing NCM Infra K8s / OS Adapter CloudFoundry Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server 10.114.209.209 HTTP and/or HTTPS traffic Server Pool 1 Server Pool 2 Rule 2 /bar/ Rule 1 /foo/ LB Service NCM Infra K8s / OS Adapter CloudFoundry Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server 10.114.209.212 TCP and/or UDP traffic Server Pool LB Service Offload the Openshift Router to the highly performant NSX-T LoadBalancer. It creates one single VIP for router and creates L7 rules for every Route. It also create L4 VIP for every Service of Type LoadBalancer.

13 ©2018 VMware, Inc. Distributed IDS for Containers Full visibility with NCP

Central Visibility With NSX-T you have deep visibility and troubleshooting tools.

15 ©2018 VMware, Inc. Inventory Dashboards Network Topology visualization in NSX-T UI • Network topology visualization for Pods and VMs • Search and filter based on specific IAAS or K8 constructs Feature

16 ©2018 VMware, Inc. NCP Alarms

NSX-T Data Center Values for Containers Enterprise-class Networking Advanced Security Enhanced Operations Full Network Visibility Enterprise Support Unified VM-to- Container Networking Micro- Segmentation N S X - T Va l u e s f o r C o n t a i n e r s F e a t u r e s

19 ©2018 VMware, Inc. Key values NSX-T NCP ➢ Avoiding double encapsulation and bypassing node TCP/IP stack ➢ Service type Load Balancer is realized automaticallyas NSX Virtual Server ➢ Admin Firewall policyenforced per service, per cluster, or across all clusters ➢ Distributed Firewall and Distributed Intrusion Detection Systemper Pod ➢ Reliableegress source IP address per OCP Project and per Service ➢ Mix of private and routed subnets per OpenShift Project ➢ Single pane of glass for OpenShift, Kubernetes, VM , and BM workload ➢ Network Qualityof Service, Multicast Routing, VRF ➢ Service Insertion to redirect traffic between Pods to third party security appliance ➢ Visibilityand Troubleshooting tools like NSX Traceflow, IPFIX, Port Mirroring, vRNI

20 ©2018 VMware, Inc. Installation

21 ©2018 VMware, Inc. Operator is essentially a custom controller NSX-NCP operator watches for Network CRD Triggers NCP deployment if networkType field in the CRD is ncp. Applies tags on NSX Segment Ports Once done the operator updates the network CRD status RedHat Universal Base Image (UBI) Streamlines Installation, Updates, and Management of NCP NSX-T Network Cluster Operator NSX-NCP operator OCP/K8s master etcd API-Server Controllers Network CRD cluster NCP NCP NCP bootstrap bootstrap bootstrap bootstrap bootstrap node-agent node-agent node-agent node-agent node-agent Deployment With replica of 3 Daemon-set All nodes Daemon-set All nodes nsx-system project/namespace Schedul er

22 ©2018 VMware, Inc. Integrated with NSX Installing Openshift 4 It is recommended to set cluster name as it is in the configmap above Network name NCP will create IP Block fin NSX for Pod networking openshift-install create manifests --dir= Configure NSX parameters Set NCP image name and location https://github.com/vmware/nsx-container-plugin-operator/tree/master/deploy Copy those files to manifest folder openshift-install create ignition-configs --dir= https://docs.openshift.com/container-platform/4.4/installing/installing_vsphere/installing-vsphere.html

23 ©2018 VMware, Inc. Hipster Shop Cloud-Native Microservices Demo Application https://github.com/GoogleCloudPlatform/microservices-demo 1 6 © 201 8 VMware, Inc. https://github.com/GoogleCloudPlatform/microservices-demo

Hipster Shop User Interface Home Page Checkout Screen

Hipster Shop Architecture