Upgrade to Pro — share decks privately, control downloads, hide ads and more …

NSX-T Data Center & Red Hat OpenShift

NSX-T Data Center & Red Hat OpenShift

Details of the NSX-T Data Center integration with Red Hat Openshift


Red Hat OpenShift

December 02, 2020


  1. 1 ©2018 VMware, Inc. NSX-T Data Center & Openshift Details

    of the NSX-T Data Center integration with Openshift
  2. 2 ©2018 VMware, Inc. Key design goals of the NSX-T

    Data Center OCP/K8S Integration Don't stand in the way of the developer ! Provide solutions to map the Kubernetes constructs to enterprise networking constructs Secure Containers, VMs and any other endpoints with overarching Firewall Policies Provide visibility & troubleshooting tools to ease the container adoption in the enterprise
  3. 3 ©2018 VMware, Inc. VMs Containers Bare Metal Servers Consistent

    Networking and Security Policy across all workloads NSX: Networking and Security for Any Workload NSX Data Center
  4. 4 ©2018 VMware, Inc. Shared T1 router for all Namespaces

    in a Cluster Network Automation for Kubernetes NSX / K8s topology T1 admin@k8s-master:~$ oc new-project foo namespace ”foo" created admin@k8s-master:~$ oc new-project bar namespace ”bar" created admin@k8s-master:~$ oc run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ oc run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Active/StandBy T0 Active/Active T0 NAT boundary EBGP/Static Physical Router 1 Physical Router 2 SNAT IP per Project is plumbed here K8s nodes K8s masters
  5. 5 ©2018 VMware, Inc. NSX / OCP topology

    T1 Active/StandBy T0 Active/Active T0 EBGP/Static Physical Router 1 Physical Router 2 LB for Service of type LB OCP compute nodes OCP Control Plane Logical Segment and subnet per OC Project vSphere, NSX-T, Storage SNAT IP per NS is plumbed here T1 per OCP Cluster Openshift 4.4
  6. 6 ©2018 VMware, Inc. OCP / NSX-T Data Center Components

    NCP is a software component provided by VMware in form of a container image, e.g. to be run as a K8s/OCP Pod. NCP is build in a modular way, so that individual adapters can be added for different CaaS and PaaS systems at some point NSX Container Plugin (NCP) NCM Infra Openshift Adapter CloudFoundry Adapter NSX Container Plugin K8S Adapter NSX Manager API Client NSX Manager Project: foo Project: bar NSX/ OCP topology OCP/K8s master etcd API-Server Scheduler
  7. 7 ©2018 VMware, Inc. Tenancy / Topology Mapping Persistent IPs

    for OCP Project With NSX-T each Tenant (OCP Project) either gets its own SNAT IP (NAT Mode), or is directly identifiable by its source subnet (No NAT Mode) Node VM OpenvSwitch mgmt IP vnic Project. Foo T1 router PAS VMs T1 router VLAN Trunk NSX-T Logical Switch Project. Bar T1 router Pods Database (VM based or Physical) Physical DC Firewall A new SNAT IP is allocated on the T1 GW for each Tenant for NAT Mode In NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source SNAT IP that is allocated to a specific Tenant. Tenant: foo Tenant: bar In No-NAT Mode, the external DC Firewall and the DB can distinguish tenant 'foo' and tenant 'bar' using the source IP Subnet that is allocated to a specific Tenant.
  8. 8 ©2018 VMware, Inc. Infrastructure Teams can pre-create Firewall rules

    in existing DC physical Firewalls to allow traffic from specific workloads in Openshift The OCP user / DevOps can deploy applications that are easily identifiable in the physical network With this feature a set of Kubernetes Workloads (Pods) can be assigned to use a specific IP or group of SNAT IPs to source their traffic from Before this feature only a SNAT IP to a OCP Project was assigned Feature Benefits Persistent SNAT IP per K8s/OCP Service Specifying the source IP Kubernetes Workloads using the K8s service Tier0 LR Corporate network DB allow – from: (App) to: (DB) Tier1 LR Openshift Project: Foo Web-Frontend Pods App Logic Pods K8S/OCP Svc for App K8S/OCP Svc for Web Namespace LS(s) SNAT App Svc Pods to: For all other Pods use projects’s SNAT IP
  9. K8s/OCP Metadata / NSX Logical Port Mapping ▶ kubectl get

    pod nsx-demo-rc-c7x65 -o yaml apiVersion: v1 kind: Pod metadata: creationTimestamp: 2018-07-25T12:05:56Z generateName: nsx-demo-rc- labels: app: nsx-demo name: nsx-demo-rc-c7x65 namespace: nsx-ujo Metadata within Kubernetes like Namespace, Pod names, Labels all get copied to the NSX Logical Port as Port Tags
  10. Pre-Created Security Groups / Firewall rules (admin rules) NSX can

    be configured to collect ports and switches in dynamic security groups based on Tags (Kubernetes Metadata) and apply Firewall rules on them Match on Port Tags Matching Pods are part of the Group Groups are used in Firewall sections as src and dst
  11. Policy support – Security per Category Environment Health-checks Admin Rules

    Application Kubernetes Network Policy Default rule: 1. Allow Cluster 2. Allow Project 3. None apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: nsx-demo-policy spec: podSelector: matchLabels: app: nsx-demo policyTypes: - Ingress ingress: - from: - ipBlock: cidr: ports: - port: 80 protocol: TCP
  12. Built-in Load Balancing NCM Infra K8s / OS Adapter CloudFoundry

    Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server HTTP and/or HTTPS traffic Server Pool 1 Server Pool 2 Rule 2 /bar/ Rule 1 /foo/ LB Service NCM Infra K8s / OS Adapter CloudFoundry Adapter Libnetwork Adapter NSX Container Plugin More… NSX Manager API Client NSX Manager K8s/ocp master etcd API-Server Scheduler Virtual Server TCP and/or UDP traffic Server Pool LB Service Offload the Openshift Router to the highly performant NSX-T LoadBalancer. It creates one single VIP for router and creates L7 rules for every Route. It also create L4 VIP for every Service of Type LoadBalancer.
  13. 13 ©2018 VMware, Inc. Distributed IDS for Containers Full visibility

    with NCP
  14. Central Visibility With NSX-T you have deep visibility and troubleshooting

  15. 15 ©2018 VMware, Inc. Inventory Dashboards Network Topology visualization in

    NSX-T UI • Network topology visualization for Pods and VMs • Search and filter based on specific IAAS or K8 constructs Feature
  16. 16 ©2018 VMware, Inc. NCP Alarms

  17. NSX-T Data Center Values for Containers Enterprise-class Networking Advanced Security

    Enhanced Operations Full Network Visibility Enterprise Support Unified VM-to- Container Networking Micro- Segmentation N S X - T Va l u e s f o r C o n t a i n e r s F e a t u r e s
  18. 19 ©2018 VMware, Inc. Key values NSX-T NCP ➢ Avoiding

    double encapsulation and bypassing node TCP/IP stack ➢ Service type Load Balancer is realized automaticallyas NSX Virtual Server ➢ Admin Firewall policyenforced per service, per cluster, or across all clusters ➢ Distributed Firewall and Distributed Intrusion Detection Systemper Pod ➢ Reliableegress source IP address per OCP Project and per Service ➢ Mix of private and routed subnets per OpenShift Project ➢ Single pane of glass for OpenShift, Kubernetes, VM , and BM workload ➢ Network Qualityof Service, Multicast Routing, VRF ➢ Service Insertion to redirect traffic between Pods to third party security appliance ➢ Visibilityand Troubleshooting tools like NSX Traceflow, IPFIX, Port Mirroring, vRNI
  19. 20 ©2018 VMware, Inc. Installation

  20. 21 ©2018 VMware, Inc. Operator is essentially a custom controller

    NSX-NCP operator watches for Network CRD Triggers NCP deployment if networkType field in the CRD is ncp. Applies tags on NSX Segment Ports Once done the operator updates the network CRD status RedHat Universal Base Image (UBI) Streamlines Installation, Updates, and Management of NCP NSX-T Network Cluster Operator NSX-NCP operator OCP/K8s master etcd API-Server Controllers Network CRD cluster NCP NCP NCP bootstrap bootstrap bootstrap bootstrap bootstrap node-agent node-agent node-agent node-agent node-agent Deployment With replica of 3 Daemon-set All nodes Daemon-set All nodes nsx-system project/namespace Schedul er
  21. 22 ©2018 VMware, Inc. Integrated with NSX Installing Openshift 4

    It is recommended to set cluster name as it is in the configmap above Network name NCP will create IP Block fin NSX for Pod networking openshift-install create manifests --dir=<installation_directory> Configure NSX parameters Set NCP image name and location https://github.com/vmware/nsx-container-plugin-operator/tree/master/deploy Copy those files to manifest folder openshift-install create ignition-configs --dir=<installation_directory> https://docs.openshift.com/container-platform/4.4/installing/installing_vsphere/installing-vsphere.html
  22. 23 ©2018 VMware, Inc. Hipster Shop Cloud-Native Microservices Demo Application

    https://github.com/GoogleCloudPlatform/microservices-demo 1 6 © 201 8 VMware, Inc. https://github.com/GoogleCloudPlatform/microservices-demo
  23. Hipster Shop User Interface Home Page Checkout Screen

  24. Hipster Shop Architecture