Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Contents IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 2 Network Security Ingress & Routes Service Mesh Secrets Application Hardening Practices and Tools IBM Garage Cloud Native Toolkit Why security? Risk Management Frameworks Legal Requirements Kubernetes Basics Kubernetes’ Origin and Star Trek references What’s Red Hat OpenShift? Kubernetes Security Kubernetes Architecture Kubernetes Attack Vectors Container Security Identity and Access Management (IAM) Role-Based Access Control (RBAC) OpenShift Projects

Slide 3

Slide 3 text

IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 3 To help keep you entertained, There will be plenty of Star Trek GIFs! General Martok Captain Sisko Elim Garak

Slide 4

Slide 4 text

IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation #DigitalDefence Hack Presentation link will be available at

Slide 5

Slide 5 text

Legislative Requirements IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 5 Financial Services Industry Must comply and adhere to APRA standards and guidelines for technology choices Payment and Credit Cards Payment Card Industry (PCI) Data Security Standard (DSS) requirements Healthcare Industry HIPAA (Health Insurance Portability and Accountability Act) that defines security measures to protect electronic protected health information (EPHI) HITECH Act covers data requirements for protected health information (PHI), electronics health records (EHR)

Slide 6

Slide 6 text

Risk Management Frameworks IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 6 International Risk Management Frameworks • NIST SP 800-53: security and privacy controls • ISO 270001: information security management Australian Risk Management Frameworks • Australian Cyber Security Centre (ACSC)’s Information Security Manual (ISM): another source of risk management guidance • Essential Eight (E8): introduces maturity levels and provide a baseline of mitigation strategies Certification • OpenFAIR Certification: provides a taxonomy of terms, definitions, relationships and describes processes in performing risk analysis Dimensions Confidentiality Means of protecting personal privacy and proprietary information. Integrity Guarding against improper information modification or destruction Availability Ensuring timely and reliable access to and use of information

Slide 7

Slide 7 text

Why do we (app developers) care? IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 7 Broaden our horizon Understand privacy laws, data management and legislative requirements across industries Help with start-up adoption Prepare for a larger market, or just know which vendor to pick when choosing a deployment platform Commander Riker Data Lieutenant Worf

Slide 8

Slide 8 text

Star Trek (Voyager) References Seven spokes Whilst still a Borg After becoming human Seven of Nine Original code name was Project 7 Heavily inspired by Project Borg IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation

Slide 9

Slide 9 text

Kubernetes (K8s) Core Concepts IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation

Slide 10

Slide 10 text

Containers A container is the smallest compute unit CONTAINER

Slide 11

Slide 11 text

Containers are created from container images IMAGE CONTAINER Containers BINARY RUNTIME

Slide 12

Slide 12 text

Container images are stored in an image registry IMAGE CONTAINER Containers IMAGE REGISTRY RUNTIME IMAGE IMAGE IMAGE

Slide 13

Slide 13 text

Containers are wrapped in pods which are units of deployment and management POD CONTAINER 10.140.4.44 POD CONTAINER 10.15.6.55 CONTAINER Pods and Services

Slide 14

Slide 14 text

Labels are simple key-value assignments used as a grouping mechanism POD CONTAINER role: backend Pods and Services

Slide 15

Slide 15 text

Services provide internal load-balancing and service discovery across pods POD SERVICE “backend” CONTAINER 10.110.1.11 role: backend POD CONTAINER 10.120.2.22 role: backend POD CONTAINER 10.140.4.44 role: frontend role: backend Pods and Services

Slide 16

Slide 16 text

apps can talk to each other via services POD SERVICE “backend” CONTAINER 10.110.1.11 role: backend POD CONTAINER 10.120.2.22 role: backend POD CONTAINER 10.140.4.44 role: frontend role: backend Accesses via services Pods and Services

Slide 17

Slide 17 text

Add new pods to the list of endpoints load- balanced by the service POD SERVICE “backend” CONTAINER 10.110.1.11 role: backend POD CONTAINER 10.120.2.22 role: backend POD CONTAINER 10.130.3.33 role: backend POD CONTAINER 10.140.4.44 role: frontend role: backend Pods and Services

Slide 18

Slide 18 text

Routes make services accessible to clients outside the environment via real-world URLs > curl http://app-prod.mycompany.com POD SERVICE “frontend” CONTAINER role: frontend POD CONTAINER role: frontend POD CONTAINER role: frontend role: frontend ROUTE app-prod.mycompany.com Routes

Slide 19

Slide 19 text

Persistent Volume andClaims My app isstateful. 2Gi PersistentVolumeClaim 2Gi PersistentVolume POD CONTAINER 21

Slide 20

Slide 20 text

Red Hat OpenShift is Kubernetes with batteries included IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 22 OpenShift Service Mesh OpenShift Serverless odo / Dev Catalog Chief of Security from DS9 Sources: ANZ to move all traffic to re-platform internet banking by March 2021 Case Study: OpenShift at Macquarie

Slide 21

Slide 21 text

Kubernetes Security IBM Developer A/NZ / Practical Security for Kubernetes Developers / Nov 21, 2020 / © 2020 IBM Corporation 23

Slide 22

Slide 22 text

Kubernetes Cluster Worker Node(s) Kubernetes Pod(s) Master Node(s) Kubernetes Control Plane Kubernetes Architecture 24 API Server Controller Scheduler etcd Kubelet Containers Applications Attack Vector Attack Vector Risk Mitigation Access to node Compromise entire cluster SSH on secure ports, authn, authz Access to etcd Access secrets, labels and resources, understand system layout Run a private network, within a VPC, employ secure comms channels Access to K8s API server Compromise entire cluster TLS, use certificates, authn Inject control plane traffic Replace authentic K8s modules TLS, rotate credentials, patches, upgrades Access Kubelet Apply malicious actions to workloads Closed to external networks Container runtime Compromising workloads, escalate privileges to host Use hardened container runtimes Escape container to host Compromise entire cluster No root privileges, prevent escalation Intercept app traffic Compromise user data, potentially escape container to host if root is used Limit ports, persisted data are scanned for malware and viruses, good app sec hygiene Platform’s responsibilities Dev’s responsibilities Admin’s responsibilities As a platform, OpenShift handles these for you Source: OpenShift Security Guide

Slide 23

Slide 23 text

Containers in Linux IBM Developer A/NZ / © 2020 IBM Corporation 25 Source: OpenShift Security Guide All containers share the OS kernel layer from host A container runs as a minimal user layer tailored to that specific application.

Slide 24

Slide 24 text

Container Engine IBM Developer A/NZ / © 2020 IBM Corporation 26 Source: OpenShift Security Guide A Practical Introduction to Container Terminology CRI-O is a container engine. It implements the Container Runtime Interface (CRI), to use OCI. CRI-O stands for Container Runtime Interface for OpenShift. runc is the container runtime that actually create containers. It’s OCI runtime standard reference implementation.

Slide 25

Slide 25 text

Container Security IBM Developer A/NZ / © 2020 IBM Corporation 27 Sources: Configure a Security Context for a Pod or Container Managing Security Context Constraints SecurityContext defines privilege and access control settings for a Pod or Container, including but not limited to: • Permission to access an object based on UID and GID • Running as privileged or unprivileged • Give a process some additional Linux capabilities (privileges) • Filter a process’ system calls with Seccomp • AllowPrivilegeEscalation: control if a process can gain more privileges than its parents These are set with SecurityContextConstraints (SCC) can only be managed by users with cluster-admin Guidelines for application developers: • Don’t use root • Do use Red Hat Universal Base Image (UBI) as the base image – touched in Application Hardening section. Commander Data

Slide 26

Slide 26 text

Identity and Access Management (IAM) IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 28 Doctor Bashir Colonel Kira Nerys

Slide 27

Slide 27 text

Role-Based Access Control – Users IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 29 • Regular users: real people, include cluster admins and regular users. • System users: infrastructure services, components that talk to the Kubernetes APIs. kubeadmin is the first user created with Kubernetes and has super privileges. • Service accounts: specialised user objects intended for infrastructure or non-interactive workloads. Service accounts are created with a secret mounted on disk. • Virtual system users: hard-coded with a system prefix, such as system:anonymous. User User User Group Role Role Binding assigned to via Kubernetes Resources Actions defines taken against Cluster-wide Namespace-wide Source: OpenShift Security Guide

Slide 28

Slide 28 text

RBAC – Authentication Methods IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 30 OOTB, Kubernetes comes with: • Password files for individual authentication • X.509 certification verification • Token files • Requires external integration for protocols (LDAP, Kerberos, SAML and OAuth2, etc.) Red Hat OpenShift comes with an integrated OAuth2 server. User User User Group Role Role Binding assigned to via Kubernetes Resources Actions defines taken against Cluster-wide Namespace-wide Source: Kubernetes Authentication Strategies OpenShift Security Guide

Slide 29

Slide 29 text

RBAC – Roles, Actions and Role Bindings IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 31 A role is used to define rules for actions – Kubernetes verbs (list, get, edit, etc.) – on Kubernetes resources. • Cluster-wide role: actions affect cluster-wide • Namespace-wide: role and role-binding within the context of a namespace Kubernetes comes with default roles, including: • User facing roles: cluster-admin, admin (for namespaces), edit, view • Core component: control-plane component roles such as system:kube-controller-manager or system:node Role bindings attach a role to a user. User User User Group Role Role Binding assigned to via Kubernetes Resources Actions defines taken against Cluster-wide Namespace-wide Source: Kubernetes Using RBAC Authorisation

Slide 30

Slide 30 text

Projects isolate apps across environments, teams, groups and departments PAYMENT DEV PAYMENT PROD CATALOG INVENTORY ❌ ❌ ❌ POD C POD C POD C POD C POD C POD C POD C POD C POD C POD C POD C POD C Projects and Users

Slide 31

Slide 31 text

Networking Security IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 33 Commander Burnham Captain Janeway Covering networking security can get very deep very fast So we’ll cover what we need: Ingress, Routes and Service Mesh

Slide 32

Slide 32 text

Routes make services accessible to clients outside the environment via real-world URLs > curl http://app-prod.mycompany.com POD SERVICE “frontend” CONTAINER role: frontend POD CONTAINER role: frontend POD CONTAINER role: frontend role: frontend ROUTE app-prod.mycompany.com Routes Recap

Slide 33

Slide 33 text

Kubernetes Ingress vs. OpenShift Routes IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 35 Source: OpenShift Security Guide

Slide 34

Slide 34 text

OpenShift Routes Terminations IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 36 Source: IBM Cloud OpenShift 4 Route Types OpenShift Security Guide Diagram from OpenShift Blog: Self-Service End-to-End Encryption Approaches for Applications Deployed in OpenShift OpenShift Route objects support four TLS termination methods: • None/Simple/Clear for HTTP routes • Edge: your application pod is exposed on a unencrypted endpoint. The router uses a certificate from the OpenShift cluster to encrypt the connection between the client and the router. • Passthrough: router lets traffic pass through as is, this can be used for HTTP/2 and non-HTTP TLS endpoints. • Re-encrypt: the encrypted connection is terminated at the router but then re-encrypted. Red Hat also provides a example with video on how these terminations are used to implement an end-to-end encryption demo.

Slide 35

Slide 35 text

Control Plane Service Mesh - Istio IBM Developer A/NZ / Practical Security for Cloud Native Developers / © 2020 IBM Corporation 37 Istio provides: • Traffic encryption to defend against MITM attacks • Mutual TLS and Fine- grained access control policies to provide service access controls • Auditing tools to determine who did what and when Certificate Authority Authentication Policies Network Configurations Authorisation Policies istiod Pod Service A Proxy Pod Service B Proxy Ingress Egress API Server JWT + TLS JWT + TLS HTTP, gRPC, TCP Data Plane Data plane traffic, encrypted with mTLS Control plane traffic Types of transport protocol supported Source: Istio Security Concepts Introducing OpenShift Service Mesh 2.0 Service Mesh 2.0 vs Istio Adding External CA for Service Mesh 2.0

Slide 36

Slide 36 text

Secrets IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 38 Odo Chief O’Brien Doctor Bashir

Slide 37

Slide 37 text

Secrets Management IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 39 Etcd Datastore Encryption • By default, etcd stores secrets with base64 encoding – cluster admins can decode application secrets. • Additional protection can be enabled by encrypting the etcd datastore. • In OpenShift, keys are automatically created and rotated by the Kubernetes Operators. • Keys are still available as secrets via the kube API to a cluster administrator. • Use an external Key Management Services to encrypt your secrets Passing Secrets to Containers Secrets should be protect in-transit and at-rest. For applications, there are three ways of passing secrets to containers: • Build secrets into images • Use environment variables • Mount a volume into a container that contains a file with secrets It is recommended to use secret data volumes.

Slide 38

Slide 38 text

Application Hardening IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 40

Slide 39

Slide 39 text

Red Hat Universal Base Image IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 41 • A trusted supply chain attempt to resolve untrusted base image issue, by enforcing more stringent requirements, such as: • Image comes from a trusted source • Every time it’s extended, the supply chain keeps track of how it was done and scans for potential vulnerabilities or non-compliance Red Hat provides Universal Base Image (UBI) • UBIs are based on RHEL and follows RHEL’s release cycle • Freely shared and deployed on any Kubernetes platform • Language images available in Python, Node, Ruby, PHP, .NET Core, .NET 5.0, NGINX https://catalog.redhat.com/software/containers/search

Slide 40

Slide 40 text

Golden Images IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 42 Example CI/CD Base Image Hardening Pipeline using UBI on the right →

Slide 41

Slide 41 text

IBM Assets IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 43 The Cloud Native Toolkit is an open-source asset created by IBM Garage. It helps unify the developer experience for both managed Red Hat OpenShift and Kubernetes environments on IBM Cloud. • Offers production-ready code patterns • Enable delivery of production-ready solutions using a robust SDLC • Built upon key CNCF open-source technologies and tools • Enables code change, continuous integration, testing, continuous delivery, tracing and monitoring in a unified experience • Help teams get started quickly • https://cloudnativetoolkit.dev Spock IBM has a Container Software Certification Program that discusses over 150 certification requirements, best practices and standards.

Slide 42

Slide 42 text

44 Continuous Integration Deploy Observe Code Test and Coverage Build and Package DEV TEST STAGE Toolkit Components

Slide 43

Slide 43 text

IBM Developer A/NZ / Practical Security for Cloud Native Developers / Nov 21, 2020 / © 2020 IBM Corporation 45 Q&A Uhura