Kubernetes and service mesh applications

This presentation gives audiences a broad viewpoint from old to modern architecture. How Kubernetes and service mesh (istio) can help developers in those missions:

- Explain from traditional to modern architecture. The role of Kubernetes in modern architecture.
- Build basic k8s components from the ground up with illustrations: Pod; Node; Service; ReplicaSet; Deployment; Namespace; Ingress ...
- Kubernetes under the developer viewpoint: setup, write and deploy a k8s application.
- Kubernetes advanced concepts: master node architecture, how does the auto-scale for pods/nodes work, Kubernetes networking model.
- Discuss microservice challenges. The role of the service mesh in the microservice ecosystem.
- Introduce Envoy, istio and their application in the service mesh.

Transcript

1. Kubernetes & service mesh Huynh Quang Thao Trusting Social
2. What is the Kubernetes
3. Traditional to Modern
4. Traditional to Modern
5. Containers are great • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable •Isolation. •Immutability •EfBicient resource utilization. •Lightweight •Portable
6. Containers are great but… • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable •Isolation. •Immutability •EfBicient resource utilization. •Lightweight •Portable But … •Dozens, even thousands of containers over time. •How to manage/deploy/connected/updated ? •Integrate and orchestrate these modular parts •Provide communication across a cluster •Make them fault tolerant
7. Docker orchestration • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
8. Kubernetes comes to help • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Desire state management. • Resilience. • Automate roll-out and roll-back. • Elastic. • Cloud-agnostic. • EfBicient resource management. • Abstract infrastructure layer. •
9. Core Components
10. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container • The smallest and simplest unit in the k8s object model
11. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container • The smallest and simplest unit in the k8s object model • Each pod will have a unique internal IP address. 10.1.1.1
12. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container • The smallest and simplest unit in the k8s object model • Each pod will have a unique internal IP address. • There are many containers in one single pod. 10.1.1.1 Container
13. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container • The smallest and simplest unit in the k8s object model • Each pod will have a unique internal IP address. • There are many containers in one single pod. • Containers in pods share network namespace, volume 10.1.1.1 Container 3000 3306 10.1.1.1:3000 10.1.1.1:3306
14. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container • The smallest and simplest unit in the k8s object model • Each pod will have a unique internal IP address. • There are many containers in one single pod. • Containers in pods share network namespace, volume 10.1.1.1 Container 3000 3000
15. Pod • The smallest and simplest unit in the k8s object model • Each pod will have a unique internal IP address. • There are many containers in one single pod. • Containers in pods share network namespace, volume • Pods are logical hosts and behave much like VMs • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306
16. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system.
17. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system. • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
18. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system. • Containers in the same pod are guaranteed schedule together • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
19. Pod • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3306 5432 • There are many pods in the system. • Containers in the same pod are guaranteed schedule together ==> The smallest and simplest unit in the k8s object model • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 8000 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
20. Node • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system.
21. Node • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system. • Many pods will be inside one node.
22. Node • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • There are many pods in the system. • Many pods will be inside one node. • A node maybe a physical VM or physical machine. Compute Engine Droplet EC2 17.5.7.8
23. • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Might be same type of pods in the same node
24. • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Might be same type of pods in the same node • Different types of pods in the same node
25. Node • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 Depends on: • node’s resource availability • pod’s resource requirements (explain later)
26. Replica Set replicas: 3 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306
27. Replica Set replicas: 3 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306
28. Replica Set replicas: 3 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 3000 3306
29. Deployment replicas: 2 template: Manages Replica Set state transitions Deployment replicas: 2 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306
30. Deployment replicas: 3 template: Deployment • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 replicas: 3 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 replicas: 2 template: Declarative over Imperative
31. Deployment replicas: 2 template: Manages Replica Set state transitions Deployment replicas: 3 template: replicas: 2 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 replicas: 2 template:
32. Deployment replicas: 2 template: Update can be: • rolling update • recreate Deployment replicas: 2 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 5432 replicas: 2 template: replicas: 3 template:
33. Deployment replicas: 2 template: Manages Replica Set state transitions Deployment replicas: 3 template: replicas: 2 template: • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 Rollback replicas: 2 template:
34. Pod ... again • Each pod has its own IP address • Pods are expected to be stateless • New Pod will have the new IP Node: 8GB RAM Node: 8GB RAM 4GB RAM 5 GB RAM 2 GB RAM Node: 8GB RAM Node: 8GB RAM 4GB RAM 5 GB RAM 2 GB RAM 5 GB RAM
35. Service Login Service 10.10.2.3 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Each service is assigned unique IP which is unchanged • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306
36. Service Login Service 10.10.2.3 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 app=credit version=v1 app=fraud version=v1 app=fraud version=v1 • Each service is assigned unique IP which is unchanged • Each pod is assigned some labels • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 app=fraud version=v1
37. Service Login Service 10.10.2.3 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 app=credit version=v1 app=fraud version=v1 app=fraud version=v1 match label app=fraud • Each service is assigned unique IP which is unchanged • Each pod is assigned some labels • Service rules using label matching • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 app=fraud version=v1
38. Service Login Service 10.10.2.3 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 app=credit version=v1 app=fraud version=v1 app=fraud version=v1 match label app=fraud • Each service is assigned unique IP which is unchanged • Each pod is assigned some labels • Service rules using label matching • Randomly select a single pod that matches the rule • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 app=fraud version=v1
39. Service Login Service 10.10.2.3 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 app=credit version=v1 app=fraud version=v1 app=fraud version=v1 match label app=fraud • Each service is assigned unique IP which is unchanged • Each pod is assigned some labels • Service rules using label matching • Randomly select a single pod that matches the rule • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 app=fraud version=v1
40. General pictures label matching label matching label matching service Deployment label a label b ReplicaSet label a label b Pods label a label b Services label a label b
41. DaemonSets • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
42. DaemonSets • Ensure nodes run a copy of a Pod • Example: • a log collector • a resource monitor • Kube proxy • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 116.109.65.208 116.109.65.210 Container 10.1.1.3 9600 Container 10.1.1.5 9600
43. StatefulSets 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 Name=web-abc • Every pod has a unique name web-abc.default.svc.cluster.local.
44. StatefulSets 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.5 Container 8000 5432 Name=web-xyz • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 Name=web-abc • Every pod has a unique name • And change every time redeploy • KaBka needs a stable hostname when deploying a cluster. web-xyz.default.svc.cluster.local.
45. StatefulSets 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.5 Container 8000 5432 Name=web-xyz • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 Name=web-abc • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 ID=1 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 ID=2 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 ID=3 • Every pod has a unique name • And change every time redeploy • KaBka needs a stable hostname when deploying a cluster. StafulSets: • Keeps unique IDs in replicas
46. StatefulSets 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.5 Container 8000 5432 Name=web-xyz • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 Name=web-abc • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 ID=1 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 ID=2 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 ID=3 • Every pod has a unique name • And change every time redeploy • KaBka needs a stable hostname when deploying a cluster. StafulSets: • Keeps unique IDs in replicas • db-01.foo.default.svc.cluster.local. Name=db-01 Name=db-02 Name=db-03
47. StatefulSets 116.109.65.208 116.109.65.210 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.5 Container 8000 5432 Name=web-xyz • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 Name=web-abc • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 ID=1 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 ID=2 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 ID=3 • Every pod has a unique name • And change every time redeploy • KaBka needs a stable hostname when deploying a cluster. StafulSets: • Keeps unique IDs in replicas • db-01.foo.default.svc.cluster.local. • deployed in sequential order and terminated in reverse ordinal order. (optional) • KaBka, MySQL, Redis, ZooKeeper ... : need unique, persistent identities and stable hostnames. Name=db-01 Name=db-02 Name=db-03
48. Jobs • Perform a single completable task. • Useful for ad hoc tasks, where it’s crucial that the task finishes properly. • There are 2 types: batch job and cron job.
49. General pictures label matching label matching label matching service Deployment label a label b ReplicaSet label a label b DaemonSet label a label b StatefulSet label a label b Job label a label b Pods label a label b Services label a label b
50. Data - ConBigMap - Secret key1=value1 name=prod-env key2=value2 key3=value3 • Configmap is a map containing key/value pairs • The values ranging from short literals to full config files.
51. Data - ConBigMap - Secret key1=value1 name=prod-env key2=value2 key3=value3 key1=B64(v1) name=db-passwd key2=B64(v2) key3=B64(v3) • Configmap is a map containing key/value pairs • The values ranging from short literals to full config files. • Secret: similar to ConfigMap • let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys • Currently no encryption. just encoding.
52. Data - Volume • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 /data /output /logs Volume • Using volume to store / share data between containers
53. Data - Volume • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 /data /output /logs Volume • Using volume to store / share data between containers • Volume can be built from secret / configmap
54. Data - Volume • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.3 Container 3000 3306 /data /output /logs Volume Persistent Volume Claim 30GB Persistent Volume GCEPersistentDisk 50GB Persistent Volume AzureDisk 20GB Persistent Volume CephFS 20GB • Using volume to store / share data between containers • Volume can be built from secret / configmap • Or from the persistent disk
55. General pictures label matching label matching label matching service Deployment label a label b ReplicaSet label a label b DaemonSet label a label b StatefulSet label a label b Job label a label b Pods label a label b Con]igMap Secret PersistentVolume name Services label a label b
56. Namespace • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
57. Namespace • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432
58. Namespace • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource Container Container 3000 3306 • Isolation. • Immutability • Efficient resource Container Container 8000 5432 Staging testing production
59. Namespace • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.4 Container 8000 5432 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 Container 3000 3306 • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.2 Container 3000 3306 • Isolation. • Immutability • Efficient resource Container Container 3000 3306 • Isolation. • Immutability • Efficient resource Container Container 8000 5432 namespace=Staging namespace=testing namespace=production • Namespace is a virtual cluster or a logical cluster • organized projects (telco / avay) • organize environment (testing/ staging) • Some important namespaces: default / kube-system
60. General pictures label matching label matching label matching service Deployment label a label b ReplicaSet label a label b DaemonSet label a label b StatefulSet label a label b Job label a label b namespace: telco-staging Pods label a label b Con]igMap Secret PersistentVolume name Services label a label b
61. Ingress • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • How to expose multiple service? • Service is L4 proxy. • SSL? node_ip:port/url -> internal_service_ip:node_port/url
62. Ingress • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • How to expose multiple service? • Service is L4 proxy. • SSL? Ingress • name-based virtual hosting • Load-balancing • SSL terminating • L7 proxy
63. General pictures label matching label matching label matching service Deployment label a label b ReplicaSet label a label b DaemonSet label a label b StatefulSet label a label b Job label a label b namespace: telco-staging Pods label a label b Con]igMap Secret PersistentVolume name Ingress Services label a label b example.com/login
64. bring everything together
65. Kubernetes for developers
66. Pod template • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • 1 Descriptor conforms to version v1 of Kubernetes API • 2 You’re describing a pod. • 3 The name of the pod • 4 Container image to create the container from • 5 Name of the container • 6 The port the app is listening on
67. Organize Pod with label • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
68. Readiness probes • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable readiness probes to know when a Container is ready to start accepting traffic. 3 mechanisms: • HTTP GET probe. • TCP socket probe. • Exec probe.
69. Readiness probes • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
70. Liveness probes • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable liveness probes to know when to restart the container. 3 mechanisms: • HTTP GET probe. • TCP socket probe. • Exec probe.
71. Liveness probes • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
72. Pod Lifecycle • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
73. Lifecycle • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
74. Init containers • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable init containers, which are run before the app containers are started. Init containers are exactly like regular containers, except: • Init containers always run to completion. • Each init container must complete successfully before the next one starts. Init containers can contain utilities or custom code for setup that are not present in an app image. (e.g.: sed, awk, python ...) --> The application image builder and deployer roles can work independently without the need to jointly build a single app image.
75. init containers • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
76. init containers • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
77. Resources • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Compute resources are measurable quantities that can be requested, allocated, and consumed
78. Resources • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
79. Service 4 types: • ClusterIP: exposes the service on a cluster-internal IP. • NodePort: exposes service on each Node's IP at the static port. ClusterIP will automatically create. • LoadBalancer: expose the service externally using a cloud provider’s load balancer. • ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com) 116.109.65.208 service 10.1.1.1 Container 31452 8888:3306 3306 116.109.65.208:31452/login NodePort
80. ConBigMap Configmap is a map containing key/value pairs with the values ranging from short literals to full config files.
81. Secret • let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys • Currently no encryption. just encoding.
82. Basic command lines kubectl <action> <resource_type> <resource_name> kubectl get pods kubectl get pods fraud-score-service-d45h3dh09 kubectl get services kubectl delete services fraud-score-service action: get / delete / edit / describe resource_type: pods / deployment / service / ingress / sercret / conBigmaps / .... https://kubernetes.io/docs/reference/kubectl/cheatsheet/ Important ]lags: [-A] All namespace [--show-labels] include the labels [-l ] selector. e.g: -l='app=web,version=1' [-f] Bile name. [-o] output format. can be yaml, json or wide (view more information)
83. Repository https://github.com/tsocial/telco_integration/pull/24 Contains: • All resources yaml • All scripts to generate config files / deploy to the k8s cluster. • Some configurations to enable the log infrastructure / service mesh Implement Kubernetes infrastructure PR
84. Repository • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
85. Development environment
86. minikube Pros: • Cheap • Easy to get start Cons: • Very different from real production environment. • Local machine
87. Cloud • DigitalOcean • Google Cloud • Amazon Pros: easy to manage. Cons: prices ?
88. Manual - kubeadm • Network driver ? • High availability ? • Making sure .... k8s is running ?
89. k3s • Kubernetes for IOT
90. Deploying app to k8s
91. Basic steps • Dockerize. • Write deployment/service. • Define configmap or variable env. • Resource usage. • Liveness/Ready probe.
92. Basic steps • Dockerize. • Write deployment/service. • Define configmap or variable env. • Resource usage. • Liveness/Ready probe. • How to structure application into pods? Multiple container in 1 pod or multiple pod? • How to integrate with other service? • Does it need to communicate with outside? • Does it need stateful?
93. Basic steps kubectl apply -f resource.yaml API Server 10.1.1.1 resource.yaml fraud_score: image: docker-registry.trustingsocial.com/telco_platform/fraud_score_service:${FRAUD_SCORE_TAG} env_file: .test.env
94. Basic steps kubectl apply -f resource.yaml API Server 10.1.1.1 resource.yaml fraud_score: image: docker-registry.trustingsocial.com/telco_platform/fraud_score_service:${FRAUD_SCORE_TAG} env_file: .test.env fraud_score: image: docker-registry.trustingsocial.com/telco_platform/fraud_score_service:master.fa38ac4
95. Helm Pros: • Package manager for Kubernetes • Life cycle management: create / install / Upgrade / rollback / delete / status / versioning • Helm Charts help you define, install, and upgrade even the most complex Kubernetes application. • Helm repository https://github.com/helm/charts/tree/master/stable Helm 3 Helm 2
96. Helm Pros: • Package manager for Kubernetes • Life cycle management: create / install / Upgrade / rollback / delete / status / versioning • Helm Charts help you define, install, and upgrade even the most complex Kubernetes application. • Helm repository https://github.com/helm/charts/tree/master/stable Cons: • Easy to use, but hard to write / debug • Yaml format is not compatible with k8s resource format. Helm 3 Helm 2
97. Kustomization • Customization of kubernetes YAML configurations + kustomize build ${directory} > output.yaml
98. Advance
99. The architecture • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
100. The API Server • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable API server is the central component used by all other components and by clients, such as kubectl. It provides a CRUD (Create, Read, Update, Delete) interface for querying and modifying the cluster state over a RESTful API. It stores that state in etcd.
101. The API Server • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
102. The Scheduler • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Filtering the list of all nodes to obtain a list of acceptable nodes the pod can be scheduled to • Prioritizing the acceptable nodes and choosing the best one. • If multiple nodes have the highest score, round-robin is used to ensure pods are deployed across all of them evenly.
103. The controller manager • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • ReplicaSet, DaemonSet, and Job controllers. • Deployment controller. • StatefulSet controller. • Node controller. • Service controller. • Others Controllers do many different things, but they all watch the API server for changes to resources (Deployments, Services, and so on) and perform operations for each change, whether it’s a creation of a new object or an update or deletion of an existing object.
104. kubelet • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
105. kube proxy • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • The iptables proxy mode doesn’t load balance—it selects pods randomly. • When only a few clients use a service, they may not be spread evenly across pods. kube-proxy makes sure connections to the service IP and port end up at one of the pods backing that service
106. How they cooperate • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
107. Horizontal Pod Autoscaler • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
108. Cluster Autoscaler • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
109. Auto scale interact together • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
110. Kubernetes networking • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • all Pods can communicate with all other Pods without using network address translation (NAT). • all Nodes can communicate with all Pods without NAT. • the IP that a Pod sees itself as is the same IP that others see it as. 1. Container-to-Container networking 2. Pod-to-Pod networking 3. Pod-to-Service networking 4. Internet-to-Service networking https://sookocheff.com/post/kubernetes/understanding-kubernetes-networking-model/ Problems
111. Pod-to-Pod same node • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
112. Pod-to-Pod different nodes • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
113. micro-service challenges
114. Microservices • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Technology Heterogeneity • Resilience • Scaling • Ease of deployment • Optimizing for Replaceability Benefits:
115. Microservices • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem. This sounds great! But it turns out it’s really, really hard.
116. But the network is hard • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Communication between services • Load Balance • Discovery Service • Observability • Distributed tracing • Logs • Monitoring • Fault Tolerance • Circuit breaker • Retry mechanism
117. Communication between services • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Client side load balancer Server side load balancer yet another highly available system component that you need to set up and manage couples the client with the service registry. make application speciBic load balancing decisions (e.g.hashing consistently) eliminates the need to implement discovery logic
118. Client Libraries: The First Service Meshes? • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • The restriction use of multiple language- specific frameworks and/or application servers to run them. • Complexity when upgrade version library. • Forward compatibility and Backward compatibility
119. Observability • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Logging • Metrics • Tracing https://www.slideshare.net/hqt/observability-and-its-application How well do you really understand what’s going on in these environments?
120. Network failure • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable With our services communicating with numerous external resources, failures can be caused by: • Networking issues • System overload • Resource starvation (e.g. out of memory) • Bad deployment/conBiguration •
121. Service Mesh
122. Service Mesh • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often using a sidecar proxy. (Wikipedia)
123. Service Mesh • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often using a sidecar proxy. (Wikipedia) • Service engineer focus only on service business. • Don’t restrict to any language/framework.
124. Control plane vs Data plane • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Data Plane: • Touches every packet/request in the system. Control Plane: • Does not touch any packet/request in the system.
125. Control plane vs Data plane • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Data Plane: • Touches every packet/request in the system. • Service discovery • Health checking • Routing. • Observability. • Authentication/authorization. • Load balancing Control Plane: • Does not touch any packet/request in the system. • Provide policy. • Provide configuration. • Unifies telemetry collection.
126. istio
127. Envoy Proxy • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
128. Envoy Proxy • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Out of process architecture: Let’s do a lot of really hard stuff in one place • Modern C++11 code base: Fast and productive. • L3/L4 Bilter architecture: A TCP proxy at its core. HTTP; MongoDB; Redis; TCP rate limiter • HTTP L7 Bilter architecture. • HTTP/2 and GRPC proxy. • Service discovery and active health checking. • Advanced load balancing: Retry, timeouts, circuit breaking, rate limiting, shadowing, etc • Observability: stats, logging, and tracing. • Edge proxy: routing and TLS. • ...
129. Envoy How ? • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 3000 17.5.7.8 IPTables Service (1) 10.1.1.1:3000 (2) 10.1.1.1:3000
130. Envoy How ? • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 3000 17.5.7.8 IPTables Service Envoy 15001 protocol source destination rule tcp anywhere :8888 :8888 tcp anywhere :8080 :8080 tcp anywhere anywhere redirect port 15001
131. Envoy How ? • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Container 10.1.1.1 3000 17.5.7.8 IPTables Service Envoy 15001 protocol source destination rule tcp anywhere :8888 :8888 tcp anywhere :8080 :8080 tcp anywhere anywhere redirect port 15001 (1) 10.1.1.1:3000 (2) 10.1.1.1:15001 (3) 10.1.1.1:3000
132. Envoy - Observability • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Having all SoA trafBic transit through Envoy gives us a single place where we can: • Produce consistent statistics for every hop • Create and propagate a stable request ID • Consistent logging • Distributed tracing
133. Envoy - Advanced load balancer • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Different service discovery types • Zone aware least request load balancing. • Dynamic stats: Per zone, canary speciBic stats, etc. • Circuit breaking: Max connections, requests, and retries. • Rate limiting: Integration with global rate limit service. • Shadowing: Fork trafBic to a test cluster. • Retries: HTTP router has built in retry capability with different policies. • Timeouts: Both “outer” (including all retries) and “inner” (per try) timeouts. • Outlier detection: Consecutive 5xx • Deploy control: Blue/green, canary, etc
134. istio • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Data plane: Envoy proxy as Sidecar • Control plane: • Pilot • Galley • Citadel • Mixer
135. istio • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Data plane: Envoy proxy as Sidecar • Control plane: • Pilot • Galley • Citadel • Mixer Functionality: • Fine-grained control traffic • A pluggable policy layer like rate limits, access control, quotas. • Automatic metrics, logs, traces. • Secure service-to-service
136. istio - trafBic management
137. TrafBic splitting • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
138. TrafBic mirroring • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
139. istio - resilience
140. Load balancing strategy • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Client side load balancing • Do not need reverse proxy -> remove single point of failure.
141. retry and timeout • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
142. fault injection • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
143. circuit breaker • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
144. Observability
145. Metrics • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
146. Logs • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
147. Traces • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Envoy proxy is responsible for generating the initial trace headers and doing so in an OpenTelemetry–compatible way • Your application requires a thin-client library to collect and propagate a small set of HTTP headers: • x-request-id • x-b3-traceid • x-b3-spanid • x-b3-parentspanid • x-b3-sampled • x-b3-flags • x-ot-span-context
148. Traces • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable Life is not always easy ... Need the cooperation from the application
149. Visualization • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
150. Apply istio to the existing infrastructure
151. step-by-step • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Step 1: Install istio to the Kubernetes system • Step 2: Add label istio=enabled to the namespace.
152. istio Binish installation • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable
153. Result • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable After Before
154. Result • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • Network interface might change Before After
155. References • Isolation. • Immutability • Efficient resource utilization. • Lightweight • Portable • https://sookocheff.com/post/kubernetes/understanding-kubernetes-networking-model/ • https://kubernetes.io/docs/home/ • https://www.slideshare.net/lestrrat/kubernetes-in-30-minutes-20170310 • https://blog.getambassador.io/kubernetes-ingress-nodeport-load-balancers-and-ingress- controllers-6e29f1c44f2d • https://jimmysong.io/posts/envoy-sidecar-injection-in-istio-service-mesh-deep-dive/ • https://medium.com/flant-com/comparing-ingress-controllers-for-kubernetes-9b397483b46b • https://github.com/hjacobs/kubernetes-failure-stories • Kubernetes with swap partition: https://github.com/kubernetes/kubernetes/issues/53533 • https://github.com/ramitsurana/awesome-kubernetes Credit to: Huy Vo, engineer manager, @Axon.
156. Q&A