Tour of Istio

Transcript

1. Tour of Istio Gopherಓ৔ ಉ૭ձ2021 Yuki Ito @mrno110
2. Merpay Architect Mercari Micorservices Platform @yuki.ito
3. Agenda ɾWhat / Why ɾHow Istio works
4. Agenda ɾWhat / Why ɾHow Istio works
5. Microserivces at Mercari/Merpay API Gateawy QR Payment iD Payment XXX yyy zzz ɾɾɾ ɾɾɾ
6. Common Concerns ɾAuthentication / Authorization ɾLoad Balance ɾCanary Release etc...
7. Current Solutions e.g. k8sdns Resolver Pod 10.28.1.11 Pod 10.28.1.12 Pod 10.28.1.13 Headless Service hs-serivce.foo.svc.cluster.local gRPC Client 10.28.1.11 10.28.1.12 10.28.1.13 k8sdns:///hs-service.foo
8. Current Solutions e.g. Pod ratio based canary release Pod Pod Pod Service selector: app=foo Main Deployment app=foo Canary Deployment app=foo
9. Problems ɾMust modify the application itself ɾPolyglot ɾNot a fi ne grained canary releasing
10. Service Mesh The term service mesh is used to describe the network of microservices that make up such applications and the interactions between them. https://istio.io/latest/docs/concepts/what-is-istio/
11. Istio https://istio.io/latest/docs/concepts/what-is-istio/
12. Agenda ɾWhat / Why ɾHow Istio works
13. Without Istio Application Application Pod Pod ᶃ ᶄ ᶅ ᶆ
14. Without Istio -- - kind: Namespac e metadata : name: a -- - kind: Deploymen t metadata : name: a namespace: a spec : template : spec : containers : - name: a image: nginx
15. With Istio -- - kind: Namespac e metadata : name: a labels: istio-injection: enabled -- - kind: Deploymen t metadata : name: a namespace: a spec : template : metadata : annotations : sidecar.istio.io/inject: tru e spec : containers : - name: a image: nginx
16. With Istio istio-proxy Application istio-proxy Application Pod Pod ᶃ ᶄ ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ
17. Roll up for the Mystery Istio Tour!
18. Sidecar Injection Application Application Pod Pod
19. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod
20. Sidecar Pattern Sidecar Application Pod Extract non application related logics to the Sidecar container from the Application container. - Networking - Authentication / Authorization - Tracing etc...
21. Sidecar Pattern https://learning.oreilly.com/library/view/designing-distributed-systems/9781491983638/
22. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod
23. Sidecar Injection -- - kind: Namespac e metadata : name: a labels: istio-injection: enabled -- - kind: Deploymen t metadata : name: a namespace: a spec : template : metadata : annotations : sidecar.istio.io/inject: tru e spec : containers : - name: ap p image: nginx
24. Sidecar Injection kind: Po d spec : containers : - name: ap p image: nginx
25. Sidecar Injection kind: Po d spec : containers : - name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8.1
26. Sidecar Injection kubectl kube-apiserver etcd YAML YAML
27. Sidecar Injection kubectl kube-apiserver etcd Mutating Admission Webhook YAML Modi fi ed YAML YAML Modi fi ed YAML https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
28. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod
29. Routing istio-proxy Application istio-proxy Application Pod Pod
30. Routing istio-proxy Application istio-proxy Application Pod Pod
31. Init Container Specialized containers that run before app containers in a Pod. Init containers can contain utilities or setup scripts not present in an app image. https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
32. Init Container kind: Po d spec : containers : - name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8.1
33. Init Container kind: Po d spec : containers : - name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8. 1 initContainers : - name: istio-ini t image: docker.io/istio/proxyv2:1.8. 1 args : - istio-iptables (via Mutating Admission Webhook)
34. Linux Namespaces for Pod Network / IPC / PID / Mount / UTS Container 1 Container 2 cgroup cgroup
35. Linux Namespaces for Pod https://learning.oreilly.com/library/view/container-security/9781492056690/
36. Init Container > pilot-agent \ istio-iptables \ -p 15001 \ -z 15006 \ -u 1337 \ -m REDIRECT \ -i * \ -x \ -b * \ -d
37. iptables > ssh <Kubernetes Node > > sudo nsenter --net --target <app container PID > > iptables --table nat --lis t #.. . Chain ISTIO_IN_REDIRECT (3 references ) target prot opt source destinatio n REDIRECT tcp -- anywhere anywhere redir ports 1500 6 Chain ISTIO_REDIRECT (1 references ) target prot opt source destinatio n REDIRECT tcp -- anywhere anywhere redir ports 15001
38. Inbound Tra ffi c PREROUTING POSTROUTING Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ
39. Inbound Tra ffi c PREROUTING ISTIO_INBOUND ISTIO_IN_REDIRECT ISTIO_OUTPUT POSTROUTING OUTPUT istio-proxy PORT: 15006 Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ
40. Outbound Tra ffi c OUTPUT POSTROUTING Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ
41. Outbound Tra ffi c OUTPUT ISTIO_REDIRECT POSTROUTING ISTIO_OUTPUT istio-proxy PORT: 15001 Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ ᶆ ᶈ ᶇ ᶉ ᶊ ᶋ
42. Istio iptables Implementation func (r *RealDependencies) execute(cmd string, redirectStdout bool, args ...string) error { fmt.Printf("%s %s\n", cmd, strings.Join(args, " ") ) externalCommand := exec.Command(cmd, args...) externalCommand.Stdout = os.Stdou t // TODO Check naming and redirection logic if !redirectStdout { externalCommand.Stderr = os.Stder r } return externalCommand.Run( ) } https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/tools/istio-iptables/pkg/dependencies/implementation.go#L27-L36
43. Routing istio-proxy Application istio-proxy Application Pod Pod ᶃ ᶄ ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ
44. Envoy Envoy Application Envoy Application Pod Pod ᶃ ᶄ ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ
45. Envoy Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures. The project was born out of the belief that:ɹ The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem. https://www.envoyproxy.io/docs/envoy/v1.16.2/intro/what_is_envoy
46. Envoy Con fi gurations Listener Route Cluster Endpoint Endpoint Endpoint Endpoint Cluster
47. Envoy Con fi gurations 0.0.0.0:5000 Listener Route Service-1 Cluster 10.28.1.11 10.28.1.12 10.28.1.13 10.28.1.14 Service-2 Cluster Path: /service-1 Path: /service-2
48. x Discovery Service API Control Plane Route Listener Cluster xDS API
49. x Discovery Service API •Listener Discovery Service •Route Discovery Service •Cluster Discovery Service •Endpoint Discovery Service
50. Control Plane Control Plane Con fi g Con fi g Con fi g xDS API
51. Control Plane istiod Con fi g Con fi g Con fi g xDS API
52. Control Plane istiod Con fi g Con fi g Con fi g xDS API
53. go-control-plane envoyproxy/go-control-plane
54. go-control-plane import ( cache ".../go-control-plane/pkg/cache/v3" server ".../go-control-plane/pkg/server/v3 " discovery ".../go-control-plane/envoy/service/discovery/v3" "google.golang.org/grpc" ) // ... snapshotCache := cache.NewSnapshotCache(... ) server := server.NewServer(ctx, snapshotCache, ... ) grpcServer := grpc.NewServer( ) lis, _ := net.Listen("tcp", ":8081" ) discovery.RegisterAggregatedDiscoveryServiceServer(grpcServer, server ) grpcServer.Serve(lis) Minimum Implementation
55. go-control-plane https://envoytokyo.connpass.com/event/175256/
56. go-control-plane https://gihyo.jp/magazine/SD/archive/2020/202008
57. istiod uses spf13/cobra import ( //.. "github.com/spf13/cobra" ) var ( //.. . rootCmd = &cobra.Command { Use: "pilot-discovery" , Short: "Istio Pilot." , Long: "..." , SilenceUsage: true , } https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/pilot/cmd/pilot-discovery/main.go#L43-L48
58. Istio https://istio.io/latest/docs/concepts/what-is-istio/
59. e.g. VirtualService apiVersion: networking.istio.io/v1alpha 3 kind: VirtualServic e metadata : name: microservice- a namespace: microservice- a spec : hosts : - microservice-a.microservice-a.svc.cluster.loca l http : - match : - headers : target : exact: fo o route : - destination : host: microservice-a-foo.microservice-a.svc.cluster.loca l # ... VirtualService
60. e.g. VirtualService Route microservice-a 10.28.1.11 10.28.1.12 10.28.1.13 10.28.1.14 microservice-a-foo default target: foo microservice-a.microservice-a.svc.cluster.local
61. Wrap up https://istio.io/latest/docs/concepts/what-is-istio/