Tour of Istio

Transcript

1. Tour of Istio Gopherಓ৔ ಉ૭ձ2021 Yuki Ito @mrno110

2. Merpay Architect Mercari Micorservices Platform @yuki.ito

3. Agenda ɾWhat / Why ɾHow Istio works

4. Agenda ɾWhat / Why ɾHow Istio works

5. Microserivces at Mercari/Merpay API Gateawy QR Payment iD Payment XXX

   yyy zzz ɾɾɾ ɾɾɾ

6. Common Concerns ɾAuthentication / Authorization ɾLoad Balance ɾCanary Release etc...

7. Current Solutions e.g. k8sdns Resolver Pod 10.28.1.11 Pod 10.28.1.12 Pod

   10.28.1.13 Headless Service hs-serivce.foo.svc.cluster.local gRPC Client 10.28.1.11 10.28.1.12 10.28.1.13 k8sdns:///hs-service.foo

8. Current Solutions e.g. Pod ratio based canary release Pod Pod

   Pod Service selector: app=foo Main Deployment app=foo Canary Deployment app=foo

9. Problems ɾMust modify the application itself ɾPolyglot ɾNot a fi

   ne grained canary releasing

10. Service Mesh The term service mesh is used to describe

    the network of microservices that make up such applications and the interactions between them. https://istio.io/latest/docs/concepts/what-is-istio/

11. Istio https://istio.io/latest/docs/concepts/what-is-istio/

12. Agenda ɾWhat / Why ɾHow Istio works

13. Without Istio Application Application Pod Pod ᶃ ᶄ ᶅ ᶆ

14. Without Istio -- - kind: Namespac e metadata : name:

    a -- - kind: Deploymen t metadata : name: a namespace: a spec : template : spec : containers : - name: a image: nginx

15. With Istio -- - kind: Namespac e metadata : name:

    a labels: istio-injection: enabled -- - kind: Deploymen t metadata : name: a namespace: a spec : template : metadata : annotations : sidecar.istio.io/inject: tru e spec : containers : - name: a image: nginx

16. With Istio istio-proxy Application istio-proxy Application Pod Pod ᶃ ᶄ

    ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ

17. Roll up for the Mystery Istio Tour!

18. Sidecar Injection Application Application Pod Pod

19. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod

20. Sidecar Pattern Sidecar Application Pod Extract non application related logics

    to the Sidecar container from the Application container. - Networking - Authentication / Authorization - Tracing etc...

21. Sidecar Pattern https://learning.oreilly.com/library/view/designing-distributed-systems/9781491983638/

22. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod

23. Sidecar Injection -- - kind: Namespac e metadata : name:

    a labels: istio-injection: enabled -- - kind: Deploymen t metadata : name: a namespace: a spec : template : metadata : annotations : sidecar.istio.io/inject: tru e spec : containers : - name: ap p image: nginx

24. Sidecar Injection kind: Po d spec : containers : -

    name: ap p image: nginx

25. Sidecar Injection kind: Po d spec : containers : -

    name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8.1

26. Sidecar Injection kubectl kube-apiserver etcd YAML YAML

27. Sidecar Injection kubectl kube-apiserver etcd Mutating Admission Webhook YAML Modi

    fi ed YAML YAML Modi fi ed YAML https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/

28. Sidecar Injection istio-proxy Application istio-proxy Application Pod Pod

29. Routing istio-proxy Application istio-proxy Application Pod Pod

30. Routing istio-proxy Application istio-proxy Application Pod Pod

31. Init Container Specialized containers that run before app containers in

    a Pod. Init containers can contain utilities or setup scripts not present in an app image. https://kubernetes.io/docs/concepts/workloads/pods/init-containers/

32. Init Container kind: Po d spec : containers : -

    name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8.1

33. Init Container kind: Po d spec : containers : -

    name: ap p image: ngin x - name: istio-prox y image: docker.io/istio/proxyv2:1.8. 1 initContainers : - name: istio-ini t image: docker.io/istio/proxyv2:1.8. 1 args : - istio-iptables (via Mutating Admission Webhook)

34. Linux Namespaces for Pod Network / IPC / PID /

    Mount / UTS Container 1 Container 2 cgroup cgroup

35. Linux Namespaces for Pod https://learning.oreilly.com/library/view/container-security/9781492056690/

36. Init Container > pilot-agent \ istio-iptables \ -p 15001 \

    -z 15006 \ -u 1337 \ -m REDIRECT \ -i * \ -x \ -b * \ -d

37. iptables > ssh <Kubernetes Node > > sudo nsenter --net

    --target <app container PID > > iptables --table nat --lis t #.. . Chain ISTIO_IN_REDIRECT (3 references ) target prot opt source destinatio n REDIRECT tcp -- anywhere anywhere redir ports 1500 6 Chain ISTIO_REDIRECT (1 references ) target prot opt source destinatio n REDIRECT tcp -- anywhere anywhere redir ports 15001

38. Inbound Tra ffi c PREROUTING POSTROUTING Application Linux Kernel Space

    (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ

39. Inbound Tra ffi c PREROUTING ISTIO_INBOUND ISTIO_IN_REDIRECT ISTIO_OUTPUT POSTROUTING OUTPUT

    istio-proxy PORT: 15006 Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ ᶆ ᶇ ᶈ ᶉ ᶊ

40. Outbound Tra ffi c OUTPUT POSTROUTING Application Linux Kernel Space

    (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ

41. Outbound Tra ffi c OUTPUT ISTIO_REDIRECT POSTROUTING ISTIO_OUTPUT istio-proxy PORT:

    15001 Application Linux Kernel Space (iptables / net fi lter) Linux User Space ᶃ ᶄ ᶅ ᶆ ᶈ ᶇ ᶉ ᶊ ᶋ

42. Istio iptables Implementation func (r *RealDependencies) execute(cmd string, redirectStdout bool,

    args ...string) error { fmt.Printf("%s %s\n", cmd, strings.Join(args, " ") ) externalCommand := exec.Command(cmd, args...) externalCommand.Stdout = os.Stdou t // TODO Check naming and redirection logic if !redirectStdout { externalCommand.Stderr = os.Stder r } return externalCommand.Run( ) } https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/tools/istio-iptables/pkg/dependencies/implementation.go#L27-L36

43. Routing istio-proxy Application istio-proxy Application Pod Pod ᶃ ᶄ ᶅ

    ᶆ ᶇ ᶈ ᶉ ᶊ

44. Envoy Envoy Application Envoy Application Pod Pod ᶃ ᶄ ᶅ

    ᶆ ᶇ ᶈ ᶉ ᶊ

45. Envoy Envoy is an L7 proxy and communication bus designed

    for large modern service oriented architectures. The project was born out of the belief that:ɹ The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem. https://www.envoyproxy.io/docs/envoy/v1.16.2/intro/what_is_envoy

46. Envoy Con fi gurations Listener Route Cluster Endpoint Endpoint Endpoint

    Endpoint Cluster

47. Envoy Con fi gurations 0.0.0.0:5000 Listener Route Service-1 Cluster 10.28.1.11

    10.28.1.12 10.28.1.13 10.28.1.14 Service-2 Cluster Path: /service-1 Path: /service-2

48. x Discovery Service API Control Plane Route Listener Cluster xDS

    API

49. x Discovery Service API •Listener Discovery Service •Route Discovery Service

    •Cluster Discovery Service •Endpoint Discovery Service

50. Control Plane Control Plane Con fi g Con fi g

    Con fi g xDS API

51. Control Plane istiod Con fi g Con fi g Con

    fi g xDS API

52. Control Plane istiod Con fi g Con fi g Con

    fi g xDS API

53. go-control-plane envoyproxy/go-control-plane

54. go-control-plane import ( cache ".../go-control-plane/pkg/cache/v3" server ".../go-control-plane/pkg/server/v3 " discovery ".../go-control-plane/envoy/service/discovery/v3"

    "google.golang.org/grpc" ) // ... snapshotCache := cache.NewSnapshotCache(... ) server := server.NewServer(ctx, snapshotCache, ... ) grpcServer := grpc.NewServer( ) lis, _ := net.Listen("tcp", ":8081" ) discovery.RegisterAggregatedDiscoveryServiceServer(grpcServer, server ) grpcServer.Serve(lis) Minimum Implementation

55. go-control-plane https://envoytokyo.connpass.com/event/175256/

56. go-control-plane https://gihyo.jp/magazine/SD/archive/2020/202008

57. istiod uses spf13/cobra import ( //.. "github.com/spf13/cobra" ) var (

    //.. . rootCmd = &cobra.Command { Use: "pilot-discovery" , Short: "Istio Pilot." , Long: "..." , SilenceUsage: true , } https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/pilot/cmd/pilot-discovery/main.go#L43-L48

58. Istio https://istio.io/latest/docs/concepts/what-is-istio/

59. e.g. VirtualService apiVersion: networking.istio.io/v1alpha 3 kind: VirtualServic e metadata :

    name: microservice- a namespace: microservice- a spec : hosts : - microservice-a.microservice-a.svc.cluster.loca l http : - match : - headers : target : exact: fo o route : - destination : host: microservice-a-foo.microservice-a.svc.cluster.loca l # ... VirtualService

60. e.g. VirtualService Route microservice-a 10.28.1.11 10.28.1.12 10.28.1.13 10.28.1.14 microservice-a-foo default

    target: foo microservice-a.microservice-a.svc.cluster.local

61. Wrap up https://istio.io/latest/docs/concepts/what-is-istio/