Slide 1
Slide 1 text
Tour of Istio
Gopherಓ ಉ૭ձ2021
Yuki Ito
@mrno110
Slide 2
Slide 2 text
Merpay
Architect
Mercari
Micorservices Platform
@yuki.ito
Slide 3
Slide 3 text
Agenda
ɾWhat / Why
ɾHow Istio works
Slide 4
Slide 4 text
Agenda
ɾWhat / Why
ɾHow Istio works
Slide 5
Slide 5 text
Microserivces at Mercari/Merpay
API Gateawy
QR Payment iD Payment
XXX yyy zzz
ɾɾɾ
ɾɾɾ
Slide 6
Slide 6 text
Common Concerns
ɾAuthentication / Authorization
ɾLoad Balance
ɾCanary Release
etc...
Slide 7
Slide 7 text
Current Solutions
e.g. k8sdns Resolver
Pod
10.28.1.11
Pod
10.28.1.12
Pod
10.28.1.13
Headless Service
hs-serivce.foo.svc.cluster.local
gRPC Client
10.28.1.11
10.28.1.12
10.28.1.13
k8sdns:///hs-service.foo
Slide 8
Slide 8 text
Current Solutions
e.g. Pod ratio based canary release
Pod Pod Pod
Service
selector: app=foo
Main Deployment
app=foo
Canary Deployment
app=foo
Slide 9
Slide 9 text
Problems
ɾMust modify the application itself
ɾPolyglot
ɾNot a
fi
ne grained canary releasing
Slide 10
Slide 10 text
Service Mesh
The term service mesh is used to describe the
network of microservices that make up such
applications and the interactions between them.
https://istio.io/latest/docs/concepts/what-is-istio/
Slide 11
Slide 11 text
Istio
https://istio.io/latest/docs/concepts/what-is-istio/
Slide 12
Slide 12 text
Agenda
ɾWhat / Why
ɾHow Istio works
Slide 13
Slide 13 text
Without Istio
Application
Application
Pod
Pod
ᶃ
ᶄ ᶅ
ᶆ
Slide 14
Slide 14 text
Without Istio
--
-
kind: Namespac
e
metadata
:
name:
a
--
-
kind: Deploymen
t
metadata
:
name:
a
namespace:
a
spec
:
template
:
spec
:
containers
:
- name:
a
image: nginx
Slide 15
Slide 15 text
With Istio
--
-
kind: Namespac
e
metadata
:
name:
a
labels:
istio-injection: enabled
--
-
kind: Deploymen
t
metadata
:
name:
a
namespace:
a
spec
:
template
:
metadata
:
annotations
:
sidecar.istio.io/inject: tru
e
spec
:
containers
:
- name:
a
image: nginx
Slide 16
Slide 16 text
With Istio
istio-proxy Application
istio-proxy Application
Pod
Pod
ᶃ
ᶄ
ᶅ
ᶆ
ᶇ
ᶈ
ᶉ
ᶊ
Slide 17
Slide 17 text
Roll up for the
Mystery Istio Tour!
Slide 18
Slide 18 text
Sidecar Injection
Application
Application
Pod
Pod
Slide 19
Slide 19 text
Sidecar Injection
istio-proxy Application
istio-proxy Application
Pod
Pod
Slide 20
Slide 20 text
Sidecar Pattern
Sidecar Application
Pod
Extract non application related logics to the
Sidecar container from the Application container.
- Networking
- Authentication / Authorization
- Tracing
etc...
Slide 21
Slide 21 text
Sidecar Pattern
https://learning.oreilly.com/library/view/designing-distributed-systems/9781491983638/
Slide 22
Slide 22 text
Sidecar Injection
istio-proxy Application
istio-proxy Application
Pod
Pod
Slide 23
Slide 23 text
Sidecar Injection
--
-
kind: Namespac
e
metadata
:
name:
a
labels:
istio-injection: enabled
--
-
kind: Deploymen
t
metadata
:
name:
a
namespace:
a
spec
:
template
:
metadata
:
annotations
:
sidecar.istio.io/inject: tru
e
spec
:
containers
:
- name: ap
p
image: nginx
Slide 24
Slide 24 text
Sidecar Injection
kind: Po
d
spec
:
containers
:
- name: ap
p
image: nginx
Slide 25
Slide 25 text
Sidecar Injection
kind: Po
d
spec
:
containers
:
- name: ap
p
image: ngin
x
- name: istio-prox
y
image: docker.io/istio/proxyv2:1.8.1
Slide 26
Slide 26 text
Sidecar Injection
kubectl
kube-apiserver
etcd
YAML
YAML
Slide 27
Slide 27 text
Sidecar Injection
kubectl
kube-apiserver
etcd
Mutating
Admission
Webhook
YAML
Modi
fi
ed YAML
YAML
Modi
fi
ed YAML
https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
Slide 28
Slide 28 text
Sidecar Injection
istio-proxy Application
istio-proxy Application
Pod
Pod
Slide 29
Slide 29 text
Routing
istio-proxy Application
istio-proxy Application
Pod
Pod
Slide 30
Slide 30 text
Routing
istio-proxy Application
istio-proxy Application
Pod
Pod
Slide 31
Slide 31 text
Init Container
Specialized containers that run before
app containers in a Pod.
Init containers can contain utilities or
setup scripts not present in an app
image.
https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
Slide 32
Slide 32 text
Init Container
kind: Po
d
spec
:
containers
:
- name: ap
p
image: ngin
x
- name: istio-prox
y
image: docker.io/istio/proxyv2:1.8.1
Slide 33
Slide 33 text
Init Container
kind: Po
d
spec
:
containers
:
- name: ap
p
image: ngin
x
- name: istio-prox
y
image: docker.io/istio/proxyv2:1.8.
1
initContainers
:
- name: istio-ini
t
image: docker.io/istio/proxyv2:1.8.
1
args
:
- istio-iptables
(via Mutating Admission Webhook)
Slide 34
Slide 34 text
Linux Namespaces for Pod
Network / IPC / PID / Mount / UTS
Container 1 Container 2
cgroup cgroup
Slide 35
Slide 35 text
Linux Namespaces for Pod
https://learning.oreilly.com/library/view/container-security/9781492056690/
Slide 36
Slide 36 text
Init Container
> pilot-agent
\
istio-iptables
\
-p 15001
\
-z 15006
\
-u 1337
\
-m REDIRECT
\
-i *
\
-x
\
-b *
\
-d
Slide 37
Slide 37 text
iptables
> ssh
> sudo nsenter --net --target
> iptables --table nat --lis
t
#..
.
Chain ISTIO_IN_REDIRECT (3 references
)
target prot opt source destinatio
n
REDIRECT tcp -- anywhere anywhere redir ports 1500
6
Chain ISTIO_REDIRECT (1 references
)
target prot opt source destinatio
n
REDIRECT tcp -- anywhere anywhere redir ports 15001
Slide 38
Slide 38 text
Inbound Tra
ffi
c
PREROUTING
POSTROUTING
Application
Linux Kernel Space (iptables / net
fi
lter) Linux User Space
ᶃ
ᶄ
ᶅ
Slide 39
Slide 39 text
Inbound Tra
ffi
c
PREROUTING
ISTIO_INBOUND
ISTIO_IN_REDIRECT
ISTIO_OUTPUT
POSTROUTING
OUTPUT
istio-proxy
PORT: 15006
Application
Linux Kernel Space (iptables / net
fi
lter) Linux User Space
ᶃ
ᶄ
ᶅ
ᶆ
ᶇ
ᶈ
ᶉ
ᶊ
Slide 40
Slide 40 text
Outbound Tra
ffi
c
OUTPUT
POSTROUTING Application
Linux Kernel Space (iptables / net
fi
lter) Linux User Space
ᶃ
ᶄ
ᶅ
Slide 41
Slide 41 text
Outbound Tra
ffi
c
OUTPUT
ISTIO_REDIRECT
POSTROUTING
ISTIO_OUTPUT
istio-proxy
PORT: 15001
Application
Linux Kernel Space (iptables / net
fi
lter) Linux User Space
ᶃ
ᶄ
ᶅ
ᶆ
ᶈ
ᶇ
ᶉ
ᶊ
ᶋ
Slide 42
Slide 42 text
Istio iptables Implementation
func (r *RealDependencies) execute(cmd string, redirectStdout bool, args ...string) error
{
fmt.Printf("%s %s\n", cmd, strings.Join(args, " ")
)
externalCommand := exec.Command(cmd, args...)
externalCommand.Stdout = os.Stdou
t
// TODO Check naming and redirection logic
if !redirectStdout
{
externalCommand.Stderr = os.Stder
r
}
return externalCommand.Run(
)
}
https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/tools/istio-iptables/pkg/dependencies/implementation.go#L27-L36
Slide 43
Slide 43 text
Routing
istio-proxy Application
istio-proxy Application
Pod
Pod
ᶃ
ᶄ
ᶅ
ᶆ
ᶇ
ᶈ
ᶉ
ᶊ
Slide 44
Slide 44 text
Envoy
Envoy Application
Envoy Application
Pod
Pod
ᶃ
ᶄ
ᶅ
ᶆ
ᶇ
ᶈ
ᶉ
ᶊ
Slide 45
Slide 45 text
Envoy
Envoy is an L7 proxy and communication bus designed
for large modern service oriented architectures. The
project was born out of the belief that:ɹ
The network should be transparent to applications.
When network and application problems do occur it
should be easy to determine the source of the problem.
https://www.envoyproxy.io/docs/envoy/v1.16.2/intro/what_is_envoy
Slide 46
Slide 46 text
Envoy Con
fi
gurations
Listener
Route
Cluster
Endpoint Endpoint Endpoint Endpoint
Cluster
Slide 47
Slide 47 text
Envoy Con
fi
gurations
0.0.0.0:5000 Listener
Route
Service-1 Cluster
10.28.1.11 10.28.1.12 10.28.1.13 10.28.1.14
Service-2 Cluster
Path: /service-1 Path: /service-2
Slide 48
Slide 48 text
x Discovery Service API
Control Plane
Route
Listener Cluster
xDS API
Slide 49
Slide 49 text
x Discovery Service API
•Listener Discovery Service
•Route Discovery Service
•Cluster Discovery Service
•Endpoint Discovery Service
Slide 50
Slide 50 text
Control Plane
Control Plane
Con
fi
g
Con
fi
g Con
fi
g
xDS API
Slide 51
Slide 51 text
Control Plane
istiod
Con
fi
g
Con
fi
g Con
fi
g
xDS API
Slide 52
Slide 52 text
Control Plane
istiod
Con
fi
g
Con
fi
g Con
fi
g
xDS API
Slide 53
Slide 53 text
go-control-plane
envoyproxy/go-control-plane
Slide 54
Slide 54 text
go-control-plane
import (
cache ".../go-control-plane/pkg/cache/v3"
server ".../go-control-plane/pkg/server/v3
"
discovery ".../go-control-plane/envoy/service/discovery/v3"
"google.golang.org/grpc"
)
// ...
snapshotCache := cache.NewSnapshotCache(...
)
server := server.NewServer(ctx, snapshotCache, ...
)
grpcServer := grpc.NewServer(
)
lis, _ := net.Listen("tcp", ":8081"
)
discovery.RegisterAggregatedDiscoveryServiceServer(grpcServer, server
)
grpcServer.Serve(lis)
Minimum Implementation
Slide 55
Slide 55 text
go-control-plane
https://envoytokyo.connpass.com/event/175256/
Slide 56
Slide 56 text
go-control-plane
https://gihyo.jp/magazine/SD/archive/2020/202008
Slide 57
Slide 57 text
istiod uses spf13/cobra
import
(
//..
"github.com/spf13/cobra"
)
var
(
//..
.
rootCmd = &cobra.Command
{
Use: "pilot-discovery"
,
Short: "Istio Pilot."
,
Long: "..."
,
SilenceUsage: true
,
}
https://github.com/istio/istio/blob/95c5fe4026f5a395893e92e1c9297a03b06b7dd4/pilot/cmd/pilot-discovery/main.go#L43-L48
Slide 58
Slide 58 text
Istio
https://istio.io/latest/docs/concepts/what-is-istio/
Slide 59
Slide 59 text
e.g. VirtualService
apiVersion: networking.istio.io/v1alpha
3
kind: VirtualServic
e
metadata
:
name: microservice-
a
namespace: microservice-
a
spec
:
hosts
:
- microservice-a.microservice-a.svc.cluster.loca
l
http
:
- match
:
- headers
:
target
:
exact: fo
o
route
:
- destination
:
host: microservice-a-foo.microservice-a.svc.cluster.loca
l
# ...
VirtualService
Slide 60
Slide 60 text
e.g. VirtualService
Route
microservice-a
10.28.1.11 10.28.1.12 10.28.1.13 10.28.1.14
microservice-a-foo
default target: foo
microservice-a.microservice-a.svc.cluster.local
Slide 61
Slide 61 text
Wrap up
https://istio.io/latest/docs/concepts/what-is-istio/