kube-proxy running in iptables mode Backend Pod 1 labels: app=MyApp port: 9376 Backend Pod 2 labels: app=MyApp port: 9376 Backend Pod 3 labels: app=MyApp port: 9376 Client kube-proxy apiserver clusterIP (iptables) Node ( S e e : h t t p s : / / k u b e r n e t e s . i o / d o c s / c o n c e p t s / s e r v i c e s - n e t w o r k i n g / s e r v i c e / # p r o x y - m o d e - i p t a b l e s ) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
Just a sample Kubernetes Service object --- apiVersion: v1 kind: Service metadata: name: myservice # <- can be resolved with internal DNS, f.e. spec: # myservice.default.svc.cluster.local selector: app: myservice # <- route traffic to Pods with this label ports: - port: 80 # <- virtual port of the Service one connects to targetPort: 80 # <- port of a Pod, can be named, like "http" type: ClusterIP # <- usually set as default ( S e e : h t t p s : / / k u b e r n e t e s . i o / d o c s / c o n c e p t s / s e r v i c e s - n e t w o r k i n g / s e r v i c e / # d e f i n i n g - a - s e r v i c e ) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
in Kubernetes there are five types of services LoadBalancer NodePort ClusterIP headless Service (used for Service Discovery) and the special one: externalIPs ( S e e : h t t p s : / / k u b e r n e t e s . i o / d o c s / c o n c e p t s / s e r v i c e s - n e t w o r k i n g / s e r v i c e / # p u b l i s h i n g - s e r v i c e s - s e r v i c e - t y p e s ) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those externalIPs. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port, will be routed to one of the Service endpoints. externalIPs are not managed by Kubernetes and are the responsibility of the cluster administrator. ( S e e : h t t p s : / / k u b e r n e t e s . i o / d o c s / c o n c e p t s / s e r v i c e s - n e t w o r k i n g / s e r v i c e / # e x t e r n a l - i p s ) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
when you need to route the traffic to the nodes without a load-balancer (f.e. dev setups, bare metal, a.s.o.) well, if you have some external IPs and special routing configuration IP failover maybe some more :) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
what happens if we add some IP address from an external service as externalIPs to our cluster? --> yes, traffic will be routet to that service! what is the impact? well, you can do some fancy MITM :) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
check if externalIPs specified in the service object and reject it if you still need it, add exceptions to your rule :) you can use the Validation Webhook from kubernetes-sigs or use eBPF based CNIs (cilium) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
References https://github.com/kubernetes/kubernetes/issues/97076 https://kubernetes.io/docs/concepts/services-networking/service https://medium.com/swlh/kubernetes-external-ip-service-type-5e5e9ad62fcd https://serenafeng.github.io/2020/03/26/kube-proxy-in-iptables-mode/ https://kubernetes.io/docs/reference/access-authn-authz/admission- controllers/ https://cilium.io/ https://github.com/kubernetes-sigs/externalip-webhook ( C r e a t e d u s i n g h t t p s : / / m a r p i t . m a r p . a p p / ) WhizUs GmbH WhizUs GmbH KCSP & KTP KCSP & KTP 01/2021 01/2021
dklesev
pospome
yutoy
kei_s
tomoaki25
cimadai
oracle4engineer
nishiuma
wakamatsu_takumu
leveragestech
nayuts
keiko713
iamctodd
ammeep
mongodb
morganepeng
jnunemaker
notwaldorf
thoeni
lara
tmm1
nonsquared
holman
rasmusluckow