Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
OpenShift-SDNとNetworkPolicy
orimanabu
May 28, 2019
Technology
2
1k
OpenShift-SDNとNetworkPolicy
Deep dive in OpenShift-SDN (ovs-networkpolicy)
orimanabu
May 28, 2019
Tweet
Share
More Decks by orimanabu
See All by orimanabu
skupper-introduction
orimanabu
0
110
OpenShift.Run-2022-makaizo
orimanabu
1
690
CRI-O Introduction
orimanabu
5
1.7k
Submariner-RHTN-20210114.pdf
orimanabu
0
170
Submariner-ONIC2020-ja
orimanabu
3
190
OVNのご紹介
orimanabu
4
530
Other Decks in Technology
See All in Technology
The Fractal Geometry of Software Design
vladikk
1
1.4k
Strategyパターン
hankehly
0
170
【配布資料】複数データソースのデータを仮想化してまとめて分析してみよう(AWS、IBM Cloud)
tkhresk
0
110
What's new in Vision
satotakeshi
0
220
20220628event_ogura_part
caddi_eng
0
150
Security Hub のマルチアカウント 管理・運用をサーバレスでやってみる
ch6noota
0
950
20220705-BASEDMM
ryotaumebayashi
0
120
【toranoana.deno#7】Denoからwasmを呼び出す基礎
toranoana
0
130
開発組織の生産性を可視化する State of DevOpsとFour Keysとは / deep dive into State of DevOps
yfcgpsebp
0
310
Google Cloud Updates 2022/05/16-05/31
no24oka
2
110
Retca Cloud
bau
0
580
今どきのLinux事情
tokida
45
36k
Featured
See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
237
19k
Six Lessons from altMBA
skipperchong
14
1.4k
Writing Fast Ruby
sferik
612
57k
Designing Experiences People Love
moore
130
22k
Keith and Marios Guide to Fast Websites
keithpitt
404
21k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
15k
The World Runs on Bad Software
bkeepers
PRO
57
5.3k
Code Reviewing Like a Champion
maltzj
506
37k
In The Pink: A Labor of Love
frogandcode
131
21k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
12
930
How To Stay Up To Date on Web Technology
chriscoyier
780
250k
Fireside Chat
paigeccino
12
1.3k
Transcript
OpenShift-SDNと NetworkPolicy Manabu Ori @orimanabu 1 2019-05-28 Red Hat Tech
Night 2019.05
CNIとは 2 • CNIとは ◦ Kubernetes でコンテナのネットワークインタフェースを設定 するための仕様 • やること
◦ Linux コンテナが作成された際のネットワーク接続性の確保 ◦ コンテナが 削除された際のリソース解放を行う
OpenShift-SDN 3 • OpenShiftのデフォルトのCNIプラグイン • OpenShift-SDN以外に、一応サードパーティのCNIプラグインも 使えます ◦ Flannel (host-gw)
◦ VMware NSX-T ◦ Nuage VSP ◦ Kuryr ◦ Juniper Contrail ◦ Calico ◦ ...
OpenShift-SDN 4 • 3つの動作モード ◦ ovs-subnet ▪ ぶっとおし ◦ ovs-multitenant
▪ 同じプロジェクト内は疎通できる、異なるプロジェクトの Podとは通信できない ◦ ovs-networkpolicy ▪ NetworkPolicy v1 • Open vSwitch (OVS)を活用 • VXLANでオーバーレイ
パケットの流れ 5 br0 vxlan0 tun0 veth6f5cb1b4 kernel iptables eth3 eth0
httpd 4 1 2 node2 172.16.99/0/24 vxlan0 kernel iptables eth3 1 2 node1 server1 server1- 1-6qc4j 172.16.99.42 172.16.99.41 10.129.0.1 10.129.0.13 tun0 172.30.255.14 10.130.0.1 br0 eth0 httpd client1- 1-zn9ff 10.130.0.17 14
VNID 6 [ori@ocp311-master1 RHTN]$ oc get netnamespaces NAME NETID EGRESS
IPS default 0 [] kube-public 536622 [] kube-system 7695582 [] management-infra 14065074 [] openshift 2031527 [] openshift-console 2954107 [] openshift-infra 12640971 [] openshift-logging 13439836 [] openshift-node 3244486 [] openshift-sdn 14688704 [] openshift-web-console 7072175 [] proj1 4610606 [] proj2 10513584 [] 0x465a2e a06cb0
node1のflow entry 7 table=0, priority=400, ip,in_port=2,nw_src=10.130.0.1 actions=goto_table:30 table=0, priority=300, ct_state=-trk,ip
actions=ct(table=0) table=0, priority=300, ip,in_port=2,nw_src=10.130.0.0/23,nw_dst=10.128.0.0/14 actions=goto_table:25 table=0, priority=250, ip,in_port=2,nw_dst=224.0.0.0/4 actions=drop table=0, priority=200, arp,in_port=1,arp_spa=10.128.0.0/14,arp_tpa=10.130.0.0/23 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 table=0, priority=200, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 table=0, priority=200, ip,in_port=1,nw_dst=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 table=0, priority=200, arp,in_port=2,arp_spa=10.130.0.1,arp_tpa=10.128.0.0/14 actions=goto_table:30 table=0, priority=200, ip,in_port=2 actions=goto_table:30 table=0, priority=150, in_port=1 actions=drop table=0, priority=150, in_port=2 actions=drop table=0, priority=100, arp actions=goto_table:20 table=0, priority=100, ip actions=goto_table:20 table=0, priority=0 actions=drop table=10, priority=100, tun_src=172.16.99.31 actions=goto_table:30 table=10, priority=100, tun_src=172.16.99.21 actions=goto_table:30 table=10, priority=100, tun_src=172.16.99.42 actions=goto_table:30 table=10, priority=0 actions=drop table=20, priority=100, arp,in_port=14,arp_spa=10.130.0.17,arp_sha=00:00:0a:82:00:11/00:00:ff:ff:ff:ff actions=load:0x465a2e->NXM_NX_REG0[],goto_table:21 table=20, priority=100, arp,in_port=18,arp_spa=10.130.0.21,arp_sha=00:00:0a:82:00:15/00:00:ff:ff:ff:ff actions=load:0xa06cb0->NXM_NX_REG0[],goto_table:21 table=20, priority=100, ip,in_port=14,nw_src=10.130.0.17 actions=load:0x465a2e->NXM_NX_REG0[],goto_table:21 table=20, priority=100, ip,in_port=18,nw_src=10.130.0.21 actions=load:0xa06cb0->NXM_NX_REG0[],goto_table:21 table=20, priority=0 actions=drop table=21, priority=200, ip,nw_dst=10.128.0.0/14 actions=ct(commit,table=30) table=21, priority=0 actions=goto_table:30 table=25, priority=100, ip,nw_src=10.130.0.17 actions=load:0x465a2e->NXM_NX_REG0[],goto_table:30 table=25, priority=100, ip,nw_src=10.130.0.21 actions=load:0xa06cb0->NXM_NX_REG0[],goto_table:30 table=25, priority=0 actions=drop table=30, priority=300, arp,arp_tpa=10.130.0.1 actions=output:2 table=30, priority=300, ip,nw_dst=10.130.0.1 actions=output:2 table=30, priority=300, ct_state=+rpl,ip,nw_dst=10.130.0.0/23 actions=ct(table=70,nat) table=30, priority=200, arp,arp_tpa=10.130.0.0/23 actions=goto_table:40 table=30, priority=200, ip,nw_dst=10.130.0.0/23 actions=goto_table:70 table=30, priority=100, arp,arp_tpa=10.128.0.0/14 actions=goto_table:50 table=30, priority=100, ip,nw_dst=10.128.0.0/14 actions=goto_table:90 table=30, priority=100, ip,nw_dst=172.30.0.0/16 actions=goto_table:60 table=30, priority=50, ip,in_port=1,nw_dst=224.0.0.0/4 actions=goto_table:120 table=30, priority=25, ip,nw_dst=224.0.0.0/4 actions=goto_table:110 table=30, priority=0, ip actions=goto_table:100 table=30, priority=0, arp actions=drop table=40, priority=100, arp,arp_tpa=10.130.0.17 actions=output:14 table=40, priority=100, arp,arp_tpa=10.130.0.21 actions=output:18 table=40, priority=0 actions=drop table=50, priority=100, arp,arp_tpa=10.131.0.0/23 actions=move:NXM_NX_REG0[]->NXM_NX_TUN_ID[0..31],set_field:172.16.99.31->tun_dst,output:1 table=50, priority=100, arp,arp_tpa=10.128.0.0/23 actions=move:NXM_NX_REG0[]->NXM_NX_TUN_ID[0..31],set_field:172.16.99.21->tun_dst,output:1 table=50, priority=100, arp,arp_tpa=10.129.0.0/23 actions=move:NXM_NX_REG0[]->NXM_NX_TUN_ID[0..31],set_field:172.16.99.42->tun_dst,output:1 table=50, priority=0 actions=drop table=60, priority=200 actions=output:2 table=60, priority=0 actions=drop table=70, priority=100, ip,nw_dst=10.130.0.17 actions=load:0x465a2e->NXM_NX_REG1[],load:0xe->NXM_NX_REG2[],goto_table:80 table=70, priority=100, ip,nw_dst=10.130.0.21 actions=load:0xa06cb0->NXM_NX_REG1[],load:0x12->NXM_NX_REG2[],goto_table:80 table=70, priority=0 actions=drop table=80, priority=300, ip,nw_src=10.130.0.1 actions=output:NXM_NX_REG2[] table=80, priority=200, ct_state=+rpl,ip actions=output:NXM_NX_REG2[] table=80, priority=50, reg1=0xa4fb4b actions=output:NXM_NX_REG2[] table=80, priority=50, reg1=0xa7d717 actions=output:NXM_NX_REG2[] table=80, priority=50, reg1=0x465a2e actions=output:NXM_NX_REG2[] table=80, priority=50, reg1=0xa06cb0 actions=output:NXM_NX_REG2[] table=80, priority=0 actions=drop
client1 → server1 on node1 8 • table=0, ip actions=goto_table:20
• table=20, ip,in_port=14,nw_src=10.130.0.17 actions=load:0x465a2e->NXM_NX_REG0[],goto_table:21 • table=21, ip,nw_dst=10.128.0.0/14 actions=ct(commit,table=30) • table=30, ip,nw_dst=10.128.0.0/14 actions=goto_table:90 • table=90, ip,nw_dst=10.129.0.0/23 actions=move:NXM_NX_REG0[]->NXM_NX_TUN_ID[0..31],set_field:172.16.99.42->tun_dst,output:1
client1 → server1 on node2 9 • table=0, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10
• table=10, tun_src=172.16.99.41 actions=goto_table:30 • table=30, ip,nw_dst=10.129.0.0/23 actions=goto_table:70 • table=70, ip,nw_dst=10.129.0.13 actions=load:0x465a2e->NXM_NX_REG1[],load:0x8->NXM_NX_REG2[],goto_table:80 • table=80, reg1=0x465a2e actions=output:NXM_NX_REG2[]
NetworkPolicy適用 (1) 10 • table=0, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 • table=10, tun_src=172.16.99.41
actions=goto_table:30 • table=30, ip,nw_dst=10.129.0.0/23 actions=goto_table:70 • table=70, ip,nw_dst=10.129.0.13 actions=load:0x465a2e->NXM_NX_REG1[],load:0x8->NXM_NX_REG2[],goto_table:80 • table=80, actions=drop kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: deny-by-default spec: podSelector: ingress: [] Flow rules for [client1 → server1] on node2
NetworkPolicy適用 (2) 11 • table=0, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 • table=10, tun_src=172.16.99.41
actions=goto_table:30 • table=30, ip,nw_dst=10.129.0.0/23 actions=goto_table:70 • table=70, ip,nw_dst=10.129.0.13 actions=load:0x465a2e->NXM_NX_REG1[],load:0x8->NXM_NX_REG2[],goto_table:80 • table=80, reg0=0x465a2e,reg1=0x465a2e actions=output:NXM_NX_REG2[] kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: allow-same-namespace spec: podSelector: ingress: - from: - podSelector: {} Flow rules for [client1 → server1] on node2
NetworkPolicy適用 (3) 12 • table=0, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 • table=10, tun_src=172.16.99.41
actions=goto_table:30 • table=30, ip,nw_dst=10.129.0.0/23 actions=goto_table:70 • table=70, ip,nw_dst=10.129.0.13 actions=load:0x465a2e->NXM_NX_REG1[],load:0x8->NXM_NX_REG2[],goto_table:80 • table=80, ip,reg0=0x465a2e,reg1=0x465a2e,nw_src=10.130.0.17,nw_dst=10.129.0.13 actions=output:NXM_NX_REG2[] kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: allow-server1-from-client 1 spec: podSelector: matchLabels: app: server1 ingress: - from: - podSelector: matchLabels: app: client1 Flow rules for [client1 → server1] on node2
NetworkPolicy適用 (4) 13 • table=0, ip,in_port=1,nw_src=10.128.0.0/14 actions=move:NXM_NX_TUN_ID[0..31]->NXM_NX_REG0[],goto_table:10 • table=10, tun_src=172.16.99.41
actions=goto_table:30 • table=30, ip,nw_dst=10.129.0.0/23 actions=goto_table:70 • table=70, ip,nw_dst=10.129.0.13 actions=load:0x465a2e->NXM_NX_REG1[],load:0x8->NXM_NX_REG2[],goto_table:80 • table=80, ip,reg0=0x465a2e,reg1=0xa06cb0,nw_dst=10.129.0.14 actions=output:NXM_NX_REG2[] kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: allow-from-proj1-client1 spec: podSelector: matchLabels: app: server2 ingress: - from: - namespaceSelector: matchLabels: name: proj1 Flow rules for [client1 → server1] on node2
OpenShift-SDNにおける NetworkPolicyの注意点 14 • NetworkPolicyオブジェクトに紐付かないPodは全通し • OpenShift-SDNでサポートするNetworkPolicyはv1 APIのみ ◦ Ingressのみサポート
◦ 以下は使えない ▪ Egress ▪ IPBlock ▪ namespaceSelectorとpodSelectorの両方指定 • podSelectorでPod個別に指定すると、マッチするPod(のIPアド レス)ごとにFlow entryが増える ◦ なるべくnamespaceSelector、もしくは空のpodSelectorで 指定する ◦ 細かいPod間の制御は最低限にする
CONFIDENTIAL Designator linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat Red Hat is the
world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you 15