Amazon EKS上でのVNF開発奮闘記

by Kenta Shinohara

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

VPC (Virtual Private Cloud) public-plane private-plane I-Gateway bastion NAT-Gateway EC2 EC2 EC2 EC2 RDS EKS Worker EKS Worker EKS Worker Endpoint Endpoint Copyright©2020 NTT Corp. All Rights Reserved. 8 NTT Tech Conf#4 @2020/1/31

Slide 9

Slide 9 text

VPC (Virtual Private Cloud) private-plane NAT-Gateway EC2 EC2 EC2 EC2 RDS EKS Worker EKS Worker EKS Worker Endpoint Endpoint public-plane I-Gateway bastion • インターネットリーチャビリティ：双方向 Copyright©2020 NTT Corp. All Rights Reserved. 9 NTT Tech Conf#4 @2020/1/31

Slide 10

Slide 10 text

VPC (Virtual Private Cloud) public-plane I-Gateway bastion • インターネットリーチャビリティ：外方向のみ • 外抜け通信の細かな制御はNetworkACL + security group NAT-Gateway EC2 EC2 EC2 EC2 RDS EKS Worker EKS Worker EKS Worker Endpoint Endpoint Copyright©2020 NTT Corp. All Rights Reserved. 10 NTT Tech Conf#4 @2020/1/31 private-plane

Slide 11

Slide 11 text

VPC (Virtual Private Cloud) public-plane I-Gateway bastion NAT-Gateway private-plane RDS EKS Worker EKS Worker EKS Worker Endpoint Endpoint EC2 EC2 EC2 EC2 • 静的なIPアドレスを実施するサブネット（VNFはIPアドレス狙い撃ちなものが多いので） Copyright©2020 NTT Corp. All Rights Reserved. 11 NTT Tech Conf#4 @2020/1/31

Slide 12

Slide 12 text

VPC (Virtual Private Cloud) public-plane I-Gateway bastion NAT-Gateway private-plane EC2 EC2 EC2 EC2 • IPアドレスを動的に払い出すサブネット（マネージドサービス等） EKSのデフォルトCNI:amazon-vpc-cniはPodに対しこのサブネットからIPアドレスを直接払い出す RDS EKS Worker EKS Worker EKS Worker Endpoint Endpoint Copyright©2020 NTT Corp. All Rights Reserved. 12 NTT Tech Conf#4 @2020/1/31

Slide 13

Slide 13 text

VPC (Virtual Private Cloud) public-plane I-Gateway bastion NAT-Gateway private-plane EC2 EC2 EC2 EC2 RDS Endpoint Endpoint • Worker(コンテナが稼働するEC2インスタンス)⇔Master（EKS）間で常時通信 • EKS API serverはVPC外にあるため、VPC外への通信が発生してしまう EKS Worker EKS Worker EKS Worker EKS master (API server) VPC外への通信が発生 ECR (container registry) Copyright©2020 NTT Corp. All Rights Reserved. 13 NTT Tech Conf#4 @2020/1/31

Slide 14

Slide 14 text

VPC (Virtual Private Cloud) public-plane I-Gateway bastion NAT-Gateway private-plane EC2 EC2 EC2 EC2 RDS Endpoint Endpoint EKSの閉域利用はver 1.13から対応 • kubectl<->EKS API serverのprivate化 (2019/3) • EKS Worker⇔master間通信のprivate化(2019/6) • EKS Worker ⇔ ECR間通信のprivate化(2019/6) • CloudFormationでの完全private EKSクラスタ生成は公式ドキュメント・手順は存在しないが，実施方法有り EKS Worker EKS Worker EKS Worker EKS master (API server) 閉域通信(PrivateLink接続) ECR (container registry) Copyright©2020 NTT Corp. All Rights Reserved. 14 NTT Tech Conf#4 @2020/1/31

Slide 15

Slide 15 text

Radius client Radius server Radius server Radius server LB 10.0.0.5 10.0.0.6 10.0.2.123 10.0.2.141 10.0.2.138 10.0.2.139 10.0.2.142 Copyright©2020 NTT Corp. All Rights Reserved. 15 NTT Tech Conf#4 @2020/1/31

Slide 16

Slide 16 text

Radius client LB Radius request/response from radius client Radius request/response from radius server NAPT 10.0.0.5 10.0.0.6 10.0.2.123 10.0.2.141 Radius server Radius server Radius server 10.0.2.138 10.0.2.139 10.0.2.142 Copyright©2020 NTT Corp. All Rights Reserved. 16 NTT Tech Conf#4 @2020/1/31

Slide 17

Slide 17 text

Radius client LB Error: unknown server Radius request/response from radius client Radius request/response from radius server 10.0.0.5 10.0.0.6 10.0.2.123 10.0.2.141 NAPT Radius server Radius server Radius server 10.0.2.138 10.0.2.139 10.0.2.142 Copyright©2020 NTT Corp. All Rights Reserved. 17 NTT Tech Conf#4 @2020/1/31

Slide 18

Slide 18 text

Radius client LB Radius request/response from radius client Radius request/response from radius server 10.0.0.5 10.0.0.6 10.0.2.123 10.0.2.141 NAPT Radius server Radius server Radius server 10.0.2.138 10.0.2.139 10.0.2.142 PodからEKS外への通信経路をLBを経由する様に変更 & マネージドLBではこれが出来ないので、 NGINXを使い構築 (※type:LoadBalancerが使えないので、振分先の動的設定等の機能実装が必要) Copyright©2020 NTT Corp. All Rights Reserved. 18 NTT Tech Conf#4 @2020/1/31