本番環境のKubernetesマニフェストに 最低限必要な 7 のこと @ Japan Container Days v18.12 / jkd1812-prd-manifests

本番環境のKubernetesマニフェストに 最低限必要な 7 のこと @ Japan Container Days v18.12 / jkd1812-prd-manifests

Kubernetesのマニフェストに記述可能な項目は多岐にわたります。本セッションでは、プロダクションで利用するにあたり、最低限設定する必要のある頻出パターンとして、下記の7項目について説明します。

コンテナのライフサイクル(起動時)
コンテナのライフサイクル(停止時)
ヘルスチェック
メンテナンスとアップデート
スケジューリング
リソースの割り当てと基準
カーネルパラメータのチューニング
インターネットからのアクセス制御

De266761b955b2636e454a1bc7a99ed4?s=128

Masaya Aoyama (@amsy810)

December 04, 2018
Tweet

Transcript

  1. Masaya Aoyama CyberAgent adtech studio ຊ൪؀ڥͷKubernetesϚχϑΣετʹ ࠷௿ݶඞཁͳ  ͷ͜ͱ Japan

    Container Days v18.12 MasayaAoyama @amsy810
  2. Japan Container Days v18.04 Keynote  Cloud Native Day Tokyo

    co-chair Cloud Native Meetup Tokyo Organizer (+ KubeCon)    for Kubernetes CKA #138CKAD #2 OpenStack / Kubernetes Contributor Masaya Aoyama (@amsy810) Infrastructure Engineer
  3. Hobby == Kubernetes    

  4. Agenda page 04 • &# %32+/4 • &# %3(0/4 •

    %  • !&&' •  "'$& • $'*-,1 • '%#!'"'& • &' ).
  5.      spec.containers[].command spec.initContainers[] spec.containers[] spec.containers[].lifecycle.postStart

  6. 0<3/>@:= page 06 1. Entrypoint +2 />@ • $(( $*

    4"&5; 2. Init Containers1? • $(( 1?"&-8)5; • $(( 1? "&. , 3. Sidecar !*( *;9 • *( *;9 4. postStart %#1? • $(#'6@7 $(( 6@
  7. 1. Entrypoint! " page 07 • Entrypoint! " • ex)

    #   %  $
  8. 1. Entrypoint! " page 08 • Entrypoint! " • ex)

    #   %  $
  9. 2. Init Container' page 09 • #&*  , •

    $("Volume %) • +    -! Volume Store data Use data
  10. 2. Init Container page 010     

    Immutable Infrastructure
  11. 3. Sidecar  %$ page 011 • Pod  Main

    , Sidecar  • "'( Init Container !)*( %$ + • ex)  #&(  !& Volume Store data Use data
  12. 4. postStart  $ page 012 •   &%!

    ()  &* •   (#"' postStart script main process (entrypoint)
  13. !  page 013     $ %#

    1. entrypoint $ % 2. initContainers $ % 3. Sidecar $ " 4. postStart $ %
  14.       spec.restartPolicy spec.containers[].lifecycle.preStop Spec.terminationGracePeriodSeconds

  15. spec.restartPolicy (Pod) page 015 • Always • Pod  Pod

     • OnFailure • PodPod  • Never • Pod Pod    Deployment Always  Job OnFailure or Never 
  16.     page 016 +0s Terminating SIGTERM 

    preStop  (optional) Service   SIGKILL  Running spec.terminationGracePeriodSeconds = 30 (   30 ) ~ 30s
  17.  !  page 017 +0s Terminating SIGTERM " preStop

    " Service  &" SIGKILL " Running spec.terminationGracePeriodSeconds = 30 ( 30 #) preStop "$  2# SIGTERM "% +32s +30s
  18.  spec.containers[].livenessProbe spec.containers[].readinessProbe

  19.  "-#  page 019 spec.containers[].livenessProbe Check %)*!+ 0" (1

    ,'&/  spec.containers[].readinessProbe Check %)* Service .$ Service-In ,
  20.     PodDisruptionBudget

  21. ); $ &? page 021 kubectl drain' #$ $<+0@,> A7Pod4.

    -96*,>3… !%$%5/PDB =2(5/B "%-:18
  22.  spec.nodeSelector spec.affinity.nodeAffinity spec.affinity.podAffinity spec.affinity.podAntiAffinity spec.torelations (+ taints) spec.priorityClassName (+

    PriorityClass)
  23. *( page 023 1. 2+0 Node Affinity5nodeSelector6 •  &

    / 2. Node Affinity / Node Anti-Affinity • -3 ,!(# .'   • )4,!"$,! 3. Inter-Pod Affinity / Inter-Pod Anti-Affinity • .' Pod51%6 
  24. NodePool / Instance Group page 024 +#)%   &$'

    • GKE  NodePool • EKS  Instance Group K8s  "!    *(Node
  25. Kubernetes  page 025  Taints / Tolerations  PriorityClass

    
  26.    spec.containers[].resources.requests spec.containers[].resources.limits LimitRange

  27. $+ page 027 Limits5"/3 !.2   Requests5"/ 3 ,

    #/ *4 ! " (& ClusterAutoscaler  Pending status  Pod )'0% Requests -1
  28. page 028 Kubernetes Node Allocatable CPU: 1000m Requests(%) ": 1000m

    &  $: 10m Request(%) : 100m & #!     Cluster Autoscale  Requests '
  29. page 029 Requests(&) #: 100m ' !%: 10000m Request(&) :

    10m ' $" (   Cluster Autoscale   Requests   Kubernetes Node Allocatable CPU: 1000m
  30. '+ 4/) page 030 • Requests 3%  • Requests

    / Limits * 5   CPU .($#1-  Requests / Limits/) ! " &0 ,2 !4/
  31. LimitRange  "8*- page 031 LimitRange  (3% +$ /

    +& (3% requests / limits '73% GKE #!CPU  Requests  100m 13% ,3%#!.74 6 #! OOM -  CPU #!952/) Container: Pod 7 0
  32.      spec.securityContext.sysctls

  33.    page 033 1. spec.securityContext.sysctls • Kubernetes v1.11

     2. Annotations (security.alpha.kubernetes.io/sysctls) • Kubernetes v1.10  3. Privileged InitContainer •    • 
  34.    page 034 Kubernetes v1.11  spec.securityContext.sysctls 

     safe, unsafe
  35. %  page 035 Kubernetes v1.10  Annotation # security.alpha.kubernetes.io/sysctls'"

     & security.alpha.kubernetes.io/unsafe-sysctls'"  & $!
  36.    page 036  or  InitContainer +

        
  37.    spec.loadBalancerSourceRanges (Service) spec.ingress (NetworkPolicy) BackendConfig (Ingress [GKE])

  38.  ! %2! &3 page 038 1. “type: LoadBalancer” LB

    $ &3 • LB$ &+!$ *4)  2. NetworkPolicy Node $ &3 • NetworkPolicy !#Node$ iptables &3 • -.!" !1/ 3. Ingress LB $ &3 • GKE ('BackendConfig # Cloud Armor 0,
  39. 1. ”type: LoadBalancer” Service  page 039   

     IP Address  Load Balancer NIC NIC iptables iptables
  40. 2. Network Policy   Node   • In-bound

    / Out-bound • Label   • Namespace   Load Balancer NIC NIC iptables iptables page 040
  41. 3. Ingress % page 041 GCP Cloud Armor$ Load Balancer

    NIC NIC iptables iptables  Service  Annotation #"   !K8s 
  42. Conclusion summary

  43. Conclusion page 043 • &# %32+/4 • &# %3(0/4 •

    %  • !&&' •  "'$& • $'*-,1 • '%#!'"'& • &' ).
  44.   

  45. 3 I H uNs P /D 6 IBK c a

    io sy P   0 C 1 DD 3 I H W3 I HS M 3 I H & 72 uNs C ID 9 CD H uNs 0BH K L 4 uNs / B 8I uNs /D HI uNs 5 I I uNs uNs N snN s h l o y f fl y s N snr N tk ye & hs c N uNsu ghd v o y /2 /0 fl N s 8 KB 5 HA 3 I H dNkyl 3 I H b
  46. Thank you for your attention follow me: @amsy810