本番環境のKubernetesマニフェストに 最低限必要な 7 のこと @ Japan Container Days v18.12 / jkd1812-prd-manifests

Kubernetesのマニフェストに記述可能な項目は多岐にわたります。本セッションでは、プロダクションで利用するにあたり、最低限設定する必要のある頻出パターンとして、下記の7項目について説明します。

コンテナのライフサイクル（起動時）
コンテナのライフサイクル（停止時）
ヘルスチェック
メンテナンスとアップデート
スケジューリング
リソースの割り当てと基準
カーネルパラメータのチューニング
インターネットからのアクセス制御

Transcript

1. Masaya Aoyama
   CyberAgent adtech studio
   ຊ൪؀ڥͷKubernetesϚχϑΣετʹ
   ࠷௿ݶඞཁͳ ͷ͜ͱ
   Japan Container Days v18.12
   MasayaAoyama @amsy810
   

   View Slide

2. Japan Container Days v18.04 Keynote
   Cloud Native Day Tokyo co-chair
   Cloud Native Meetup Tokyo Organizer (+ KubeCon)
   
   for Kubernetes
   CKA #138CKAD #2
   OpenStack / Kubernetes Contributor
   Masaya Aoyama (@amsy810)
   Infrastructure Engineer
   

   View Slide

3. Hobby == Kubernetes
   
   
   

   View Slide

4. Agenda
   page
   04
   • &#%32+/4
   • &#%3(0/4
   • %
   • !&&'
   • "'$&
   • $'*-,1
   •
   '%#!'"'&
   • &').
   

   View Slide

5. 
   
   spec.containers[].command
   spec.initContainers[]
   spec.containers[]
   spec.containers[].lifecycle.postStart
   

   View Slide

6. 0<3/>@:=
   page
   06
   1. Entrypoint
   +2/>@
   • $(( $*4"&5;
   2. Init Containers1?
   • $((
   1?"&-8)5;
   • $((
   1?"&. ,
   3. Sidecar !*(
   *;9
   • *( *;9
   4. postStart %#1?
   • $(#'6@7$((
   6@
   

   View Slide

7. 1. Entrypoint!
   "
   page
   07
   • Entrypoint!
   "
   • ex) #
   % $
   

   View Slide

8. 1. Entrypoint!
   "
   page
   08
   • Entrypoint!
   "
   • ex) #
   % $
   

   View Slide

9. 2. Init Container'
   page
   09
   • #&* ,
   • $("Volume%)
   • +
   -!
   Volume
   Store data Use data
   

   View Slide

10. 2. Init Container
    page
    010
    
    
    
    Immutable Infrastructure
    

    View Slide

11. 3. Sidecar %$
    page
    011
    • Pod
    Main , Sidecar
    • "'(
    Init Container !)*(
    %$ +
    • ex) #&(
    !&
    Volume
    Store data Use data
    

    View Slide

12. 4. postStart $
    page
    012
    •
    &%! () &*
    •
    (#"'
    postStart
    script
    main process (entrypoint)
    

    View Slide

13. !
    page
    013
    
    
    $
    %#
    1. entrypoint $ %
    2. initContainers $ %
    3. Sidecar $ "
    4. postStart $ %
    

    View Slide

14. 
    
    spec.restartPolicy
    spec.containers[].lifecycle.preStop
    Spec.terminationGracePeriodSeconds
    

    View Slide

15. spec.restartPolicy (Pod)
    page
    015
    • Always
    • Pod Pod
    • OnFailure
    • PodPod
    • Never
    • Pod
    Pod
    Deployment Always
    Job OnFailure or Never
    

    View Slide

16. 
    page
    016
    +0s
    Terminating
    SIGTERM
    preStop
    (optional)
    Service
    
    SIGKILL
    Running
    spec.terminationGracePeriodSeconds = 30
    (
    30 )
    ~ 30s
    

    View Slide

17. !
    page
    017
    +0s
    Terminating
    SIGTERM
    "
    preStop "
    Service
    &"
    SIGKILL "
    Running
    spec.terminationGracePeriodSeconds = 30
    ( 30 #)
    preStop "$
    2# SIGTERM "%
    +32s
    +30s
    

    View Slide

18. 
    spec.containers[].livenessProbe
    spec.containers[].readinessProbe
    

    View Slide

19. "-#
    
    page
    019
    spec.containers[].livenessProbe
    Check %)*!+
    0" (1
    ,'&/
    spec.containers[].readinessProbe
    Check %)* Service .$
    Service-In ,
    

    View Slide

20. 
    
    PodDisruptionBudget
    

    View Slide

21. ); $ &?
    page
    021
    kubectl drain' #$ $<+0@,>
    A7Pod4.
    -96*,>3…
    !%$%5/PDB
    =2(5/B
    "%-:18
    

    View Slide

22. 
    spec.nodeSelector
    spec.affinity.nodeAffinity
    spec.affinity.podAffinity
    spec.affinity.podAntiAffinity
    spec.torelations (+ taints)
    spec.priorityClassName (+ PriorityClass)
    

    View Slide

23. *(
    page
    023
    1. 2+0 Node Affinity5nodeSelector6
    • &/
    2. Node Affinity / Node Anti-Affinity
    • -3 ,!(#
    .'
    • )4,!"$,!
    3. Inter-Pod Affinity / Inter-Pod Anti-Affinity
    • .'Pod51%6
    

    View Slide

24. NodePool / Instance Group
    page
    024
    +#)% &$'
    • GKE NodePool
    • EKS Instance Group
    K8s "!
    *(Node
    

    View Slide

25. Kubernetes
    
    page
    025
    Taints / Tolerations PriorityClass
    

    View Slide

26. 
    
    spec.containers[].resources.requests
    spec.containers[].resources.limits
    LimitRange
    

    View Slide

27. $+
    page
    027
    Limits5"/3
    !.2
    Requests5"/ 3
    , #/
    *4 ! "(&
    ClusterAutoscaler Pending status Pod )'0%
    Requests
    -1
    

    View Slide

28. page
    028
    Kubernetes Node
    Allocatable CPU: 1000m
    Requests(%) ": 1000m
    & $: 10m
    Request(%) : 100m
    & #!
    
    
    Cluster Autoscale
    Requests '
    

    View Slide

29. page
    029
    Requests(&) #: 100m
    ' !%: 10000m
    Request(&) : 10m
    ' $"(
    
    
    Cluster Autoscale
    Requests
    Kubernetes Node
    Allocatable CPU:
    1000m
    

    View Slide

30. '+
    4/)
    page
    030
    • Requests 3%
    • Requests / Limits *5
    CPU
    .($#1-
    Requests / Limits/)
    !
    " &0 ,2!4/
    

    View Slide

31. LimitRange "8*-
    page
    031
    LimitRange
    (3%
    +$ / +&(3%
    requests / limits '73%
    GKE #!CPU Requests 100m 13%
    ,3%#!.74
    6
    #! OOM - CPU #!952/)
    Container: Pod
    7 0
    

    View Slide

32. 
    
    spec.securityContext.sysctls
    

    View Slide

33. 
    
    page
    033
    1. spec.securityContext.sysctls
    • Kubernetes v1.11
    2. Annotations (security.alpha.kubernetes.io/sysctls)
    • Kubernetes v1.10
    3. Privileged InitContainer
    •
    •
    

    View Slide

34. 
    
    page
    034
    Kubernetes v1.11 spec.securityContext.sysctls
    safe, unsafe
    

    View Slide

35. %
    page
    035
    Kubernetes v1.10 Annotation
    #
    security.alpha.kubernetes.io/sysctls'" &
    security.alpha.kubernetes.io/unsafe-sysctls'" & $!
    

    View Slide

36. 
    
    page
    036
    or
    InitContainer +
    
    
    

    View Slide

37. 
    
    spec.loadBalancerSourceRanges (Service)
    spec.ingress (NetworkPolicy)
    BackendConfig (Ingress [GKE])
    

    View Slide

38. !%2! &3
    page
    038
    1. “type: LoadBalancer” LB $ &3
    • LB$ &+!$ *4)
    2. NetworkPolicy Node $ &3
    • NetworkPolicy !#Node$ iptables &3
    • -.!" !1/
    3. Ingress LB $ &3
    • GKE ('BackendConfig # Cloud Armor
    0,
    

    View Slide

39. 1. ”type: LoadBalancer” Service
    page
    039
    
    
    IP Address
    Load
    Balancer
    NIC NIC
    iptables iptables
    

    View Slide

40. 2. Network Policy
    Node
    
    • In-bound / Out-bound
    • Label
    • Namespace
    Load
    Balancer
    NIC NIC
    iptables iptables
    page
    040
    

    View Slide

41. 3. Ingress %
    page
    041
    GCP
    Cloud Armor$
    Load
    Balancer
    NIC NIC
    iptables iptables
    Service
    Annotation #"
    !K8s
    
    

    View Slide

42. Conclusion
    summary
    

    View Slide

43. Conclusion
    page
    043
    • &#%32+/4
    • &#%3(0/4
    • %
    • !&&'
    • "'$&
    • $'*-,1
    •
    '%#!'"'&
    • &').
    

    View Slide

44. 
    
    

    View Slide

45. 3 I H uNs P
    /D 6 IBK c a io sy P
    0 C 1 DD 3 I H
    W3 I HS M
    3 I H
    & 72 uNs C ID
    9 CD H uNs
    0BH K L 4 uNs
    / B 8I uNs
    /D HI uNs 5 I I uNs
    uNs N snN
    s h l o y f fl
    y s N
    snr N
    tk ye
    & hs c N uNsu ghd
    v
    o y
    /2 /0
    fl N s 8 KB 5 HA
    3 I H dNkyl
    3 I H b
    

    View Slide

46. Thank you for your attention
    follow me: @amsy810
    

    View Slide