CloudNativeDaysTokyo 2019 slide

Transcript

1. 1.

   KubernetesΛӡ༻ͨ͜͠ͱ ͰֶΜͩΞϯνύλʔϯ גࣜձࣾαΠόʔΤʔδΣϯτ ٕज़ຊ෦ Private Cloud Group
 ޙ౻ल৴ !1

2. 2.

   Profile • ໊લ ޙ౻ ल৴ • ॴଐ ٕज़ຊ෦ PrivateCloudGroup ։ൃνʔϜ

   • ܦྺ 2014 / 04 ~ 2016/08 SIer 2016 / 09 ~ ݱࡏ ࡏ੶த • ۀ຿ Kubernetes, OpenShiftͷॾʑ !2
3. 3.

   1. ͜Ε·ͰͷPrivateCloudPlatform 2. Kubernetesͷߏ੒ 3. Ξϯνύλʔϯ8બ 4. ·ͱΊ !3

4. 4.

   ͜Ε·Ͱͷ PrivateCloudPlatform !4

5. 5.

   BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack

   Kilo
 Service VM BareMetal Kubernetes OpenStack Queens Service VM ͜Ε·ͰͷPrivateCloudPlatform 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த !5
6. 6.

   BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack

   Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த ͜Ε·ͰͷPrivateCloudPlatform !6
7. 7.

   BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack

   Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த ͜Ε·ͰͷPrivateCloudPlatform !7
8. 8.

   ͳͥͦ͜ʹKubernetesʁ !8

9. 9.

   BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack

   Kilo
 Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - - 2013 ͜Ε·ͰͷPrivateCloudPlatform !9
10. 10.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ !10
11. 11.

    ͜ͷΑ͏ͳ՝୊͕͋Γͭͭ
 ࣍ظPlatform͸Ͳ͏͠Α͏͔... !11

12. 12.

    !12

13. 13.

    ͋ͬͨ !13

14. 14.

    !14

15. 15.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ !15
16. 16.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ !16
17. 17.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ !17
18. 18.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ ContainerͰղܾ !18
19. 19.

    • ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ •

    όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ DaemonSetͰղܾ ConfigMapͰղܾ ContainerͰղܾ Helm౳Ͱղܾ !19
20. 20.

    Kubernetesͷߏ੒ !20

21. 21.

    ৄࡉ৘ใ ,VCFSOFUFT  %PDLFS  6CVOUV  *OTUBMM ,VCFBEN $POUSPM
    1MBOF

    ୆ 8PSLFS
    /PEF ໿୆ !21
22. 22.

    Control Plane Node - baremetal Worker Node (OpenStack: Compute Node)

    - baremetal Neutron-proxy Nova-compute Neutron- openvswitch VM Ceph Openstack master Kubernetes master Nginx-ingress etc... Prometheus Harbor fluentd / exporter VM VM Maxsacle GaleraCluster × 5 × 135 (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Plane Worker Node !22
23. 23.

    
 ՝୊Λղܾ !23

24. 24.

    ͜͜·Ͱ͸ྑ͔ͬͨ
 (໿1೥൒લ) !24

25. 25.

    ͠͹Βͯ͘͠ !25

26. 26.

    !26

27. 27.

    !27

28. 28.

    ͱ͍͏͜ͱͰ !28

29. 29.

    Ξϯνύλʔϯ8બ • ग़͘Θͨ͠ΞϯνύλʔϯΛ8ݸ΄Ͳ͓఻͑͠·͢ • όʔδϣϯʹΑͬͯ౰ͯ͸·Βͳ͍৔߹΋͋Γ·͢ ※ ղܾํ๏ͳͲ͸͋͘·ͰݸਓతͳݟղͰ͢ ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰ʹΑͬͯੜͨ͡ଛ֐౳ͷҰ੾ͷ੹೚Λෛ͍͔Ͷ·͢ͷͰ
 ྃ͝ঝ͍ͩ͘͞.

    ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰Λར༻͢Δ৔߹͸ࣗݾ੹೚Ͱ͓ئ͍͠·͢ !29
30. 30.

    Ξϯνύλʔϯ8બ !30

31. 31.

    Ξϯνύλʔϯ1
 Control PlaneʹͦͷଞPod΋૬৐Γ !31

32. 32.

    • ͳͥͦ͏ͨ͠ʁ - ࠓճ͸OpenStackͷͨΊͷKubernetes
 ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨ(ӡ༻తͳ໘Ͱ)
 (ϊʔυ਺ΛՄೳͳݶΓ཈͍͑ͨཉٻ͸Ͳ͜ʹͰ΋͋Δ͸ͣ) Kubernetes Control Plane Nova-XXXX

    kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Planeʹͦͷଞpod΋૬৐Γ !32
33. 33.

    • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes

    Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !33
34. 34.

    • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes

    Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Original App !34
35. 35.

    • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes

    Control Plane Nova-XXXX lube-apiserver Kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Original App !35
36. 36.

    • Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes

    Control Plane Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !36
37. 37.

    • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - Control PlaneʹͦͷଞͷPod͕৐ͤͳ͍Α͏ʹͨ͠ํ͕ྑ͍
 ؅ཧ༻ͷInfra Workerϊʔυ౳Λ༻ҙ͢Δ (σϑΥϧτͰ͸৐Βͳ͍) Kubernetes Control

    Plane Infra Worker Node Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !37
38. 38.

    • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - Control PlaneʹͦͷଞͷPod͕৐ͤͳ͍Α͏ʹͨ͠ํ͕ྑ͍
 ؅ཧ༻ͷInfra Workerϊʔυ౳Λ༻ҙ͢Δ (σϑΥϧτͰ͸৐Βͳ͍) Kubernetes Control

    Plane Infra Worker Node Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !38
39. 39.

    Ξϯνύλʔϯ2
 Ϋϥελ಺ͷϛχϚϜCeph !39

40. 40.

    • ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨʢӡ༻తͳ໘Ͱʣ - osdͱ͔mon͕མͪͯ΋ࣗಈͰ෮چͯ͘͠ΕΔ - PVCͱ͔Ͱ࢖͍͍ͨͳ.....
 - CephΛߏங͢ΔVM͕͍ͳ͍


    - ΦϒδΣΫτετϨʔδͱͯ͠࢖͍͍ͨͳ.... Ϋϥελ಺ͷϛχϚϜCeph !40
41. 41.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ࢒Γ
 ϝϯςத (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !41
42. 42.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ
 ϝϯςத !42
43. 43.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ
 Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ
 ϝϯςத !43
44. 44.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !44
45. 45.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !45
46. 46.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !46
47. 47.

    • Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ
 - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control

    Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ
 ϝϯςத !47
48. 48.

    Infra Worker Node osd - A Ceph-mon Infra Worker Node

    Infra Worker Node osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ɾɾN • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ϊʔυ਺͸ͨ͘͞Μ͋ͬͨํ͕ྑ͍ !48
49. 49.

    Ξϯνύλʔϯ3 Ϋϥελ಺ͷPrivate Registry !49

50. 50.

    • ͳͥͦ͏ͨ͠ʁ - Registryͷؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Kubernetes Control Plane Kubernetes

    
 Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane Ϋϥελ಺ͷPrivate Registry !50
51. 51.

    Kubernetes Control Plane Kubernetes 
 Control Plane XXXXXX kube-apiserver kube-controller

    kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ !51
52. 52.

    Kubernetes Control Plane Kubernetes 
 Control Plane XXXXXX kube-apiserver kube-controller

    kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ ͋Δϊʔυ͕μ΢ϯ
 (ϨδετϦͷؔ࿈ίϯϙʔωϯτ͕ଘࡏ) !52
53. 53.

    • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane

    Infra Worker Node kube-apiserver kube-controller kube-scheduler Registry Infra Worker Node XXXXXX XXXXXX ϨδετϦ͕෮چ͢Δ·Ͱ͸
 ϊʔυ্ʹPod͕࡞ΒΕͳ͍ XXXXXX XXXXXX Infra Worker Node XXXXXX XXXXXX Kubernetes 
 Control Plane Kubernetes 
 Control Plane Kubernetes 
 Control Plane !53
54. 54.

    • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍
 (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ 
 (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes

    Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ͋Δϊʔυ͕μ΢ϯ Kubernetes 
 Control Plane Kubernetes 
 Control Plane !54
55. 55.

    • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍
 (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ 
 (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes

    Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ଞͷϊʔυ্ʹpod͕࡞ΒΕΔ Kubernetes 
 Control Plane Kubernetes 
 Control Plane !55
56. 56.

    Ξϯνύλʔϯ4
 Ϋϥελ಺ͷPrometheus !56

57. 57.

    • ͳͥͦ͏ͨ͠ʁ - ؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Ϋϥελ಺ͷPrometheus !57

58. 58.

    • Կ͕ىͬͨ͜ - Ceph → Prometheus ͜ͷॱ൪Ͱར༻ෆՄʹͳͬͨ - ౰࣌֎ܗ͔Βͷ؂ࢹ͕ͳ͍ͷͰɺ্هͷ໰୊͕ى͖ͯ΋੩͔ͩͬͨ •

    Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ֎ܗ؂ࢹΛ͢ΔͳΓɺ֎͔ΒKubernetsͷAPIΛୟ͍ͯSDͰ؂ࢹ͢Δύλʔϯ͕ྑ͔ͬͨ ฐࣾͷத੢͕Prometheus Tokyo Meetup #2 ʹͯৄࡉ͸ൃදͯ͠·͢ (ࢿྉ) !58
59. 59.

    Ξϯνύλʔϯ5
 େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !59

60. 60.

    • ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ(ry • Կ͕ىͬͨ͜ - Ұ୆ʹ৐ΔPodͷྔ͕͔ͳΓଟ͘ͳΓɺҰ୆མͪͨͱ͖ͷΠϯύΫτ͕େ͖͘ͳͬͨ - ؾܰʹର৅ϊʔυͷϝϯςφϯεΛ͠ਏ͍

    - Ұ୆མͪͨͱ͖ʹPodͷߦ͖ઌ͕ͳ͘ͳ͍ͬͯͨ • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ͋Δఔ౓খ͞ͳϊʔυΛͨ͘͞Μฒ΂ͨํ͕ྑ͍(೔ʑͷӡ༻ϝϦοτ͸͋Δ) େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !60
61. 61.

    Ξϯνύλʔϯ6
 ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !61

62. 62.

    • ͳͥͦ͏ͨ͠ʁ - લఏͱͯ͠໰୊͕ى͖ͨͷ͸ComputeϊʔυͷϗετͰ
 VM༻ʹϦιʔεΛ֬อ͢Δ΂͘KubernetesͰ࢖͏Ϧιʔε͸ͳΔ΂͘࢖Θͳ͍Α͏ʹ͍ͯͨ͠
 ͕ϩάϩʔςͷ͜ͱΛ׬શʹࣦ೦͍ͯͨ͠ ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !62

63. 63.

    • Կ͕ىͬͨ͜ - PodσϓϩΠ࣌ʹPodͷঢ়ଶ͕Pendingʹͳͬͨ (σϑΥϧτͷϩάϩʔςͩͱ଍Γͳ͍) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - docker-logdriverͷઃఆΛద੾ʹ͢Δ΂͖(αΠζ΍ੈ୅؅ཧ) -

    ෆཁͳdebugϩάͳͲ͸ग़͞ͳ͍Α͏ʹ͢Δ !63
64. 64.

    Ξϯνύλʔϯ7
 in_tailͱϩάϩʔςͷར༻ !64

65. 65.

    • ͳͥͦ͏ͨ͠ʁ - ϩάͷංେʹΑΓPod͕εέδϡʔϦϯά͞Εͳ͍Α͏ʹͳͬͨͷͰ
 ϩάΛfluentdͰtailͭͭ͠΋1࣌ؒຖʹlogrotateΛಈ͔͍ͯͨ͠ in_tailͱϩάϩʔςͷར༻ !65

66. 66.

    • Կ͕ىͬͨ͜ - 1࣌ؒͷස౓ͰlogrotateΛͯ͠ɺಉ࣌ʹfluentdͰ”*”Ͱin_tail͍ͯ͠Δͱ fluentd͕ߴස౓ͰLimitʹ౰ͨͬͯOOMͰkill͞ΕΔΑ͏ʹͳͬͨ ௚઀ؔ܎͋Δ͔͸ௐ΂͖Εͯͳ͍͕ɺ͜ͷΑ͏ͳissues΋͋Γ (https://github.com/fluent/fluentd-kubernetes-daemonset/issues/156) (https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_tail.rb#L255-L259) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ

    - ϩάϩʔςΛ͢ΔλΠϛϯά͸௨ৗ௨Γʹ͠ɺੈ୅Λগͳ͘͢ΔΑ͏ʹ͢Δ - in_tailΛར༻͠ͳ͍ํ๏ͰؤுΔ !66
67. 67.

    Ξϯνύλʔϯ8 ա౓ͳCronJobͷར༻ !67

68. 68.

    • ͳͥͦ͏ͨ͠ʁ - όονॲཧΛຖ෼ߦͳ͍ͬͯΔ΋ͷ͕ෳ਺͍ͨ - ͦͷଞόοΫΞοϓͳͲ΋جຊతʹ͸CronJobͰճ͍ͯͨ͠ - ͜ΕΛControl PlaneͰճ͍ͯͨ͠ ա౓ͳCronJobͷར༻

    !68
69. 69.

    • Կ͕ىͬͨ͜ - Podͷ্͛Լ͕͛සൟʹߦΘΕΔ͜ͱͰɺϗετଆͷbuffer/cache͕ղ์͞Εͳ͍Ͱ free memory͕0%ʹͳΔ͜ͱͰϗετμ΢ϯ͕ൃੜͨ͠ ·ͨɺ/sys/fs/cgroup/memory഑Լ͕ҟৗʹͳ͍ͬͯͨ(Cannot Allocate Memory) -

    Control PlaneͰ͜ΕΒ͕ಈ͍͍ͯͨͨΊසൟʹControl Plane͕ෆ҆ఆʹͳͬͨ • Ͳ͏ͨ͠ํ͕ྑ͔͔ͬͨ - ৔౰ͨΓతͳରԠͱͳ͍ͬͯΔ͕ɺ ͦͷଞͷPodʹӨڹΛٴ΅͞ͳ͍Α͏ʹCronJobΛ࣮ߦ͢Δϊʔυ͸labelΛར༻ͯ͠ɺ CronJobՔಇઐ༻ͷWorker Nodeͱͯ͠ෳ਺୆Քಇ͍ͤͯ͞Δ !69
70. 70.

    ະͩରԠதͷ෦෼΋͋Γ
 ӡ༻Ͱؤு͍ͬͯΔ ߲໨΋͋Γ·͢ !70

71. 71.

    Ҏ্
 Ξϯνύλʔϯ8બͰͨ͠ !71

72. 72.

    ·ͱΊ !72

73. 73.

    ·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺKubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋
 ৭ʑͱײ͍ͯ͡Δ

    !73
74. 74.

    ·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺkubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋
 ৭ʑͱײ͍ͯ͡Δ

    ༻๏ɾ༰ྔΛकͬͯద੾ʹ !74
75. 75.

    ͋Γ͕ͱ͏͍͟͝·ͨ͠ !75