CloudNativeDaysTokyo 2019 slide

by hidenobu goto

Slide 1

Slide 1 text

KubernetesΛӡ༻ͨ͜͠ͱ ͰֶΜͩΞϯνύλʔϯ גࣜձࣾαΠόʔΤʔδΣϯτ ٕज़ຊ෦ Private Cloud Group  ޙ౻ल৴ !1

Slide 2

Slide 2 text

Proﬁle • ໊લ ޙ౻ ल৴ • ॴଐ ٕज़ຊ෦ PrivateCloudGroup ։ൃνʔϜ • ܦྺ 2014 / 04 ~ 2016/08 SIer 2016 / 09 ~ ݱࡏ ࡏ੶த • ۀ຿ Kubernetes, OpenShiftͷॾʑ !2

Slide 3

Slide 3 text

1. ͜Ε·ͰͷPrivateCloudPlatform 2. Kubernetesͷߏ੒ 3. Ξϯνύλʔϯ8બ 4. ·ͱΊ !3

Slide 4

Slide 4 text

͜Ε·Ͱͷ PrivateCloudPlatform !4

Slide 5

Slide 5 text

BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo  Service VM BareMetal Kubernetes OpenStack Queens Service VM ͜Ε·ͰͷPrivateCloudPlatform 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த !5

Slide 6

Slide 6 text

BareMetal Service (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ BareMetal ಺੡Ϋϥ΢υ Service VM BareMetal OpenStack Kilo  Service VM BareMetal Kubernetes OpenStack Queen Service VM 2013 - 2019 2015 - 2021 2019 - ݱࡏ΋Ұ෦ར༻த ͜Ε·ͰͷPrivateCloudPlatform !6

Slide 7

Slide 7 text

Slide 8

Slide 8 text

ͳͥͦ͜ʹKubernetesʁ !8

Slide 9

Slide 9 text

Slide 10

Slide 10 text

• ComputeϊʔυΛ৽ن௥Ճ͢Δͷʹ͕͔͔࣌ؒΔ(20-30෼) • ઃఆมߋΛ͢Δ৔߹΋Ansible४උͯ͠.... • ࠓճ͸dynamic inventory࢖͑ͳ͍͔ΒIP͖࣋ͬͯͯ... • ͋Εɺ͜ͷComputeϊʔυ͚ͩconf͕[changed]ʹͳͬͨΑʁ • όΠφϦมߋ͍ͨ͠Μ͚ͩͲґଘͱ͔৭ʑେৎ෉?? • devͱ؀ڥ͕ҧ͏ͷͰɺ͜ͷϊʔυ͸ςετϊʔυͱͯ͠௥Ճͯ͠ݕূ • ਃ͠ૹΒΕͨڊେͳAnsibleྲྀ͢ͷυΩυΩ͢Δ ͜Ε·Ͱ՝୊ !10

Slide 11

Slide 11 text

͜ͷΑ͏ͳ՝୊͕͋Γͭͭ  ࣍ظPlatform͸Ͳ͏͠Α͏͔... !11

Slide 12

Slide 12 text

!12

Slide 13

Slide 13 text

͋ͬͨ !13

Slide 14

Slide 14 text

!14

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Kubernetesͷߏ੒ !20

Slide 21

Slide 21 text

ৄࡉ৘ใ ,VCFSOFUFT %PDLFS 6CVOUV *OTUBMM ,VCFBEN $POUSPM1MBOF ୆ 8PSLFS/PEF ໿୆ !21

Slide 22

Slide 22 text

Control Plane Node - baremetal Worker Node (OpenStack: Compute Node) - baremetal Neutron-proxy Nova-compute Neutron- openvswitch VM Ceph Openstack master Kubernetes master Nginx-ingress etc... Prometheus Harbor ﬂuentd / exporter VM VM Maxsacle GaleraCluster × 5 × 135 (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Plane Worker Node !22

Slide 23

Slide 23 text

  ՝୊Λղܾ !23

Slide 24

Slide 24 text

͜͜·Ͱ͸ྑ͔ͬͨ  (໿1೥൒લ) !24

Slide 25

Slide 25 text

͠͹Βͯ͘͠ !25

Slide 26

Slide 26 text

!26

Slide 27

Slide 27 text

!27

Slide 28

Slide 28 text

ͱ͍͏͜ͱͰ !28

Slide 29

Slide 29 text

Ξϯνύλʔϯ8બ • ग़͘Θͨ͠ΞϯνύλʔϯΛ8ݸ΄Ͳ͓఻͑͠·͢ • όʔδϣϯʹΑͬͯ౰ͯ͸·Βͳ͍৔߹΋͋Γ·͢ ※ ղܾํ๏ͳͲ͸͋͘·ͰݸਓతͳݟղͰ͢ ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰ʹΑͬͯੜͨ͡ଛ֐౳ͷҰ੾ͷ੹೚Λෛ͍͔Ͷ·͢ͷͰ  ྃ͝ঝ͍ͩ͘͞. ※ ຊࢿྉʹܝࡌ͞Εͨ಺༰Λར༻͢Δ৔߹͸ࣗݾ੹೚Ͱ͓ئ͍͠·͢ !29

Slide 30

Slide 30 text

Ξϯνύλʔϯ8બ !30

Slide 31

Slide 31 text

Ξϯνύλʔϯ1  Control PlaneʹͦͷଞPod΋૬৐Γ !31

Slide 32

Slide 32 text

• ͳͥͦ͏ͨ͠ʁ - ࠓճ͸OpenStackͷͨΊͷKubernetes  ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨ(ӡ༻తͳ໘Ͱ)  (ϊʔυ਺ΛՄೳͳݶΓ཈͍͑ͨཉٻ͸Ͳ͜ʹͰ΋͋Δ͸ͣ) Kubernetes Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Control Planeʹͦͷଞpod΋૬৐Γ !32

Slide 33

Slide 33 text

• Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !33

Slide 34

Slide 34 text

Slide 35

Slide 35 text

• Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Nova-XXXX lube-apiserver Kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ Original App !35

Slide 36

Slide 36 text

• Կ͕ىͬͨ͜ - ͔֬ʹ෺ཧαʔόΛ؅ཧ͢Δྔ͸গͳ͘ͳͬͨ - Control PlaneͷϝϯςΛߦ͏ͷ͕ඇৗʹ೉͘͠ͳͬͨ - ͦͷଞPodͷෆ۩߹ͷϝϞϦϦʔΫͳͲͰϗετ͕ෆௐʹͳΔͱӨڹΛ΋Ζʹड͚Δ Kubernetes Control Plane Compute Worker Node Nova-XXXX Neutron-XXXX VM VM Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !36

Slide 37

Slide 37 text

• Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - Control PlaneʹͦͷଞͷPod͕৐ͤͳ͍Α͏ʹͨ͠ํ͕ྑ͍  ؅ཧ༻ͷInfra Workerϊʔυ౳Λ༻ҙ͢Δ (σϑΥϧτͰ͸৐Βͳ͍) Kubernetes Control Plane Infra Worker Node Nova-XXXX kube-apiserver kube-controller kube-scheduler Neutron-XXXX Cinder-XXXX Compute Worker Node Nova-XXXX Neutron-XXXX VM VM (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !37

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Ξϯνύλʔϯ2  Ϋϥελ಺ͷϛχϚϜCeph !39

Slide 40

Slide 40 text

• ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ؅ཧͨ͘͠ͳ͔ͬͨʢӡ༻తͳ໘Ͱʣ - osdͱ͔mon͕མͪͯ΋ࣗಈͰ෮چͯ͘͠ΕΔ - PVCͱ͔Ͱ࢖͍͍ͨͳ.....  - CephΛߏங͢ΔVM͕͍ͳ͍  - ΦϒδΣΫτετϨʔδͱͯ͠࢖͍͍ͨͳ.... Ϋϥελ಺ͷϛχϚϜCeph !40

Slide 41

Slide 41 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ࢒Γ  ϝϯςத (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ !41

Slide 42

Slide 42 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ  ϝϯςத !42

Slide 43

Slide 43 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ  Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack OpenStack ࢒Γ  ϝϯςத !43

Slide 44

Slide 44 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ  - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ  ϝϯςத !44

Slide 45

Slide 45 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ  - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ  ϝϯςத !45

Slide 46

Slide 46 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ  - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ  ϝϯςத !46

Slide 47

Slide 47 text

• Կ͕ىͬͨ͜ - ϊʔυͷϝϯςͷ࣮ࢪ + ো֐౳ͰCephͷΫϥελ͕յΕͨ  - PVCΛར༻͍ͯ͠ΔPod͕མͪɺ࠷ऴతʹOpenStack΋࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane osd - A Ceph-mon Kubernetes Control Plane Kubernetes Control Plane osd - A Ceph-mon Ceph-mgr DatabaseA DatabaseB DatabaseA (஫) ਤ͸ΠϝʔδͷͨΊߏ੒΍഑ஔ͸ݫີʹ͸ҟͳΓ·͢ OpenStack OpenStack ࢒Γ  ϝϯςத !47

Slide 48

Slide 48 text

Infra Worker Node osd - A Ceph-mon Infra Worker Node Infra Worker Node osd - A Ceph-mon osd - A Ceph-mon Ceph-mgr ɾɾN • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ϊʔυ਺͸ͨ͘͞Μ͋ͬͨํ͕ྑ͍ !48

Slide 49

Slide 49 text

Ξϯνύλʔϯ3 Ϋϥελ಺ͷPrivate Registry !49

Slide 50

Slide 50 text

• ͳͥͦ͏ͨ͠ʁ - Registryͷؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Kubernetes Control Plane Kubernetes   Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes   Control Plane Ϋϥελ಺ͷPrivate Registry !50

Slide 51

Slide 51 text

Kubernetes Control Plane Kubernetes   Control Plane XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX Kubernetes   Control Plane • Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ !51

Slide 52

Slide 52 text

Slide 53

Slide 53 text

• Կ͕ىͬͨ͜ - Registry͕ଘࡏ͢Δϊʔυ͕མͪͨ৔߹ɺRegistry͕෮چ͢Δ·ͰΠϝʔδΛϓϧͰ͖ͳ͍ ৔߹͕͋Γɺ௨ৗΑΓ෮چʹ͕͔͔࣌ؒΔ৔߹΋͋Δ - Ceph่͕յͨ͠λΠϛϯάͰ࢖͑ͳ͘ͳͬͨ Kubernetes Control Plane Infra Worker Node kube-apiserver kube-controller kube-scheduler Registry Infra Worker Node XXXXXX XXXXXX ϨδετϦ͕෮چ͢Δ·Ͱ͸  ϊʔυ্ʹPod͕࡞ΒΕͳ͍ XXXXXX XXXXXX Infra Worker Node XXXXXX XXXXXX Kubernetes   Control Plane Kubernetes   Control Plane Kubernetes   Control Plane !53

Slide 54

Slide 54 text

• Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍  (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ   (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ͋Δϊʔυ͕μ΢ϯ Kubernetes   Control Plane Kubernetes   Control Plane !54

Slide 55

Slide 55 text

• Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - k8sΫϥελͷ֎·ͨ͸ผk8sΫϥελʔͷ΋ͷΛར༻ͨ͠ํ͕ࣗ༝౓͸ߴ͍  (Ҡઃ΋ߦ͍΍͍͢) - ৑௕͠΍͍͢RegistryΛ࢖͏ඞཁ͕͋Δ   (όοΫΤϯυʹDB͕͋Δ৔߹͸ͦΕΒ΋ؚΊͯ) Kubernetes Control Plane Infra Worker Node XXXXXX kube-apiserver kube-controller kube-scheduler XXXXXX Registry Infra Worker Node XXXXXX XXXXXX ଞͷϊʔυ্ʹpod͕࡞ΒΕΔ Kubernetes   Control Plane Kubernetes   Control Plane !55

Slide 56

Slide 56 text

Ξϯνύλʔϯ4  Ϋϥελ಺ͷPrometheus !56

Slide 57

Slide 57 text

• ͳͥͦ͏ͨ͠ʁ - ؔ࿈ίϯϙʔωϯτ͕མͪͯ΋࠶ىಈͯ͘͠ΕΔ͔Β͖ͬͱָ - CephͷPVC࢖͑͹؆୯Ͱ͸ʁʁ Ϋϥελ಺ͷPrometheus !57

Slide 58

Slide 58 text

• Կ͕ىͬͨ͜ - Ceph → Prometheus ͜ͷॱ൪Ͱར༻ෆՄʹͳͬͨ - ౰࣌֎ܗ͔Βͷ؂ࢹ͕ͳ͍ͷͰɺ্هͷ໰୊͕ى͖ͯ΋੩͔ͩͬͨ • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ֎ܗ؂ࢹΛ͢ΔͳΓɺ֎͔ΒKubernetsͷAPIΛୟ͍ͯSDͰ؂ࢹ͢Δύλʔϯ͕ྑ͔ͬͨ ฐࣾͷத੢͕Prometheus Tokyo Meetup #2 ʹͯৄࡉ͸ൃදͯ͠·͢ (ࢿྉ) !58

Slide 59

Slide 59 text

Ξϯνύλʔϯ5  େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !59

Slide 60

Slide 60 text

• ͳͥͦ͏ͨ͠ʁ - ComputeϊʔυͰ͸ͳ͍෺ཧαʔόΛ(ry • Կ͕ىͬͨ͜ - Ұ୆ʹ৐ΔPodͷྔ͕͔ͳΓଟ͘ͳΓɺҰ୆མͪͨͱ͖ͷΠϯύΫτ͕େ͖͘ͳͬͨ - ؾܰʹର৅ϊʔυͷϝϯςφϯεΛ͠ਏ͍ - Ұ୆མͪͨͱ͖ʹPodͷߦ͖ઌ͕ͳ͘ͳ͍ͬͯͨ • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ͋Δఔ౓খ͞ͳϊʔυΛͨ͘͞Μฒ΂ͨํ͕ྑ͍(೔ʑͷӡ༻ϝϦοτ͸͋Δ) େ͖ͳWorker NodeΛগͳ͍਺ͰͲʔΜ !60

Slide 61

Slide 61 text

Ξϯνύλʔϯ6  ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !61

Slide 62

Slide 62 text

• ͳͥͦ͏ͨ͠ʁ - લఏͱͯ͠໰୊͕ى͖ͨͷ͸ComputeϊʔυͷϗετͰ  VM༻ʹϦιʔεΛ֬อ͢Δ΂͘KubernetesͰ࢖͏Ϧιʔε͸ͳΔ΂͘࢖Θͳ͍Α͏ʹ͍ͯͨ͠  ͕ϩάϩʔςͷ͜ͱΛ׬શʹࣦ೦͍ͯͨ͠ ίϯςφͷϩάϩʔςΛ͍ͯ͠ͳ͍ !62

Slide 63

Slide 63 text

• Կ͕ىͬͨ͜ - PodσϓϩΠ࣌ʹPodͷঢ়ଶ͕Pendingʹͳͬͨ (σϑΥϧτͷϩάϩʔςͩͱ଍Γͳ͍) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - docker-logdriverͷઃఆΛద੾ʹ͢Δ΂͖(αΠζ΍ੈ୅؅ཧ) - ෆཁͳdebugϩάͳͲ͸ग़͞ͳ͍Α͏ʹ͢Δ !63

Slide 64

Slide 64 text

Ξϯνύλʔϯ7  in_tailͱϩάϩʔςͷར༻ !64

Slide 65

Slide 65 text

• ͳͥͦ͏ͨ͠ʁ - ϩάͷංେʹΑΓPod͕εέδϡʔϦϯά͞Εͳ͍Α͏ʹͳͬͨͷͰ  ϩάΛﬂuentdͰtailͭͭ͠΋1࣌ؒຖʹlogrotateΛಈ͔͍ͯͨ͠ in_tailͱϩάϩʔςͷར༻ !65

Slide 66

Slide 66 text

• Կ͕ىͬͨ͜ - 1࣌ؒͷස౓ͰlogrotateΛͯ͠ɺಉ࣌ʹfluentdͰ”*”Ͱin_tail͍ͯ͠Δͱ fluentd͕ߴස౓ͰLimitʹ౰ͨͬͯOOMͰkill͞ΕΔΑ͏ʹͳͬͨ ௚઀ؔ܎͋Δ͔͸ௐ΂͖Εͯͳ͍͕ɺ͜ͷΑ͏ͳissues΋͋Γ (https://github.com/fluent/fluentd-kubernetes-daemonset/issues/156) (https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_tail.rb#L255-L259) • Ͳ͏ͨ͠ํ͕ྑ͔ͬͨʁ - ϩάϩʔςΛ͢ΔλΠϛϯά͸௨ৗ௨Γʹ͠ɺੈ୅Λগͳ͘͢ΔΑ͏ʹ͢Δ - in_tailΛར༻͠ͳ͍ํ๏ͰؤுΔ !66

Slide 67

Slide 67 text

Ξϯνύλʔϯ8 ա౓ͳCronJobͷར༻ !67

Slide 68

Slide 68 text

• ͳͥͦ͏ͨ͠ʁ - όονॲཧΛຖ෼ߦͳ͍ͬͯΔ΋ͷ͕ෳ਺͍ͨ - ͦͷଞόοΫΞοϓͳͲ΋جຊతʹ͸CronJobͰճ͍ͯͨ͠ - ͜ΕΛControl PlaneͰճ͍ͯͨ͠ ա౓ͳCronJobͷར༻ !68

Slide 69

Slide 69 text

• Կ͕ىͬͨ͜ - Podͷ্͛Լ͕͛සൟʹߦΘΕΔ͜ͱͰɺϗετଆͷbuffer/cache͕ղ์͞Εͳ͍Ͱ free memory͕0%ʹͳΔ͜ͱͰϗετμ΢ϯ͕ൃੜͨ͠ ·ͨɺ/sys/fs/cgroup/memory഑Լ͕ҟৗʹͳ͍ͬͯͨ(Cannot Allocate Memory) - Control PlaneͰ͜ΕΒ͕ಈ͍͍ͯͨͨΊසൟʹControl Plane͕ෆ҆ఆʹͳͬͨ • Ͳ͏ͨ͠ํ͕ྑ͔͔ͬͨ - ৔౰ͨΓతͳରԠͱͳ͍ͬͯΔ͕ɺ ͦͷଞͷPodʹӨڹΛٴ΅͞ͳ͍Α͏ʹCronJobΛ࣮ߦ͢Δϊʔυ͸labelΛར༻ͯ͠ɺ CronJobՔಇઐ༻ͷWorker Nodeͱͯ͠ෳ਺୆Քಇ͍ͤͯ͞Δ !69

Slide 70

Slide 70 text

ະͩରԠதͷ෦෼΋͋Γ  ӡ༻Ͱؤு͍ͬͯΔ ߲໨΋͋Γ·͢ !70

Slide 71

Slide 71 text

Ҏ্  Ξϯνύλʔϯ8બͰͨ͠ !71

Slide 72

Slide 72 text

·ͱΊ !72

Slide 73

Slide 73 text

·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺKubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋  ৭ʑͱײ͍ͯ͡Δ !73

Slide 74

Slide 74 text

·ͱΊ • ౰࣌͸Kubernets͕͍͍ײ͡ʹ΍ͬͯ͘ΕΔͩΖ͏ͱґଘ͍͗ͯͨ͢͠ • Kubernetesͷࢥ૝͔Β͋·Γʹ΋ζϨͨ࢖͍ํΛ͢Δͱਏ͘ͳΔ • ӡ༻͕໘౗͔ͩΒͱ͍͏ࢹ఺͚ͩͩͱٯʹਏ͘ͳΔ • ͨͩɺkubernetesΛ̍೥΄Ͳӡ༻ͯ͠Έͯศརͳͱ͜Ζ΋  ৭ʑͱײ͍ͯ͡Δ ༻๏ɾ༰ྔΛकͬͯద੾ʹ !74

Slide 75

Slide 75 text

͋Γ͕ͱ͏͍͟͝·ͨ͠ !75