HPA with custom metrics

by loftkun

Slide 1

Slide 1 text

DVTUPNNFUSJDTʹΑΔ )1"ʹֶͿ 0QFSBUPSͱ"EBQUFS CloudNative Days Kansai 2019 Meetup 2019/11/27 @loftkun

Slide 2

Slide 2 text

ࣗݾ঺հ !MPGULVO ߕ൹ ৽ޛ Ϡϑʔגࣜձࣾ αʔϏεϓϥοτϑΥʔϜຊ෦ ࡢ೔-5͖ͯ͠·ͨ͠ʂ

Slide 3

Slide 3 text

.PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPNNFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ

Slide 4

Slide 4 text

.PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPNNFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ ௐ΂ͯΈͨ

Slide 5

Slide 5 text

.PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPNNFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ ௐ΂ͯΈͨ • 0QFSBUPS΍"EBQUFSύλʔϯͷ஌ݟΛಘΔ͜ͱ ͕Ͱ͖ͨͷͰڞ༗͠·͢

Slide 6

Slide 6 text

&OWJSPONFOU .JOJLVCF

Slide 7

Slide 7 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/

Slide 8

Slide 8 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI FOBCMFNFUSJDTTFSWFS

Slide 9

Slide 9 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI SVOFYQPTFQIQBQBDIFTFSWFS

Slide 10

Slide 10 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI DSFBUF)1"

Slide 11

Slide 11 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI JODSFBTFMPBE

Slide 12

Slide 12 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI DIFDLSFQMJDBT εέʔϧͨ͠Θ

Slide 13

Slide 13 text

)PSJ[POUBM1PE"VUPTDBMFS 8BMLUISPVHI DVTUPNNFUSJDT͸ʁ ۩ମతͳखॱ͸ͳ͍Θ ࣗલͰಠࣗͷϝτϦΫεΛ༻ҙ͠ͳ͍ͱ ͔͋Μ΍Ζ͜Ε ༻ҙͨ͠ϝτϦΫε͸Ͳ͏΍ͬͯ)1"ί ϯτϩʔϥʹࢀরͤ͞ΔΜ΍ʁ

Slide 14

Slide 14 text

MVYBTLVCFBENXPSLTIPQ https://github.com/luxas/kubeadm-workshop

Slide 15

Slide 15 text

MVYBTLVCFBENXPSLTIPQ • k8sクラスタを構築するチュートリアル • DVTUPNNFUSJDTʹΑΔ)1"ͷνϡʔτϦΞϧ ͋Γʂ – l%FQMPZJOHUIF1SPNFUIFVT0QFSBUPSGPSNPOJUPSJOH4FSWJDFTJOUIF DMVTUFSz – l%FQMPZJOHBDVTUPNNFUSJDT"1*4FSWFSBOEBTBNQMFBQQz

Slide 16

Slide 16 text

TBNQMFNFUSJDTBQQZBNM ͜ͷϝτϦΫεͰ)1"͕ಈ࡞͢ΔΑ͏ ༻ҙͯ͘͠ΕͯΔΜ΍ͳɺ ͋Γ͕͍ͨΘʙ

Slide 17

Slide 17 text

EFNPTNPOJUPSJOH

Slide 18

Slide 18 text

EFNPTNPOJUPSJOH

Slide 19

Slide 19 text

EFNPTNPOJUPSJOH DVTUPNNFUSJDTͰ)1"ಈ͍ͯΔΘ ͑͑΍Μ

Slide 20

Slide 20 text

&01ɾɾɾɾʁ

Slide 21

Slide 21 text

QSPNFUIFVTPQFSBUPSZBNM Operator？

Slide 22

Slide 22 text

TBNQMFNFUSJDTBQQZBNM kind : ServiceMonitor ？

Slide 23

Slide 23 text

TBNQMFQSPNFUIFVTJOTUBODFZBNM kind : Prometheus ？

Slide 24

Slide 24 text

DVTUPNNFUSJDTZBNM Adapter？

Slide 25

Slide 25 text

ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server HFUNFUSJDT DPMMFDUNFUSJDT TDBMFSFQMJDBT

Slide 26

Slide 26 text

ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? HFUNFUSJDT TDBMFSFQMJDBT DPMMFDUNFUSJDT

Slide 27

Slide 27 text

ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? How to ? HFUNFUSJDT TDBMFSFQMJDBT DPMMFDUNFUSJDT

Slide 28

Slide 28 text

੔ཧͯ͠Έͨ ϚχϑΣετZBNM ˣ &YDFMͷςΩετϘοΫε ؔ܎ੑͷ੔ཧ͸ࠓͷͱ͜Ζਓྗ͕ඞཁ ͳΜ΍ ࣗಈԽ͍ͨ͠

Slide 29

Slide 29 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ s e r v i c e

Slide 30

Slide 30 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator s e r v i c e

Slide 31

Slide 31 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e

Slide 32

Slide 32 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Service Monitor s e r v i c e monitoring kind: Prometheus

Slide 33

Slide 33 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e check

Slide 34

Slide 34 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e create config

Slide 35

Slide 35 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e

Slide 36

Slide 36 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e get metrics

Slide 37

Slide 37 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e JNBHFDirectXMan12/k8s-prometheus-adapter

Slide 38

Slide 38 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e Kind: APIService

Slide 39

Slide 39 text

)1"XJUIDVTUPNNFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server aggregation layer s e r v i c e Kind: APIService

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? How to ? HFUNFUSJDT TDBMFSFQMJDBT DPMMFDUNFUSJDT

Slide 43

Slide 43 text

)1"XJUIDVTUPNNFUSJDT TDBMFSFQMJDBT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server s e r v i c e Kind: APIService HPA contorller DPMMFDUNFUSJDT HFUNFUSJDT aggregation layer

Slide 44

Slide 44 text

0QFSBUPSʁ 5XJUUFS࣮گ͕ࢭ·Δ΍ͭ

Slide 45

Slide 45 text

0QFSBUPSΛσϓϩΠ͢Δͱ $3%Λ࡞ͬͯ͘ΕΔ

Slide 46

Slide 46 text

զʑ͸$3%ΦϒδΣΫτΛ DSFBUFͰ͖ΔΑ͏ʹͳΔ

Slide 47

Slide 47 text

$3%ΦϒδΣΫτΛ ӡ༻ͯ͘͠ΕΔͷ͕0QFSBUPS Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e Deployment 1SPNFUIFVT 0QFSBUPSͷ৔߹ 4FSWJDF.POJUPSʹ ର৅TFSWJDFͱFOEQPJOUΛઃఆ͓ͯ͘͠ͱ 0QFSBUPS͕ QSPNFUIFVTZBNMΛॻ͍ͯ͘ΕΔʂ ӡ༻͕ίʔυͰ࣮૷͞Ε͍ͯΔΜͩʂ

Slide 48

Slide 48 text

"EBQUFSʁ

Slide 49

Slide 49 text

ޓ׵ੑΛఏڙ͢Δͷ͕"EBQUFS ޓ׵ੑ͕ͳ͍ Prometheus kube-api server aggregation layer

Slide 50

Slide 50 text

ޓ׵ੑΛఏڙ͢Δͷ͕"EBQUFS 1SPNFUIFVT"EBQUFS Custom metrics Server Prometheus kube-api server aggregation layer Adapterパターンだ！

Slide 51

Slide 51 text

·ͱΊ • 0QFSBUPSͷྫͱͯ͠ɺ1SPNFUIFVT 0QFSBUPSʹΑΔϝτϦΫεͷऩूํ๏͕෼ ͔ͬͨΜ΍ • "EBQUFSύλʔϯͷ۩ମతͳࣄྫΛ஌ͬͨΜ΍ • DVTUPNNFUSJDTʹΑΔ)1"Λಈ͔͢ํ๏͕෼ ͔ͬͨΜ΍

Slide 52

Slide 52 text

0QFSBUPSʁ ࣮گ͕੝Γ্͕Ε͹޾͍Ͱ͢ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ $/%,ͷ-5Ͱݟͨ΍ͭ΍ʂ $/%,Ͱ׬શʹཧղͨ͠Θ