HPA with custom metrics

2019/11/27 CloudNative Days Kansai 2019 Meetup
https://cloudnativedays.connpass.com/event/152321/

1. DVTUPN
   NFUSJDT
   ʹΑΔ )1"ʹֶͿ 0QFSBUPSͱ"EBQUFS CloudNative Days Kansai 2019 Meetup 2019/11/27 @loftkun
2. ࣗݾ঺հ !MPGULVO
   ߕ൹ ৽ޛ Ϡϑʔגࣜձࣾ αʔϏεϓϥοτϑΥʔϜຊ෦ ࡢ೔-5͖ͯ͠·ͨ͠ʂ
3. .PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPN
   NFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ
4. .PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPN
   NFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ 
   ௐ΂ͯΈͨ
5. .PUJWBUJPO • )1"ͷυΩϡϝϯτΛݟͯ ɾ$16࢖༻཰ʹΑΔ)1" ɾ ಈ͔ͤͨ ɾDVTUPN
   NFUSJDTʹΑΔ)1" ɾ ۩ମతͳखॱ͕ͳ͔ͬͨ 
   ௐ΂ͯΈͨ • 0QFSBUPS΍"EBQUFSύλʔϯͷ஌ݟΛಘΔ͜ͱ ͕Ͱ͖ͨͷͰڞ༗͠·͢
6. &OWJSPONFOU 
   .JOJLVCF
7. )PSJ[POUBM
   1PE
   "VUPTDBMFS 8BMLUISPVHI https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
8. )PSJ[POUBM
   1PE
   "VUPTDBMFS 8BMLUISPVHI FOBCMF
   NFUSJDT
   TFSWFS
9. )PSJ[POUBM
   1PE
   "VUPTDBMFS 8BMLUISPVHI SVO
   
   FYQPTF
   QIQBQBDIF
   TFSWFS
10. )PSJ[POUBM
    1PE
    "VUPTDBMFS 8BMLUISPVHI DSFBUF
    )1"
11. )PSJ[POUBM
    1PE
    "VUPTDBMFS 8BMLUISPVHI JODSFBTF
    MPBE
12. )PSJ[POUBM
    1PE
    "VUPTDBMFS 8BMLUISPVHI DIFDL
    SFQMJDBT εέʔϧͨ͠Θ
13. )PSJ[POUBM
    1PE
    "VUPTDBMFS 8BMLUISPVHI DVTUPN
    NFUSJDT
    ͸ʁ ۩ମతͳखॱ͸ͳ͍Θ ࣗલͰಠࣗͷϝτϦΫεΛ༻ҙ͠ͳ͍ͱ ͔͋Μ΍Ζ͜Ε ༻ҙͨ͠ϝτϦΫε͸Ͳ͏΍ͬͯ)1"ί ϯτϩʔϥʹࢀরͤ͞ΔΜ΍ʁ
14. MVYBTLVCFBENXPSLTIPQ https://github.com/luxas/kubeadm-workshop
15. MVYBTLVCFBENXPSLTIPQ • k8sクラスタを構築するチュートリアル • DVTUPN
    NFUSJDTʹΑΔ)1"ͷνϡʔτϦΞϧ ͋Γʂ – l%FQMPZJOH
    UIF
    1SPNFUIFVT
    0QFSBUPS
    GPS
    NPOJUPSJOH
    4FSWJDFT
    JO
    UIF
    DMVTUFSz – l%FQMPZJOH
    B
    DVTUPN
    NFUSJDT
    "1*
    4FSWFS
    BOE
    B
    TBNQMF
    BQQz
16. TBNQMFNFUSJDTBQQZBNM ͜ͷϝτϦΫεͰ)1"͕ಈ࡞͢ΔΑ͏ ༻ҙͯ͘͠ΕͯΔΜ΍ͳɺ ͋Γ͕͍ͨΘʙ
17. EFNPTNPOJUPSJOH
18. EFNPTNPOJUPSJOH
19. EFNPTNPOJUPSJOH DVTUPN
    NFUSJDTͰ)1"ಈ͍ͯΔΘ ͑͑΍Μ
20. &01ɾɾɾɾʁ
21. QSPNFUIFVTPQFSBUPSZBNM Operator？
22. TBNQMFNFUSJDTBQQZBNM kind : ServiceMonitor ？
23. TBNQMFQSPNFUIFVTJOTUBODFZBNM kind : Prometheus ？
24. DVTUPNNFUSJDTZBNM Adapter？
25. ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server HFU
    NFUSJDT DPMMFDU
    NFUSJDT TDBMF
    SFQMJDBT
26. ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? HFU
    NFUSJDT TDBMF
    SFQMJDBT DPMMFDU
    NFUSJDT
27. ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? How to ? HFU
    NFUSJDT TDBMF
    SFQMJDBT DPMMFDU
    NFUSJDT
28. ੔ཧͯ͠Έͨ ϚχϑΣετZBNM ˣ &YDFMͷςΩετϘοΫε ؔ܎ੑͷ੔ཧ͸ࠓͷͱ͜Ζਓྗ͕ඞཁ ͳΜ΍ ࣗಈԽ͍ͨ͠
29. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ s e r v i c e
30. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator s e r v i c e
31. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e
32. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Service Monitor s e r v i c e monitoring kind: Prometheus
33. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e check
34. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e create config
35. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e
36. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e get metrics
37. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e JNBHF
    
    
    DirectXMan12/k8s-prometheus-adapter
38. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e s e r v i c e Kind: APIService
39. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server aggregation layer s e r v i c e Kind: APIService
40. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server aggregation layer s e r v i c e Kind: APIService
41. )1"
    XJUI
    DVTUPN
    NFUSJDT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server aggregation layer s e r v i c e Kind: APIService HPA contorller
42. ཧղ͍ͨ͠ͱ͜Ζ HPA contorller Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server How to ? How to ? HFU
    NFUSJDT TDBMF
    SFQMJDBT DPMMFDU
    NFUSJDT
43. )1"
    XJUI
    DVTUPN
    NFUSJDT TDBMF
    SFQMJDBT Deployment Pod 1 Pod 2 Pod N ɾɾɾ Custom metrics Server Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e s e r v i c e kube-api server s e r v i c e Kind: APIService HPA contorller DPMMFDU
    NFUSJDT HFU
    NFUSJDT aggregation layer
44. 0QFSBUPSʁ 5XJUUFS࣮گ͕ࢭ·Δ΍ͭ
45. 0QFSBUPSΛσϓϩΠ͢Δͱ $3%Λ࡞ͬͯ͘ΕΔ
46. զʑ͸$3%ΦϒδΣΫτΛ DSFBUFͰ͖ΔΑ͏ʹͳΔ
47. $3%ΦϒδΣΫτΛ ӡ༻ͯ͘͠ΕΔͷ͕0QFSBUPS Prometheus Operator kind: Prometheus kind: Service Monitor s e r v i c e Deployment 1SPNFUIFVT 0QFSBUPSͷ৔߹ 4FSWJDF.POJUPSʹ ର৅TFSWJDFͱFOEQPJOUΛઃఆ͓ͯ͘͠ͱ 0QFSBUPS͕ QSPNFUIFVTZBNMΛॻ͍ͯ͘ΕΔʂ ӡ༻͕ίʔυͰ࣮૷͞Ε͍ͯΔΜͩʂ
48. "EBQUFSʁ
49. ޓ׵ੑΛఏڙ͢Δͷ͕"EBQUFS ޓ׵ੑ͕ͳ͍ Prometheus kube-api server aggregation layer
50. ޓ׵ੑΛఏڙ͢Δͷ͕"EBQUFS 1SPNFUIFVT
    "EBQUFS Custom metrics Server Prometheus kube-api server aggregation layer  Adapterパターンだ！
51. ·ͱΊ • 0QFSBUPSͷྫͱͯ͠ɺ1SPNFUIFVT
    0QFSBUPSʹΑΔϝτϦΫεͷऩूํ๏͕෼ ͔ͬͨΜ΍ • "EBQUFSύλʔϯͷ۩ମతͳࣄྫΛ஌ͬͨΜ΍ • DVTUPN
    NFUSJDTʹΑΔ)1"Λಈ͔͢ํ๏͕෼ ͔ͬͨΜ΍
52. 0QFSBUPSʁ ࣮گ͕੝Γ্͕Ε͹޾͍Ͱ͢ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ $/%,ͷ-5Ͱݟͨ΍ͭ΍ʂ $/%,Ͱ׬શʹཧղͨ͠Θ
53. "QQFOEJY
54. 1PE
    "VUPTDBMFS
55. 1PE
    "VUPTDBMFS SFTPVSDF Pod 1 DQV
    NFNPSZ
56. 1PE
    "VUPTDBMFS Pod 1 SFTPVSDF 7FSUJDBM DQV
    NFNPSZ
57. 1PE
    "VUPTDBMFS SFQMJDBT Pod 1
58. 1PE
    "VUPTDBMFS Horizontal SFQMJDBT Pod 1 Pod 2 Pod N ɾɾɾ