PrometheusでKubernetesの高速オートスケールを実現した話
by
Atsushi Tanaka
×
Copy
Open
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Slide 1
Slide 1 text
©2018 Wantedly, Inc. PrometheusͰKubernetesͷߴΦʔτεέʔϧΛ࣮ݱͨ͠ Cloud Native Meetup Tokyo #2 LT 1.Jun.2018 - Atsushi Tanaka @bgpat
Slide 2
Slide 2 text
©2018 Wantedly, Inc. Atsushi Tanaka • @bgpat • Infrastructure Engineer at Wantedly, Inc. • Favorites: Kubernetes, Docker, Golang JavaScript, Vue.js
Slide 3
Slide 3 text
©2018 Wantedly, Inc. w Φʔτεέʔϧ͕͍ w ͘εέʔϧ͢Δํ๏ w ϓϩμΫγϣϯͰͬͯΈͨ ͢͜ͱ
Slide 4
Slide 4 text
©2018 Wantedly, Inc. ࠓ·Ͱ͍ͬͯͨΦʔτεέʔϧ Horizontal Pod Autoscaler (HPA) + CPU $16ෛՙ͕ߴ͘ͳΔͱεέʔϧ
Slide 5
Slide 5 text
©2018 Wantedly, Inc. ࠓ·Ͱ͍ͬͯͨΦʔτεέʔϧ $16ෛՙ͕ߴ͘ͳΔͱεέʔϧ ϝϦοτ w ظతͳΞΫηεͷ૿Ճʹڧ͍ w ݹ͍όʔδϣϯͰ͑Δ Horizontal Pod Autoscaler (HPA) + CPU
Slide 6
Slide 6 text
©2018 Wantedly, Inc. ࠓ·Ͱ͍ͬͯͨΦʔτεέʔϧ $16ෛՙ͕ߴ͘ͳΔͱεέʔϧ ϝϦοτ w ظతͳΞΫηεͷ૿Ճʹڧ͍ w ݹ͍όʔδϣϯͰ͑Δ σϝϦοτ w ॠؒతͳεύΠΫʹऑ͍ Horizontal Pod Autoscaler (HPA) + CPU
Slide 7
Slide 7 text
©2018 Wantedly, Inc.
Slide 8
Slide 8 text
©2018 Wantedly, Inc.
Slide 9
Slide 9 text
©2018 Wantedly, Inc.
Slide 10
Slide 10 text
©2018 Wantedly, Inc. શવؒʹ߹͍ͬͯͳ͍
Slide 11
Slide 11 text
©2018 Wantedly, Inc. ͬͱૣ͘εέʔϧͯ͠΄͍͠
Slide 12
Slide 12 text
©2018 Wantedly, Inc. εύΠΫʹڧ͍Φʔτεέʔϧ HPA + Custom Metrics ΞϓϦέʔγϣϯ͕ܾΊͨج४Ͱεέʔϧ w W͔Β͑Δ w ྫ͑ʜϦΫΤετ ίωΫγϣϯ w Ξμϓλ͕ඞཁ
Slide 13
Slide 13 text
©2018 Wantedly, Inc. εύΠΫʹڧ͍Φʔτεέʔϧ HPA + Custom Metrics ΞϓϦέʔγϣϯ͕ܾΊͨج४Ͱεέʔϧ w W͔Β͑Δ w ྫ͑ʜϦΫΤετ ίωΫγϣϯ w Ξμϓλ͕ඞཁˠ,VCFSOFUFTʹϝτϦΫεΛૹΔͨΊͷαʔόʔ
Slide 14
Slide 14 text
©2018 Wantedly, Inc. HPA + Custom Metrics k8s-prometheus-adapter 1SPNFUIFVT͕ऩूͨ͠ϝτϦΫεΛ,VCFSOFUFTʹૹͬͯ͘ΕΔαʔόʔ
Slide 15
Slide 15 text
©2018 Wantedly, Inc. ͋ͱج४ʹ͢ΔΛܾΊͯ Prometheus͕ಡΊΔϑΥʔϚοτͰग़ྗ͢Δ͚ͩ
Slide 16
Slide 16 text
©2018 Wantedly, Inc. ͋ͱج४ʹ͢ΔΛܾΊͯ Prometheus͕ಡΊΔϑΥʔϚοτͰग़ྗ͢Δ͚ͩ Ͱ͖ΕΠϯϑϥϨΠϠʔͰղܾ͍ͨ͠
Slide 17
Slide 17 text
©2018 Wantedly, Inc. εέʔϧͷఆج४ /HJOYͷTUVC@TUBUVT location /basic_status { stub_status; } Active connections: 291 server accepts handled requests 16630948 16630948 31070465 Reading: 6 Writing: 179 Waiting: 106
Slide 18
Slide 18 text
©2018 Wantedly, Inc. εέʔϧͷఆج४ /HJOYͷTUVC@TUBUVT location /basic_status { stub_status; } Active connections: 291 server accepts handled requests 16630948 16630948 31070465 Reading: 6 Writing: 179 Waiting: 106 ͪίωΫγϣϯΛ͏
Slide 19
Slide 19 text
©2018 Wantedly, Inc. εέʔϧͷఆج४ TUVC@TUBUVTΛFYQPSUFSͷܗࣜʹ͍ͨ͠ # HELP nginx_connections_current Number of connections currently processed by nginx # TYPE nginx_connections_current gauge nginx_connections_current{state="active"} 2 nginx_connections_current{state="reading"} 0 nginx_connections_current{state="waiting"} 1 nginx_connections_current{state="writing"} 1
Slide 20
Slide 20 text
©2018 Wantedly, Inc. ,VCFSOFUFST "1*4FSWFS LTQSPNFUIFVTBEBQUFS 1SPNFUIFVT )PSJ[POUBM 1PE "VUPTDBMFS %FQMPZNFOU 3FQMJDB4FU 1PE 1PE 1PE OHJOY@FYQPSUFS /HJOY TUVC@TUBUVT
Slide 21
Slide 21 text
©2018 Wantedly, Inc. ݁Ռ w සൟʹεέʔϧ͢ΔΑ͏ʹͳͬͨ w $16ෛՙ͕҆ఆͨ͠ w Ξϥʔτ͕໐Βͳ͘ͳͬͨ
Slide 22
Slide 22 text
©2018 Wantedly, Inc. ߴΦʔτεέʔϧ͕࣮ݱͰ͖ͨ
Slide 23
Slide 23 text
©2018 Wantedly, Inc. https://github.com/bgpat/k8s-hpa-example