Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
Search
mekka
May 27, 2023
Technology
1
340
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
イベント駆動システムを構築するツールとしてKEDAを調査したものです
mekka
May 27, 2023
Tweet
Share
More Decks by mekka
See All by mekka
ArgoCDによるGitOps導入 / ArgoCD GitOps
chmikata
0
100
新サービス立ち上げに向けたCI/CD環境の構築
chmikata
0
2.5k
rakusmeetup-number-4-operation
chmikata
1
640
rakusmeetup-number-4-infrastructure
chmikata
0
570
Other Decks in Technology
See All in Technology
データエンジニアリング領域におけるDuckDBのユースケース
chanyou0311
9
2.2k
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
240
EDRの検知の仕組みと検知回避について
chayakonanaika
12
4.9k
LINEギフトにおけるバックエンド開発
lycorptech_jp
PRO
0
280
Change Managerを活用して本番環境へのセキュアなGUIアクセスを統制する / Control Secure GUI Access to the Production Environment with Change Manager
yuj1osm
0
100
AIエージェント入門
minorun365
PRO
31
18k
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
2
660
IAMポリシーのAllow/Denyについて、改めて理解する
smt7174
2
210
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
150
Snowflake ML モデルを dbt データパイプラインに組み込む
estie
0
100
内製化を加速させるlaC活用術
nrinetcom
PRO
2
140
短縮URLをお手軽に導入しよう
nakasho
0
150
Featured
See All Featured
A Philosophy of Restraint
colly
203
16k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.2k
The Cost Of JavaScript in 2023
addyosmani
47
7.4k
Become a Pro
speakerdeck
PRO
26
5.2k
Writing Fast Ruby
sferik
628
61k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
12
990
We Have a Design System, Now What?
morganepeng
51
7.4k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Visualization
eitanlees
146
15k
Practical Orchestrator
shlominoach
186
10k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
10
510
Transcript
,&%"Ͱ࢝ΊΔΠϕϯτۦಈγεςϜ ,VCFSOFUFT/PWJDF5PLZP
ͲͳͨͰ͔͢ʁ w $IJLBIJTB.JLBUBʢ5XJUUFS!NFMQP@NFMʣ w #UP#ͷ4BB4ձࣾʹॴଐ ͳʹͯ͠ΔਓͰ͔͢ʁ w όοΫΤϯυ։ൃ͕ϝΠϯɺ43&ͪΐΖͬͱ w ,VCFSOFUFTֶशதͷʔʔͰ͢
ࣗݾհ
w ,&%"ͱ w ,&%"ͷػೳͱಛ w ,&%"ͷ͍ํ w ·ͱΊ ຊͷൃද༰
,&%"ͱ
,&%"ͱ ,VCFSOFUFT&WFOUESJWF"VUPTDBMJOHʢ,&%"ʣΠϕϯτۦಈͷ ΦʔτεέʔϧΛ࣮ݱ͢ΔίϯϙʔωϯτͰ͢ɻ ʹ$/$'ͷ4BOECPYϓϩδΣΫτͱͳ͓ͬͯΓɺݱࡏ *ODVCBUJOHϓϩδΣΫτͱͳ͍ͬͯ·͢ɻ
,&%"ͱ ҙͷ,VCFSOFUFTΫϥελʹՃͰ͖ΔܰྔίϯϙʔωϯτͰɺ )1"ͳͲඪ४ͷ,VCFSOFUFTίϯϙʔωϯτͱ࿈ܞͯ͠ಈ࡞͠·͢ɻ ΦʔτεέʔϧʹػೳΛߜ͍ͬͯΔͨΊγϯϓϧͳߏͰɺগͳ͍Ϧ ιʔεͰಈ࡞ͤ͞Δ͜ͱ͕Ͱ͖·͢ɻ
,&%"ͷػೳͱಛ
,&%"ͷػೳͱಛ w Πϕϯτͷঢ়ଶʹ߹Θͤͯ1PEΛεέʔϧͰ͖·͢ɻΠϕϯτ͕ൃੜ͍ͯ͠ͳ ͍߹1PEΛʹͰ͖ΔͷϦιʔεΛ༗ޮʹ׆༻Ͱ͖·͢ɻ w Ωϡʔ%#ͳͲ༷ʑͳΠϕϯτϦιʔεΛτϦΨʔʹεέʔϧ͕ՄೳͰ͢ɻ ରԠ͍ͯ͠ΔΠϕϯτϦιʔεLBGLB 1VMTBS .Z42- 1PTUHSF42-ͳͲ
༷ʑͰ͢ɻৄࡉެࣜΛ֬ೝͯ͠Έ͍ͯͩ͘͞ɻ w ΈࠐΈͷεέʔϥʔʢΠϕϯτϦιʔεΛࢹ͢ΔΈʣͷΘΓʹಠࣗ ͷεέʔϥʔΛఆٛ͢Δ͜ͱ͕ՄೳͰ͢ɻ
w LFEBPQFSBUPS 1PEΛ͔Βεέʔϧ͠·͢ɻ w LFEBPQFSBUPSNFUSJDTBQJTFSWFS ΠϕϯτσʔλΛ)1"ʹެ։͢Δ͜ͱ Ͱ1PEΛ͔ΒOεέʔϧ͠·͢ɻ w
LFEBBENJTTJPOXFCIPPLT ϦιʔεͷมߋΛݕূ͠ߏϛεΛ ͗·͢ɻ ΞʔΩςΫνϟ
ΧελϜϦιʔε w 4DBMFE0CKFDU %FQMPZNFOUͱ4UBUFGVM4FUΛεέʔϦϯά͠·͢ɻ w 4DBMFE+PC +PCΛεέʔϦϯά͠·͢ɻ w
5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ ೝূใΛఆٛɺ4DMBFE0CKFDU͔ΒࢀরՄೳʹ͠·͢ɻ w $MVTUFS5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ /NBFTQBDFΛލ͍Ͱར༻ՄೳͳೝূใΛఆٛ͠·͢ɻ
Ϣʔεέʔε w 4DBMFE0CKFDU Ωϡʔͷ͞ʹ߹Θͤͯ1PEΛεέʔϧ͠·͢ɻΩϡʔ͕ۭʹͳΔ·Ͱ1PE ॲཧΛଓߦ͠·͢ɻ ࣌ؒʹେྔͷϦΫΤετΛॲཧ͢ΔέʔεͰར༻͠·͢ɻ w 4DBMFE+PC
ΩϡʔͷΠϕϯτʹରͯͭ͠ͷ1PEΛىಈ͠·͢ɻॲཧ͕ΓΔͱ 1PEऴྃ͠·͢ɻ ࣌ؒͷॲཧΛ࣮ߦ͢ΔΑ͏ͳέʔεͰར༻͠·͢ɻ
,&%"ͷ͍ํ
,&%" ,BGLB IUUQTHJUIVCDPN $POUBJOFS4PMVUJPOTLFEBLBGLB ͪ͜ΒͰެ։͞Ε͍ͯΔαϯϓϧΛར༻͠ ͯڥΛߏங͠·͢ɻ w 1SPEVDFSͷ1PE,BGLBͷΩϡʔʹϝ οηʔδΛొ͠·͢ɻ w
,BGLBͷΩϡʔʹ߹Θͤͯ$POTVNFSͷ 1PEΛͭ·Ͱεέʔϧ͠·͢ɻ w ,BGLBͷΩϡʔ͕ۭʹͳΔͱ$POTVNFS ͷ1PEʹεέʔϧ͠·͢ɻ ΰʔϧΠϝʔδ %FQMPZNFOU 1SPEVDFS $POTVNFS
ࣄલ४උ 4USJN[JΛར༻ͯ͠,BGLBΫϥελΛߏங͠·͢ɻ mikata: ~/ % kubectl get pods -n
kafka NAME READY STATUS RESTARTS AGE kafdrop-5549954bdb-z24kv 1/1 Running 2 (24h ago) 7d18h my-cluster-entity-operator-5d8bccc5d4-8szq4 3/3 Running 6 (24h ago) 7d19h my-cluster-kafka-0 1/1 Running 2 (24h ago) 7d19h my-cluster-zookeeper-0 1/1 Running 2 (24h ago) 7d19h strimzi-cluster-operator-687b8c77db-8qx7w 1/1 Running 192 (13m ago) 8d mikata: ~/ % kubectl get svc -n kafka NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafdrop NodePort 10.96.39.41 <none> 9000:30080/TCP 7d18h my-cluster-kafka-bootstrap ClusterIP 10.96.62.244 <none> 9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-kafka-brokers ClusterIP None <none> 9090/TCP,9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-zookeeper-client ClusterIP 10.96.141.159 <none> 2181/TCP 7d19h my-cluster-zookeeper-nodes ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 7d19h
ࣄલ४උ ΧελϜϦιʔεͰ,BGLBͷ5PQJDΛ࡞͠·͢ɻ apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: labels: strimzi.io/cluster:
my-cluster name: my-topic namespace: kafka spec: config: {} partitions: 5 replicas: 1 topicName: my-topic mikata: ~/ % kubectl get kafkatopic my-topic -n kafka NAME CLUSTER PARTITIONS REPLICATION FACTOR READY my-topic my-cluster 5 1 True
ࣄલ४උ 1SPEVDFSͱ$POTVNFSͷΞϓϦΛσϓϩΠ͠·͢ɻ mikata: ~/ % kubectl get po -n
demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 6m27s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 6m42s mikata: ~/ % kubectl get svc -n demo NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafka-producer-service ClusterIP 10.96.193.144 <none> 80/TCP 7m34s ,BGLB 1SPEVDFS $POTVNFS
ࣄલ४උ 1SPEVDFS͕ϝοηʔδΛొɺ$POTVNFSϝοηʔδΛಡΈग़͠·͢ɻ ɾɾɾ for i := 0; i <
messageCount; i++ { wg.Add(1) i := i go func() { defer wg.Done() writeMessage(w, i) }() } wg.Wait() ɾɾɾ } func writeMessage(w *kafka.Writer, i int) { message := kafka.Message{Key: []byte(fmt.Sprintf("Key- %d", i)), Value: []byte(fmt.Sprintf("Message No: %d, Time: %s", i, time.Now()))} err := w.WriteMessages(context.Background(), message) ɾɾɾ } ɾɾɾ r := kafka.NewReader(kafka.ReaderConfig{ Brokers: []string{kafkaBroker}, GroupID: consumerGroup, Topic: kafkaTopic, MinBytes: 10e1, // 1KB MaxBytes: 10e6, // 10MB }) defer r.Close() go func() { for { m, err := r.ReadMessage(context.Background()) if err != nil { break } time.Sleep(time.Second * time.Duration(sleepSeconds)) } }() }
4DBMFE0CKFDUͷ࡞ )FMNΛར༻ͯ͠,&%"ΛΠϯετʔϧ͠·͢ɻ mikata: ~/ % helm repo add kedacore
https://kedacore.github.io/charts mikata: ~/ % helm repo update mikata: ~/ % kubectl create ns keda mikata: ~/ % helm install keda kedacore -ns keda mikata: ~/ % kubectl get po -n keda NAME READY STATUS RESTARTS AGE keda-admission-webhooks-7df9d947c4-jhc4z 1/1 Running 1 (3d1h ago) 10d keda-operator-96579d64c-5tj9q 1/1 Running 3 (3d1h ago) 10d keda-operator-metrics-apiserver-8758c5874-xcmqd 1/1 Running 2 (3d1h ago) 10d mikata: ~/ % kubectl get svc -n keda NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE keda-admission-webhooks ClusterIP 10.96.129.95 <none> 443/TCP 10d keda-operator ClusterIP 10.96.152.106 <none> 9666/TCP 10d keda-operator-metrics-apiserver ClusterIP 10.96.98.38 <none> 443/TCP,80/TCP 10d
4DBMFE0CKFDUͷ࡞ ΧελϜϦιʔεͰ4DBMFE0CKFDUΛ࡞͠·͢ɻ apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: kafka-consumer-scaler
namespace: demo spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: kafka-consumer pollingInterval: 10 cooldownPeriod: 30 idleReplicaCount: 0 minReplicaCount: 1 maxReplicaCount: 5 triggers: - type: kafka metadata: bootstrapServers: my-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 consumerGroup: g1 topic: my-topic lagThreshold: "10" offsetResetPolicy: latest allowIdleConsumers: "false" version: 1.0.0 pollingInterval: τϦΨʔͷϙʔϦϯάִؒ cooldownPeriod: εέʔϧόοΫͷػظؒ idleReplicaCount: ػ࣌ͷϨϓϦΧ minReplicaCount: ࠷খϨϓϦΧ maxReplicaCount: ࠷େϨϓϦΧ bootstrapServers: kafkaͷଓઌ consumeGroup: Consumer Group໊ topic: topic໊ lagThreshold εέʔϧ͢Δᮢ
4DBMFE0CKFDUͷ࡞ 4DBMFE0CKFDUΛσϓϩΠ͠·͕͢ɺ1PEʹมԽ͋Γ·ͤΜɻ mikata: ~/ % kubectl apply -f scaled-object.yaml
scaledobject.keda.sh/kafka-consumer-scaler created mikata: ~/ % kubectl get scaledobjects -n demo NAME SCALETARGETKIND SCALETARGETNAME MIN MAX TRIGGERS AUTHENTICATION READY ACTIVE FALLBACK AGE kafka-consumer-scaler apps/v1.Deployment kafka-consumer 1 5 kafka True True False 3m23s mikata: ~/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 2d19h kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: ~/ % kubectl get hpa -n demo NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE keda-hpa-kafka-consumer-scaler Deployment/kafka-consumer 5/10 (avg) 1 5 1 3m55s HPA͕࡞͞Ε·͢
εέʔϧͷ֬ೝ ΞϓϦʹෛՙΛ͔͚εέʔϧΛ֬ೝ͠·͢ɻ ҰͷϦΫΤετͰ1SPEVDFSݸͷϝοηʔδΛLBGLBʹొ͍ͯ͠·͢ɻ mikata: ~/ % kubectl port-forward svc/kafka-producer-service 8080:80
-n demo & [1] 10843 mikata: keda-kafka/ % Forwarding from 127.0.0.1:8080 -> 5550 Forwarding from [::1]:8080 -> 5550 mikata: ~/ % curl 127.0.0.1:8080 Handling connection for 8080 mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-6dvq7 1/1 Running 0 57s kafka-consumer-5468c48766-h4mfv 1/1 Running 0 42s kafka-consumer-5468c48766-h8vwt 1/1 Running 0 42s kafka-consumer-5468c48766-k4js6 1/1 Running 0 57s kafka-consumer-5468c48766-khqs6 1/1 Running 0 74s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h
·ͱΊ
·ͱΊ Πϯετʔϧɺઃఆͷखॱ͕γϯϓϧͰಋೖͷোน͍ͷͰɺखͬऔΓૣ͘Π ϕϯτۦಈͷγεςϜΛߏங͢Δͷʹྑ͍ͱࢥ͍·͢ɻ खܰͳ໘ɺϝοηʔδϒϩʔΧʔͱͷ*'ఏڙ͞Εͳ͍ͷͰɺϒϩʔΧʔͱΞ ϓϦέʔγϣϯ͕ີ݁߹ʹͳΓ͍͢ͱײ͡·ͨ͠ɻ େنͳ৫Ͱӡ༻͍ͯ͘͠߹ɺ͜ͷ͋ͨΓΛߟྀͭͭ͠ɺಋೖΛݕ౼͢ Δͷ͕ྑ͍ͷͰͳ͍͔ͱࢥ͍·͢ɻ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠