Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
Search
mekka
May 27, 2023
Technology
1
340
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
イベント駆動システムを構築するツールとしてKEDAを調査したものです
mekka
May 27, 2023
Tweet
Share
More Decks by mekka
See All by mekka
ArgoCDによるGitOps導入 / ArgoCD GitOps
chmikata
0
98
新サービス立ち上げに向けたCI/CD環境の構築
chmikata
0
2.5k
rakusmeetup-number-4-operation
chmikata
1
640
rakusmeetup-number-4-infrastructure
chmikata
0
570
Other Decks in Technology
See All in Technology
Developers Summit 2025 浅野卓也(13-B-7 LegalOn Technologies)
legalontechnologies
PRO
0
720
OpenID Connect for Identity Assurance の概要と翻訳版のご紹介 / 20250219-BizDay17-OIDC4IDA-Intro
oidfj
0
270
バックエンドエンジニアのためのフロントエンド入門 #devsumiC
panda_program
18
7.5k
Nekko Cloud、 これまでとこれから ~学生サークルが作る、 小さなクラウド
logica0419
2
970
RECRUIT TECH CONFERENCE 2025 プレイベント【高橋】
recruitengineers
PRO
0
160
Developer Summit 2025 [14-D-1] Yuki Hattori
yuhattor
19
6.2k
2/18/25: Java meets AI: Build LLM-Powered Apps with LangChain4j
edeandrea
PRO
0
120
自動テストの世界に、この5年間で起きたこと
autifyhq
10
8.5k
Culture Deck
optfit
0
420
レビューを増やしつつ 高評価維持するテクニック
tsuzuki817
1
710
技術的負債解消の取り組みと専門チームのお話 #技術的負債_Findy
bengo4com
1
1.3k
Larkご案内資料
customercloud
PRO
0
650
Featured
See All Featured
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
114
50k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
12
960
Visualization
eitanlees
146
15k
Building Your Own Lightsaber
phodgson
104
6.2k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
193
16k
Why You Should Never Use an ORM
jnunemaker
PRO
55
9.2k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.2k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Transcript
,&%"Ͱ࢝ΊΔΠϕϯτۦಈγεςϜ ,VCFSOFUFT/PWJDF5PLZP
ͲͳͨͰ͔͢ʁ w $IJLBIJTB.JLBUBʢ5XJUUFS!NFMQP@NFMʣ w #UP#ͷ4BB4ձࣾʹॴଐ ͳʹͯ͠ΔਓͰ͔͢ʁ w όοΫΤϯυ։ൃ͕ϝΠϯɺ43&ͪΐΖͬͱ w ,VCFSOFUFTֶशதͷʔʔͰ͢
ࣗݾհ
w ,&%"ͱ w ,&%"ͷػೳͱಛ w ,&%"ͷ͍ํ w ·ͱΊ ຊͷൃද༰
,&%"ͱ
,&%"ͱ ,VCFSOFUFT&WFOUESJWF"VUPTDBMJOHʢ,&%"ʣΠϕϯτۦಈͷ ΦʔτεέʔϧΛ࣮ݱ͢ΔίϯϙʔωϯτͰ͢ɻ ʹ$/$'ͷ4BOECPYϓϩδΣΫτͱͳ͓ͬͯΓɺݱࡏ *ODVCBUJOHϓϩδΣΫτͱͳ͍ͬͯ·͢ɻ
,&%"ͱ ҙͷ,VCFSOFUFTΫϥελʹՃͰ͖ΔܰྔίϯϙʔωϯτͰɺ )1"ͳͲඪ४ͷ,VCFSOFUFTίϯϙʔωϯτͱ࿈ܞͯ͠ಈ࡞͠·͢ɻ ΦʔτεέʔϧʹػೳΛߜ͍ͬͯΔͨΊγϯϓϧͳߏͰɺগͳ͍Ϧ ιʔεͰಈ࡞ͤ͞Δ͜ͱ͕Ͱ͖·͢ɻ
,&%"ͷػೳͱಛ
,&%"ͷػೳͱಛ w Πϕϯτͷঢ়ଶʹ߹Θͤͯ1PEΛεέʔϧͰ͖·͢ɻΠϕϯτ͕ൃੜ͍ͯ͠ͳ ͍߹1PEΛʹͰ͖ΔͷϦιʔεΛ༗ޮʹ׆༻Ͱ͖·͢ɻ w Ωϡʔ%#ͳͲ༷ʑͳΠϕϯτϦιʔεΛτϦΨʔʹεέʔϧ͕ՄೳͰ͢ɻ ରԠ͍ͯ͠ΔΠϕϯτϦιʔεLBGLB 1VMTBS .Z42- 1PTUHSF42-ͳͲ
༷ʑͰ͢ɻৄࡉެࣜΛ֬ೝͯ͠Έ͍ͯͩ͘͞ɻ w ΈࠐΈͷεέʔϥʔʢΠϕϯτϦιʔεΛࢹ͢ΔΈʣͷΘΓʹಠࣗ ͷεέʔϥʔΛఆٛ͢Δ͜ͱ͕ՄೳͰ͢ɻ
w LFEBPQFSBUPS 1PEΛ͔Βεέʔϧ͠·͢ɻ w LFEBPQFSBUPSNFUSJDTBQJTFSWFS ΠϕϯτσʔλΛ)1"ʹެ։͢Δ͜ͱ Ͱ1PEΛ͔ΒOεέʔϧ͠·͢ɻ w
LFEBBENJTTJPOXFCIPPLT ϦιʔεͷมߋΛݕূ͠ߏϛεΛ ͗·͢ɻ ΞʔΩςΫνϟ
ΧελϜϦιʔε w 4DBMFE0CKFDU %FQMPZNFOUͱ4UBUFGVM4FUΛεέʔϦϯά͠·͢ɻ w 4DBMFE+PC +PCΛεέʔϦϯά͠·͢ɻ w
5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ ೝূใΛఆٛɺ4DMBFE0CKFDU͔ΒࢀরՄೳʹ͠·͢ɻ w $MVTUFS5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ /NBFTQBDFΛލ͍Ͱར༻ՄೳͳೝূใΛఆٛ͠·͢ɻ
Ϣʔεέʔε w 4DBMFE0CKFDU Ωϡʔͷ͞ʹ߹Θͤͯ1PEΛεέʔϧ͠·͢ɻΩϡʔ͕ۭʹͳΔ·Ͱ1PE ॲཧΛଓߦ͠·͢ɻ ࣌ؒʹେྔͷϦΫΤετΛॲཧ͢ΔέʔεͰར༻͠·͢ɻ w 4DBMFE+PC
ΩϡʔͷΠϕϯτʹରͯͭ͠ͷ1PEΛىಈ͠·͢ɻॲཧ͕ΓΔͱ 1PEऴྃ͠·͢ɻ ࣌ؒͷॲཧΛ࣮ߦ͢ΔΑ͏ͳέʔεͰར༻͠·͢ɻ
,&%"ͷ͍ํ
,&%" ,BGLB IUUQTHJUIVCDPN $POUBJOFS4PMVUJPOTLFEBLBGLB ͪ͜ΒͰެ։͞Ε͍ͯΔαϯϓϧΛར༻͠ ͯڥΛߏங͠·͢ɻ w 1SPEVDFSͷ1PE,BGLBͷΩϡʔʹϝ οηʔδΛొ͠·͢ɻ w
,BGLBͷΩϡʔʹ߹Θͤͯ$POTVNFSͷ 1PEΛͭ·Ͱεέʔϧ͠·͢ɻ w ,BGLBͷΩϡʔ͕ۭʹͳΔͱ$POTVNFS ͷ1PEʹεέʔϧ͠·͢ɻ ΰʔϧΠϝʔδ %FQMPZNFOU 1SPEVDFS $POTVNFS
ࣄલ४උ 4USJN[JΛར༻ͯ͠,BGLBΫϥελΛߏங͠·͢ɻ mikata: ~/ % kubectl get pods -n
kafka NAME READY STATUS RESTARTS AGE kafdrop-5549954bdb-z24kv 1/1 Running 2 (24h ago) 7d18h my-cluster-entity-operator-5d8bccc5d4-8szq4 3/3 Running 6 (24h ago) 7d19h my-cluster-kafka-0 1/1 Running 2 (24h ago) 7d19h my-cluster-zookeeper-0 1/1 Running 2 (24h ago) 7d19h strimzi-cluster-operator-687b8c77db-8qx7w 1/1 Running 192 (13m ago) 8d mikata: ~/ % kubectl get svc -n kafka NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafdrop NodePort 10.96.39.41 <none> 9000:30080/TCP 7d18h my-cluster-kafka-bootstrap ClusterIP 10.96.62.244 <none> 9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-kafka-brokers ClusterIP None <none> 9090/TCP,9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-zookeeper-client ClusterIP 10.96.141.159 <none> 2181/TCP 7d19h my-cluster-zookeeper-nodes ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 7d19h
ࣄલ४උ ΧελϜϦιʔεͰ,BGLBͷ5PQJDΛ࡞͠·͢ɻ apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: labels: strimzi.io/cluster:
my-cluster name: my-topic namespace: kafka spec: config: {} partitions: 5 replicas: 1 topicName: my-topic mikata: ~/ % kubectl get kafkatopic my-topic -n kafka NAME CLUSTER PARTITIONS REPLICATION FACTOR READY my-topic my-cluster 5 1 True
ࣄલ४උ 1SPEVDFSͱ$POTVNFSͷΞϓϦΛσϓϩΠ͠·͢ɻ mikata: ~/ % kubectl get po -n
demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 6m27s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 6m42s mikata: ~/ % kubectl get svc -n demo NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafka-producer-service ClusterIP 10.96.193.144 <none> 80/TCP 7m34s ,BGLB 1SPEVDFS $POTVNFS
ࣄલ४උ 1SPEVDFS͕ϝοηʔδΛొɺ$POTVNFSϝοηʔδΛಡΈग़͠·͢ɻ ɾɾɾ for i := 0; i <
messageCount; i++ { wg.Add(1) i := i go func() { defer wg.Done() writeMessage(w, i) }() } wg.Wait() ɾɾɾ } func writeMessage(w *kafka.Writer, i int) { message := kafka.Message{Key: []byte(fmt.Sprintf("Key- %d", i)), Value: []byte(fmt.Sprintf("Message No: %d, Time: %s", i, time.Now()))} err := w.WriteMessages(context.Background(), message) ɾɾɾ } ɾɾɾ r := kafka.NewReader(kafka.ReaderConfig{ Brokers: []string{kafkaBroker}, GroupID: consumerGroup, Topic: kafkaTopic, MinBytes: 10e1, // 1KB MaxBytes: 10e6, // 10MB }) defer r.Close() go func() { for { m, err := r.ReadMessage(context.Background()) if err != nil { break } time.Sleep(time.Second * time.Duration(sleepSeconds)) } }() }
4DBMFE0CKFDUͷ࡞ )FMNΛར༻ͯ͠,&%"ΛΠϯετʔϧ͠·͢ɻ mikata: ~/ % helm repo add kedacore
https://kedacore.github.io/charts mikata: ~/ % helm repo update mikata: ~/ % kubectl create ns keda mikata: ~/ % helm install keda kedacore -ns keda mikata: ~/ % kubectl get po -n keda NAME READY STATUS RESTARTS AGE keda-admission-webhooks-7df9d947c4-jhc4z 1/1 Running 1 (3d1h ago) 10d keda-operator-96579d64c-5tj9q 1/1 Running 3 (3d1h ago) 10d keda-operator-metrics-apiserver-8758c5874-xcmqd 1/1 Running 2 (3d1h ago) 10d mikata: ~/ % kubectl get svc -n keda NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE keda-admission-webhooks ClusterIP 10.96.129.95 <none> 443/TCP 10d keda-operator ClusterIP 10.96.152.106 <none> 9666/TCP 10d keda-operator-metrics-apiserver ClusterIP 10.96.98.38 <none> 443/TCP,80/TCP 10d
4DBMFE0CKFDUͷ࡞ ΧελϜϦιʔεͰ4DBMFE0CKFDUΛ࡞͠·͢ɻ apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: kafka-consumer-scaler
namespace: demo spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: kafka-consumer pollingInterval: 10 cooldownPeriod: 30 idleReplicaCount: 0 minReplicaCount: 1 maxReplicaCount: 5 triggers: - type: kafka metadata: bootstrapServers: my-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 consumerGroup: g1 topic: my-topic lagThreshold: "10" offsetResetPolicy: latest allowIdleConsumers: "false" version: 1.0.0 pollingInterval: τϦΨʔͷϙʔϦϯάִؒ cooldownPeriod: εέʔϧόοΫͷػظؒ idleReplicaCount: ػ࣌ͷϨϓϦΧ minReplicaCount: ࠷খϨϓϦΧ maxReplicaCount: ࠷େϨϓϦΧ bootstrapServers: kafkaͷଓઌ consumeGroup: Consumer Group໊ topic: topic໊ lagThreshold εέʔϧ͢Δᮢ
4DBMFE0CKFDUͷ࡞ 4DBMFE0CKFDUΛσϓϩΠ͠·͕͢ɺ1PEʹมԽ͋Γ·ͤΜɻ mikata: ~/ % kubectl apply -f scaled-object.yaml
scaledobject.keda.sh/kafka-consumer-scaler created mikata: ~/ % kubectl get scaledobjects -n demo NAME SCALETARGETKIND SCALETARGETNAME MIN MAX TRIGGERS AUTHENTICATION READY ACTIVE FALLBACK AGE kafka-consumer-scaler apps/v1.Deployment kafka-consumer 1 5 kafka True True False 3m23s mikata: ~/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 2d19h kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: ~/ % kubectl get hpa -n demo NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE keda-hpa-kafka-consumer-scaler Deployment/kafka-consumer 5/10 (avg) 1 5 1 3m55s HPA͕࡞͞Ε·͢
εέʔϧͷ֬ೝ ΞϓϦʹෛՙΛ͔͚εέʔϧΛ֬ೝ͠·͢ɻ ҰͷϦΫΤετͰ1SPEVDFSݸͷϝοηʔδΛLBGLBʹొ͍ͯ͠·͢ɻ mikata: ~/ % kubectl port-forward svc/kafka-producer-service 8080:80
-n demo & [1] 10843 mikata: keda-kafka/ % Forwarding from 127.0.0.1:8080 -> 5550 Forwarding from [::1]:8080 -> 5550 mikata: ~/ % curl 127.0.0.1:8080 Handling connection for 8080 mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-6dvq7 1/1 Running 0 57s kafka-consumer-5468c48766-h4mfv 1/1 Running 0 42s kafka-consumer-5468c48766-h8vwt 1/1 Running 0 42s kafka-consumer-5468c48766-k4js6 1/1 Running 0 57s kafka-consumer-5468c48766-khqs6 1/1 Running 0 74s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h
·ͱΊ
·ͱΊ Πϯετʔϧɺઃఆͷखॱ͕γϯϓϧͰಋೖͷোน͍ͷͰɺखͬऔΓૣ͘Π ϕϯτۦಈͷγεςϜΛߏங͢Δͷʹྑ͍ͱࢥ͍·͢ɻ खܰͳ໘ɺϝοηʔδϒϩʔΧʔͱͷ*'ఏڙ͞Εͳ͍ͷͰɺϒϩʔΧʔͱΞ ϓϦέʔγϣϯ͕ີ݁߹ʹͳΓ͍͢ͱײ͡·ͨ͠ɻ େنͳ৫Ͱӡ༻͍ͯ͘͠߹ɺ͜ͷ͋ͨΓΛߟྀͭͭ͠ɺಋೖΛݕ౼͢ Δͷ͕ྑ͍ͷͰͳ͍͔ͱࢥ͍·͢ɻ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠