Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
Search
mekka
May 27, 2023
Technology
1
400
KEDAで始めるイベント駆動システム #k8snovice / keda-tutorial
イベント駆動システムを構築するツールとしてKEDAを調査したものです
mekka
May 27, 2023
Tweet
Share
More Decks by mekka
See All by mekka
ArgoCDによるGitOps導入 / ArgoCD GitOps
chmikata
0
170
新サービス立ち上げに向けたCI/CD環境の構築
chmikata
0
2.7k
rakusmeetup-number-4-operation
chmikata
1
670
rakusmeetup-number-4-infrastructure
chmikata
0
600
Other Decks in Technology
See All in Technology
Snowflakeの生成AI機能を活用したデータ分析アプリの作成 〜Cortex AnalystとCortex Searchの活用とStreamlitアプリでの利用〜
nayuts
1
420
AI駆動開発に向けた新しいエンジニアマインドセット
kazue
0
300
Kubernetes における cgroup v2 でのOut-Of-Memory 問題の解決
pfn
PRO
0
470
企業の生成AIガバナンスにおけるエージェントとセキュリティ
lycorptech_jp
PRO
2
110
2025年になってもまだMySQLが好き
yoku0825
8
4.3k
Webアプリケーションにオブザーバビリティを実装するRust入門ガイド
nwiizo
3
440
Bye-Bye Query Spaghetti: Write Queries You'll Actually Understand Using Pipelined SQL Syntax
tobiaslampertlotum
0
150
開発者を支える Internal Developer Portal のイマとコレカラ / To-day and To-morrow of Internal Developer Portals: Supporting Developers
aoto
PRO
1
410
実践!カスタムインストラクション&スラッシュコマンド
puku0x
0
270
バイブスに「型」を!Kent Beckに学ぶ、AI時代のテスト駆動開発
amixedcolor
2
380
Platform開発が先行する Platform Engineeringの違和感
kintotechdev
3
510
【実演版】カンファレンス登壇者・スタッフにこそ知ってほしいマイクの使い方 / 大吉祥寺.pm 2025
arthur1
1
420
Featured
See All Featured
jQuery: Nuts, Bolts and Bling
dougneiner
64
7.9k
Optimizing for Happiness
mojombo
379
70k
Gamification - CAS2011
davidbonilla
81
5.4k
Faster Mobile Websites
deanohume
309
31k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Producing Creativity
orderedlist
PRO
347
40k
Reflections from 52 weeks, 52 projects
jeffersonlam
352
21k
Visualization
eitanlees
148
16k
Unsuck your backbone
ammeep
671
58k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
GraphQLとの向き合い方2022年版
quramy
49
14k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Transcript
,&%"Ͱ࢝ΊΔΠϕϯτۦಈγεςϜ ,VCFSOFUFT/PWJDF5PLZP
ͲͳͨͰ͔͢ʁ w $IJLBIJTB.JLBUBʢ5XJUUFS!NFMQP@NFMʣ w #UP#ͷ4BB4ձࣾʹॴଐ ͳʹͯ͠ΔਓͰ͔͢ʁ w όοΫΤϯυ։ൃ͕ϝΠϯɺ43&ͪΐΖͬͱ w ,VCFSOFUFTֶशதͷʔʔͰ͢
ࣗݾհ
w ,&%"ͱ w ,&%"ͷػೳͱಛ w ,&%"ͷ͍ํ w ·ͱΊ ຊͷൃද༰
,&%"ͱ
,&%"ͱ ,VCFSOFUFT&WFOUESJWF"VUPTDBMJOHʢ,&%"ʣΠϕϯτۦಈͷ ΦʔτεέʔϧΛ࣮ݱ͢ΔίϯϙʔωϯτͰ͢ɻ ʹ$/$'ͷ4BOECPYϓϩδΣΫτͱͳ͓ͬͯΓɺݱࡏ *ODVCBUJOHϓϩδΣΫτͱͳ͍ͬͯ·͢ɻ
,&%"ͱ ҙͷ,VCFSOFUFTΫϥελʹՃͰ͖ΔܰྔίϯϙʔωϯτͰɺ )1"ͳͲඪ४ͷ,VCFSOFUFTίϯϙʔωϯτͱ࿈ܞͯ͠ಈ࡞͠·͢ɻ ΦʔτεέʔϧʹػೳΛߜ͍ͬͯΔͨΊγϯϓϧͳߏͰɺগͳ͍Ϧ ιʔεͰಈ࡞ͤ͞Δ͜ͱ͕Ͱ͖·͢ɻ
,&%"ͷػೳͱಛ
,&%"ͷػೳͱಛ w Πϕϯτͷঢ়ଶʹ߹Θͤͯ1PEΛεέʔϧͰ͖·͢ɻΠϕϯτ͕ൃੜ͍ͯ͠ͳ ͍߹1PEΛʹͰ͖ΔͷϦιʔεΛ༗ޮʹ׆༻Ͱ͖·͢ɻ w Ωϡʔ%#ͳͲ༷ʑͳΠϕϯτϦιʔεΛτϦΨʔʹεέʔϧ͕ՄೳͰ͢ɻ ରԠ͍ͯ͠ΔΠϕϯτϦιʔεLBGLB 1VMTBS .Z42- 1PTUHSF42-ͳͲ
༷ʑͰ͢ɻৄࡉެࣜΛ֬ೝͯ͠Έ͍ͯͩ͘͞ɻ w ΈࠐΈͷεέʔϥʔʢΠϕϯτϦιʔεΛࢹ͢ΔΈʣͷΘΓʹಠࣗ ͷεέʔϥʔΛఆٛ͢Δ͜ͱ͕ՄೳͰ͢ɻ
w LFEBPQFSBUPS 1PEΛ͔Βεέʔϧ͠·͢ɻ w LFEBPQFSBUPSNFUSJDTBQJTFSWFS ΠϕϯτσʔλΛ)1"ʹެ։͢Δ͜ͱ Ͱ1PEΛ͔ΒOεέʔϧ͠·͢ɻ w
LFEBBENJTTJPOXFCIPPLT ϦιʔεͷมߋΛݕূ͠ߏϛεΛ ͗·͢ɻ ΞʔΩςΫνϟ
ΧελϜϦιʔε w 4DBMFE0CKFDU %FQMPZNFOUͱ4UBUFGVM4FUΛεέʔϦϯά͠·͢ɻ w 4DBMFE+PC +PCΛεέʔϦϯά͠·͢ɻ w
5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ ೝূใΛఆٛɺ4DMBFE0CKFDU͔ΒࢀরՄೳʹ͠·͢ɻ w $MVTUFS5SJHHFS"VUIFOUJDBUJPOʢৄ͘͠৮Ε·ͤΜʣ /NBFTQBDFΛލ͍Ͱར༻ՄೳͳೝূใΛఆٛ͠·͢ɻ
Ϣʔεέʔε w 4DBMFE0CKFDU Ωϡʔͷ͞ʹ߹Θͤͯ1PEΛεέʔϧ͠·͢ɻΩϡʔ͕ۭʹͳΔ·Ͱ1PE ॲཧΛଓߦ͠·͢ɻ ࣌ؒʹେྔͷϦΫΤετΛॲཧ͢ΔέʔεͰར༻͠·͢ɻ w 4DBMFE+PC
ΩϡʔͷΠϕϯτʹରͯͭ͠ͷ1PEΛىಈ͠·͢ɻॲཧ͕ΓΔͱ 1PEऴྃ͠·͢ɻ ࣌ؒͷॲཧΛ࣮ߦ͢ΔΑ͏ͳέʔεͰར༻͠·͢ɻ
,&%"ͷ͍ํ
,&%" ,BGLB IUUQTHJUIVCDPN $POUBJOFS4PMVUJPOTLFEBLBGLB ͪ͜ΒͰެ։͞Ε͍ͯΔαϯϓϧΛར༻͠ ͯڥΛߏங͠·͢ɻ w 1SPEVDFSͷ1PE,BGLBͷΩϡʔʹϝ οηʔδΛొ͠·͢ɻ w
,BGLBͷΩϡʔʹ߹Θͤͯ$POTVNFSͷ 1PEΛͭ·Ͱεέʔϧ͠·͢ɻ w ,BGLBͷΩϡʔ͕ۭʹͳΔͱ$POTVNFS ͷ1PEʹεέʔϧ͠·͢ɻ ΰʔϧΠϝʔδ %FQMPZNFOU 1SPEVDFS $POTVNFS
ࣄલ४උ 4USJN[JΛར༻ͯ͠,BGLBΫϥελΛߏங͠·͢ɻ mikata: ~/ % kubectl get pods -n
kafka NAME READY STATUS RESTARTS AGE kafdrop-5549954bdb-z24kv 1/1 Running 2 (24h ago) 7d18h my-cluster-entity-operator-5d8bccc5d4-8szq4 3/3 Running 6 (24h ago) 7d19h my-cluster-kafka-0 1/1 Running 2 (24h ago) 7d19h my-cluster-zookeeper-0 1/1 Running 2 (24h ago) 7d19h strimzi-cluster-operator-687b8c77db-8qx7w 1/1 Running 192 (13m ago) 8d mikata: ~/ % kubectl get svc -n kafka NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafdrop NodePort 10.96.39.41 <none> 9000:30080/TCP 7d18h my-cluster-kafka-bootstrap ClusterIP 10.96.62.244 <none> 9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-kafka-brokers ClusterIP None <none> 9090/TCP,9091/TCP,9092/TCP,9093/TCP 7d19h my-cluster-zookeeper-client ClusterIP 10.96.141.159 <none> 2181/TCP 7d19h my-cluster-zookeeper-nodes ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 7d19h
ࣄલ४උ ΧελϜϦιʔεͰ,BGLBͷ5PQJDΛ࡞͠·͢ɻ apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: labels: strimzi.io/cluster:
my-cluster name: my-topic namespace: kafka spec: config: {} partitions: 5 replicas: 1 topicName: my-topic mikata: ~/ % kubectl get kafkatopic my-topic -n kafka NAME CLUSTER PARTITIONS REPLICATION FACTOR READY my-topic my-cluster 5 1 True
ࣄલ४උ 1SPEVDFSͱ$POTVNFSͷΞϓϦΛσϓϩΠ͠·͢ɻ mikata: ~/ % kubectl get po -n
demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 6m27s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 6m42s mikata: ~/ % kubectl get svc -n demo NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kafka-producer-service ClusterIP 10.96.193.144 <none> 80/TCP 7m34s ,BGLB 1SPEVDFS $POTVNFS
ࣄલ४උ 1SPEVDFS͕ϝοηʔδΛొɺ$POTVNFSϝοηʔδΛಡΈग़͠·͢ɻ ɾɾɾ for i := 0; i <
messageCount; i++ { wg.Add(1) i := i go func() { defer wg.Done() writeMessage(w, i) }() } wg.Wait() ɾɾɾ } func writeMessage(w *kafka.Writer, i int) { message := kafka.Message{Key: []byte(fmt.Sprintf("Key- %d", i)), Value: []byte(fmt.Sprintf("Message No: %d, Time: %s", i, time.Now()))} err := w.WriteMessages(context.Background(), message) ɾɾɾ } ɾɾɾ r := kafka.NewReader(kafka.ReaderConfig{ Brokers: []string{kafkaBroker}, GroupID: consumerGroup, Topic: kafkaTopic, MinBytes: 10e1, // 1KB MaxBytes: 10e6, // 10MB }) defer r.Close() go func() { for { m, err := r.ReadMessage(context.Background()) if err != nil { break } time.Sleep(time.Second * time.Duration(sleepSeconds)) } }() }
4DBMFE0CKFDUͷ࡞ )FMNΛར༻ͯ͠,&%"ΛΠϯετʔϧ͠·͢ɻ mikata: ~/ % helm repo add kedacore
https://kedacore.github.io/charts mikata: ~/ % helm repo update mikata: ~/ % kubectl create ns keda mikata: ~/ % helm install keda kedacore -ns keda mikata: ~/ % kubectl get po -n keda NAME READY STATUS RESTARTS AGE keda-admission-webhooks-7df9d947c4-jhc4z 1/1 Running 1 (3d1h ago) 10d keda-operator-96579d64c-5tj9q 1/1 Running 3 (3d1h ago) 10d keda-operator-metrics-apiserver-8758c5874-xcmqd 1/1 Running 2 (3d1h ago) 10d mikata: ~/ % kubectl get svc -n keda NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE keda-admission-webhooks ClusterIP 10.96.129.95 <none> 443/TCP 10d keda-operator ClusterIP 10.96.152.106 <none> 9666/TCP 10d keda-operator-metrics-apiserver ClusterIP 10.96.98.38 <none> 443/TCP,80/TCP 10d
4DBMFE0CKFDUͷ࡞ ΧελϜϦιʔεͰ4DBMFE0CKFDUΛ࡞͠·͢ɻ apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: kafka-consumer-scaler
namespace: demo spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: kafka-consumer pollingInterval: 10 cooldownPeriod: 30 idleReplicaCount: 0 minReplicaCount: 1 maxReplicaCount: 5 triggers: - type: kafka metadata: bootstrapServers: my-cluster-kafka-bootstrap.kafka.svc.cluster.local:9092 consumerGroup: g1 topic: my-topic lagThreshold: "10" offsetResetPolicy: latest allowIdleConsumers: "false" version: 1.0.0 pollingInterval: τϦΨʔͷϙʔϦϯάִؒ cooldownPeriod: εέʔϧόοΫͷػظؒ idleReplicaCount: ػ࣌ͷϨϓϦΧ minReplicaCount: ࠷খϨϓϦΧ maxReplicaCount: ࠷େϨϓϦΧ bootstrapServers: kafkaͷଓઌ consumeGroup: Consumer Group໊ topic: topic໊ lagThreshold εέʔϧ͢Δᮢ
4DBMFE0CKFDUͷ࡞ 4DBMFE0CKFDUΛσϓϩΠ͠·͕͢ɺ1PEʹมԽ͋Γ·ͤΜɻ mikata: ~/ % kubectl apply -f scaled-object.yaml
scaledobject.keda.sh/kafka-consumer-scaler created mikata: ~/ % kubectl get scaledobjects -n demo NAME SCALETARGETKIND SCALETARGETNAME MIN MAX TRIGGERS AUTHENTICATION READY ACTIVE FALLBACK AGE kafka-consumer-scaler apps/v1.Deployment kafka-consumer 1 5 kafka True True False 3m23s mikata: ~/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-hnzl2 1/1 Running 0 2d19h kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: ~/ % kubectl get hpa -n demo NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE keda-hpa-kafka-consumer-scaler Deployment/kafka-consumer 5/10 (avg) 1 5 1 3m55s HPA͕࡞͞Ε·͢
εέʔϧͷ֬ೝ ΞϓϦʹෛՙΛ͔͚εέʔϧΛ֬ೝ͠·͢ɻ ҰͷϦΫΤετͰ1SPEVDFSݸͷϝοηʔδΛLBGLBʹొ͍ͯ͠·͢ɻ mikata: ~/ % kubectl port-forward svc/kafka-producer-service 8080:80
-n demo & [1] 10843 mikata: keda-kafka/ % Forwarding from 127.0.0.1:8080 -> 5550 Forwarding from [::1]:8080 -> 5550 mikata: ~/ % curl 127.0.0.1:8080 Handling connection for 8080 mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-consumer-5468c48766-6dvq7 1/1 Running 0 57s kafka-consumer-5468c48766-h4mfv 1/1 Running 0 42s kafka-consumer-5468c48766-h8vwt 1/1 Running 0 42s kafka-consumer-5468c48766-k4js6 1/1 Running 0 57s kafka-consumer-5468c48766-khqs6 1/1 Running 0 74s kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h mikata: keda-kafka/ % kubectl get po -n demo NAME READY STATUS RESTARTS AGE kafka-producer-64696bbdd7-9b2xt 1/1 Running 0 2d20h
·ͱΊ
·ͱΊ Πϯετʔϧɺઃఆͷखॱ͕γϯϓϧͰಋೖͷোน͍ͷͰɺखͬऔΓૣ͘Π ϕϯτۦಈͷγεςϜΛߏங͢Δͷʹྑ͍ͱࢥ͍·͢ɻ खܰͳ໘ɺϝοηʔδϒϩʔΧʔͱͷ*'ఏڙ͞Εͳ͍ͷͰɺϒϩʔΧʔͱΞ ϓϦέʔγϣϯ͕ີ݁߹ʹͳΓ͍͢ͱײ͡·ͨ͠ɻ େنͳ৫Ͱӡ༻͍ͯ͘͠߹ɺ͜ͷ͋ͨΓΛߟྀͭͭ͠ɺಋೖΛݕ౼͢ Δͷ͕ྑ͍ͷͰͳ͍͔ͱࢥ͍·͢ɻ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠