Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Design Philosophy of Kuroko2
Search
Naoto Takai
January 18, 2017
Programming
3
2.2k
The Design Philosophy of Kuroko2
Internal lightning talks in Cookpad
Naoto Takai
January 18, 2017
Tweet
Share
More Decks by Naoto Takai
See All by Naoto Takai
Building the Perfect Custom Keyboard
takai
1
610
キースイッチ潤滑入門 / A Brief Introduction to Lube MX Switches
takai
2
800
みなさん、分かっていますか / What is a difference?
takai
1
910
Agile and DevOps with Ruby
takai
2
1.1k
Software Quality and Testing #1
takai
6
680
Microservices in Action
takai
37
5.3k
Ruby App on Netflix Microservices Stack
takai
6
1.1k
How We Use Jenkins?
takai
21
8.4k
COOKPAD and Test Automation
takai
19
2.8k
Other Decks in Programming
See All in Programming
第9回 情シス転職ミートアップ 株式会社IVRy(アイブリー)の紹介
ivry_presentationmaterials
1
230
技術同人誌をMCP Serverにしてみた
74th
0
280
Kotlin エンジニアへ送る:Swift 案件に参加させられる日に備えて~似てるけど色々違う Swift の仕様 / from Kotlin to Swift
lovee
1
250
「ElixirでIoT!!」のこれまでとこれから
takasehideki
0
370
Azure AI Foundryではじめてのマルチエージェントワークフロー
seosoft
0
130
AIコーディング道場勉強会#2 君(エンジニア)たちはどう生きるか
misakiotb
1
240
設計やレビューに悩んでいるPHPerに贈る、クリーンなオブジェクト設計の指針たち
panda_program
6
1.2k
deno-redisの紹介とJSRパッケージの運用について (toranoana.deno #21)
uki00a
0
140
Haskell でアルゴリズムを抽象化する / 関数型言語で競技プログラミング
naoya
17
4.9k
iOSアプリ開発で 関数型プログラミングを実現する The Composable Architectureの紹介
yimajo
2
210
Blazing Fast UI Development with Compose Hot Reload (droidcon New York 2025)
zsmb
1
190
ruby.wasmで多人数リアルタイム通信ゲームを作ろう
lnit
2
250
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
462
33k
The Straight Up "How To Draw Better" Workshop
denniskardys
233
140k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
The Cult of Friendly URLs
andyhume
79
6.5k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Become a Pro
speakerdeck
PRO
28
5.4k
Making the Leap to Tech Lead
cromwellryan
134
9.3k
Side Projects
sachag
455
42k
Transcript
The Design Philosophy of
• Web-based (not “enterprisy”) • Embedded workflow engine (ruote). •
Distributed architecture • with on demand EC2 provisioning. • communicates with message queue. Kuroko1
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Overview
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Pitfalls × × × × × × × × × ×
• It’s really hard to isolate the problem in a
distributed environment. • “Programmable” means programming is always needed. • Nothing is reliable except MySQL. Lessons from Kuroko1
There is data inconsistency between the actual state and the
stored state. Lessons from Kuroko1 (cont.)
State management is a key factor.
• Monolithic is the best. • Data oriented, not process
oriented. • Monitor as possible. Kuroko2
Monolithic Architecture kuroko-console kuoko-worker workflow engine worker process exec command
web interface DB
Kuroko2 Job State job_definitions job_instances tokens noop noop noop noop
/0-noop A job definition A launched job A job state
it do engine.process(token) expect(token.path).to eq '/0-noop' engine.process(token) expect(token.path).to eq '/1-noop'
engine.process(token) expect(token).to be_finished expect(Kuroko2::Token.all.count).to eq 0 end
Kuroko2 Execution State kuroko-worker kuroko-console executions workflow-processor command-executor exec command
loop do execution = Execution.unstarted.take() execution.touch(:started_at) execute_shell(execution.shell) execution.update(exit_status: exit_status, finished_at:
Time.current) execution.finish() end
kuroko-worker kuroko-console executions workflow-processor command-executor exec command Kuroko2 Pitfalls ×
× ×
Kuroko2 Worker Processes command-executor kill process monitor process shell process
exec command
def check_assignment_delay(exec_no_pid) if exec_no_pid.started_at < 1.minutes.ago Kuroko2::Notifications .not_assigned(exec_no_pid, @hostname).deliver_now exec_no_pid.touch(:mailed_at)
end end
def check_process_absence(execution) begin Process.kill(0, execution.pid) rescue Errno::EPERM true rescue Errno::ESRCH
if Execution.exists?(execution.id) notify_process_absence(execution) end end end
• /v1/stats/waiting_execution • /v1/stats/instance Kuroko2 Monitoring API
• State management is a key factor: • monolithic is
the best. • data oriented, not process oriented. • monitor as possible. Kuroko2
is open source product. We are waiting your contributions!