Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
The Design Philosophy of Kuroko2
Naoto Takai
January 18, 2017
Programming
3
1.7k
The Design Philosophy of Kuroko2
Internal lightning talks in Cookpad
Naoto Takai
January 18, 2017
Tweet
Share
More Decks by Naoto Takai
See All by Naoto Takai
キースイッチ潤滑入門 / A Brief Introduction to Lube MX Switches
takai
2
670
みなさん、分かっていますか / What is a difference?
takai
1
710
Agile and DevOps with Ruby
takai
2
910
Software Quality and Testing #1
takai
5
580
Microservices in Action
takai
37
4.9k
Ruby App on Netflix Microservices Stack
takai
6
950
How We Use Jenkins?
takai
21
7.7k
COOKPAD and Test Automation
takai
19
2.5k
Continuous Delivery in COOKPAD
takai
36
28k
Other Decks in Programming
See All in Programming
There's an API for that!
mariatta
PRO
0
110
RustのWebフレームワーク周りの概観
hayao
0
180
Rに管理されてみる
kazutan
0
250
Amazon Lookout for Visionで 筆跡鑑定してみた
cmnakamurashogo
0
160
FargateとAthenaで作る、機械学習システム
nayuts
0
170
Rust、何もわからない...#3
estie
0
160
MLOps勉強会_20220810
strsaito
0
330
ESM移行は無理だけどおれもSindreのライブラリが使いたい!
sosukesuzuki
2
540
Regular expressions basics/正規表現の基本
kishikawakatsumi
6
260
SAM × Dockerでサーバーレス開発が超捗った話
yu_yukk_y
1
320
kintone × LINE Bot で餃子検定Botを作った話
naberina
0
330
動画合成アーキテクチャを実装してみて
satorunooshie
0
550
Featured
See All Featured
The Web Native Designer (August 2011)
paulrobertlloyd
75
2k
What’s in a name? Adding method to the madness
productmarketing
11
1.6k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
269
12k
Embracing the Ebb and Flow
colly
73
3.4k
Design by the Numbers
sachag
271
17k
Rebuilding a faster, lazier Slack
samanthasiow
62
7.3k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
15
980
The Power of CSS Pseudo Elements
geoffreycrofte
47
4k
Streamline your AJAX requests with AmplifyJS and jQuery
dougneiner
127
8.5k
Statistics for Hackers
jakevdp
782
210k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
236
1.1M
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
15k
Transcript
The Design Philosophy of
• Web-based (not “enterprisy”) • Embedded workflow engine (ruote). •
Distributed architecture • with on demand EC2 provisioning. • communicates with message queue. Kuroko1
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Overview
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Pitfalls × × × × × × × × × ×
• It’s really hard to isolate the problem in a
distributed environment. • “Programmable” means programming is always needed. • Nothing is reliable except MySQL. Lessons from Kuroko1
There is data inconsistency between the actual state and the
stored state. Lessons from Kuroko1 (cont.)
State management is a key factor.
• Monolithic is the best. • Data oriented, not process
oriented. • Monitor as possible. Kuroko2
Monolithic Architecture kuroko-console kuoko-worker workflow engine worker process exec command
web interface DB
Kuroko2 Job State job_definitions job_instances tokens noop noop noop noop
/0-noop A job definition A launched job A job state
it do engine.process(token) expect(token.path).to eq '/0-noop' engine.process(token) expect(token.path).to eq '/1-noop'
engine.process(token) expect(token).to be_finished expect(Kuroko2::Token.all.count).to eq 0 end
Kuroko2 Execution State kuroko-worker kuroko-console executions workflow-processor command-executor exec command
loop do execution = Execution.unstarted.take() execution.touch(:started_at) execute_shell(execution.shell) execution.update(exit_status: exit_status, finished_at:
Time.current) execution.finish() end
kuroko-worker kuroko-console executions workflow-processor command-executor exec command Kuroko2 Pitfalls ×
× ×
Kuroko2 Worker Processes command-executor kill process monitor process shell process
exec command
def check_assignment_delay(exec_no_pid) if exec_no_pid.started_at < 1.minutes.ago Kuroko2::Notifications .not_assigned(exec_no_pid, @hostname).deliver_now exec_no_pid.touch(:mailed_at)
end end
def check_process_absence(execution) begin Process.kill(0, execution.pid) rescue Errno::EPERM true rescue Errno::ESRCH
if Execution.exists?(execution.id) notify_process_absence(execution) end end end
• /v1/stats/waiting_execution • /v1/stats/instance Kuroko2 Monitoring API
• State management is a key factor: • monolithic is
the best. • data oriented, not process oriented. • monitor as possible. Kuroko2
is open source product. We are waiting your contributions!