Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Design Philosophy of Kuroko2
Search
Naoto Takai
January 18, 2017
Programming
3
2k
The Design Philosophy of Kuroko2
Internal lightning talks in Cookpad
Naoto Takai
January 18, 2017
Tweet
Share
More Decks by Naoto Takai
See All by Naoto Takai
Building the Perfect Custom Keyboard
takai
0
360
キースイッチ潤滑入門 / A Brief Introduction to Lube MX Switches
takai
2
740
みなさん、分かっていますか / What is a difference?
takai
1
830
Agile and DevOps with Ruby
takai
2
990
Software Quality and Testing #1
takai
5
610
Microservices in Action
takai
37
5.2k
Ruby App on Netflix Microservices Stack
takai
6
990
How We Use Jenkins?
takai
21
8.1k
COOKPAD and Test Automation
takai
19
2.6k
Other Decks in Programming
See All in Programming
Anthropic Cookbook のおすすめレシピ
schroneko
7
970
雑に思考を整理する技術と効能
konifar
60
29k
Random\Randomizer クラスで日常のあれこれを解決しよう! / Random\Randomizer class solves familiar trouble
cocoeyes02
0
240
はてなにおける CSS Modules、及び CSS Modules に足りないもの / CSS Modules in Hatena, and CSS Modules missing parts
mizdra
7
930
SIMD Parallel Programming with the Vector API
josepaumard
0
180
ONE WEDGE_company_guide
1wedge_one
0
480
Hanami and htmx
bkuhlmann
0
210
ADRを一年運用してみた/adr_after_a_year
hanhan1978
7
2.4k
TCAとKMPを用いた新規動画配信アプリ 「ABEMA Live」の設計
tomu28
1
110
Milestoner
bkuhlmann
1
410
デフォルトにして至高、RubyMineの大好きな所
ruzia
0
390
サイコロで理解する統計的仮説検定の考え方
tatamiya
4
940
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
31
46k
What's new in Ruby 2.0
geeforr
337
31k
Faster Mobile Websites
deanohume
299
30k
A Philosophy of Restraint
colly
197
16k
Build your cross-platform service in a week with App Engine
jlugia
225
17k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
125
32k
How GitHub Uses GitHub to Build GitHub
holman
468
290k
Building Applications with DynamoDB
mza
88
5.6k
Into the Great Unknown - MozCon
thekraken
10
990
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.9k
Raft: Consensus for Rubyists
vanstee
132
6.3k
Teambox: Starting and Learning
jrom
128
8.4k
Transcript
The Design Philosophy of
• Web-based (not “enterprisy”) • Embedded workflow engine (ruote). •
Distributed architecture • with on demand EC2 provisioning. • communicates with message queue. Kuroko1
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Overview
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Pitfalls × × × × × × × × × ×
• It’s really hard to isolate the problem in a
distributed environment. • “Programmable” means programming is always needed. • Nothing is reliable except MySQL. Lessons from Kuroko1
There is data inconsistency between the actual state and the
stored state. Lessons from Kuroko1 (cont.)
State management is a key factor.
• Monolithic is the best. • Data oriented, not process
oriented. • Monitor as possible. Kuroko2
Monolithic Architecture kuroko-console kuoko-worker workflow engine worker process exec command
web interface DB
Kuroko2 Job State job_definitions job_instances tokens noop noop noop noop
/0-noop A job definition A launched job A job state
it do engine.process(token) expect(token.path).to eq '/0-noop' engine.process(token) expect(token.path).to eq '/1-noop'
engine.process(token) expect(token).to be_finished expect(Kuroko2::Token.all.count).to eq 0 end
Kuroko2 Execution State kuroko-worker kuroko-console executions workflow-processor command-executor exec command
loop do execution = Execution.unstarted.take() execution.touch(:started_at) execute_shell(execution.shell) execution.update(exit_status: exit_status, finished_at:
Time.current) execution.finish() end
kuroko-worker kuroko-console executions workflow-processor command-executor exec command Kuroko2 Pitfalls ×
× ×
Kuroko2 Worker Processes command-executor kill process monitor process shell process
exec command
def check_assignment_delay(exec_no_pid) if exec_no_pid.started_at < 1.minutes.ago Kuroko2::Notifications .not_assigned(exec_no_pid, @hostname).deliver_now exec_no_pid.touch(:mailed_at)
end end
def check_process_absence(execution) begin Process.kill(0, execution.pid) rescue Errno::EPERM true rescue Errno::ESRCH
if Execution.exists?(execution.id) notify_process_absence(execution) end end end
• /v1/stats/waiting_execution • /v1/stats/instance Kuroko2 Monitoring API
• State management is a key factor: • monolithic is
the best. • data oriented, not process oriented. • monitor as possible. Kuroko2
is open source product. We are waiting your contributions!