Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Ops Tool made by Perl Beginner #yapcasia 2012
Search
riywo
September 28, 2012
Programming
5
29k
Ops Tool made by Perl Beginner #yapcasia 2012
You can talk using this slide without any permission:)
riywo
September 28, 2012
Tweet
Share
More Decks by riywo
See All by riywo
Monitoring Casual #2
riywo
1
410
ServerSchemaCoreAPI
riywo
0
1.5k
Most Powerful High Availability / Load Balancing #yapcasia 2012
riywo
3
3.7k
Other Decks in Programming
See All in Programming
初学者でも今すぐできる、Claude Codeの生産性を10倍上げるTips
s4yuba
16
13k
iOS 26にアップデートすると実機でのHot Reloadができない?
umigishiaoi
0
140
Python型ヒント完全ガイド 初心者でも分かる、現代的で実践的な使い方
mickey_kubo
1
240
バイブスあるコーディングで ~PHP~ 便利ツールをつくるプラクティス
uzulla
1
140
「テストは愚直&&網羅的に書くほどよい」という誤解 / Test Smarter, Not Harder
munetoshi
1
200
PicoRuby on Rails
makicamel
3
150
ソフトウェア設計とAI技術の活用
masuda220
PRO
22
5.7k
SQLアンチパターン第2版 データベースプログラミングで陥りがちな失敗とその対策 / Intro to SQL Antipatterns 2nd
twada
PRO
23
6.9k
NEWT Backend Evolution
xpromx
1
140
「次に何を学べばいいか分からない」あなたへ──若手エンジニアのための学習地図
panda_program
3
520
20250708_JAWS_opscdk
takuyay0ne
2
130
Android 16KBページサイズ対応をはじめからていねいに
mine2424
0
480
Featured
See All Featured
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Facilitating Awesome Meetings
lara
54
6.5k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
840
How to train your dragon (web standard)
notwaldorf
96
6.1k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
4 Signs Your Business is Dying
shpigford
184
22k
Six Lessons from altMBA
skipperchong
28
3.9k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
Testing 201, or: Great Expectations
jmmastey
43
7.6k
Transcript
Ops Tool made by Perl Beginner @riywo Sep, 28, 2012
YAPC::Asia 2012
@riywo
http://www.publicdomainpictures.net/view-image.php?image=4314&picture=t-shirt-image
None
Perl Beginner
I Love Cats
DataBase Engineer MySQL
Operation Engineer DevOps
Operation Engineer?
Trouble Shooting Design System Setup Server On Calling Isu nage
Sync Info Operation Inspection Develop Tools Capacity Plan
Trouble Shooting Design System Setup Server On Calling Isu nage?
Sync Info Operation Inspection Develop Tools Capacity Plan
=Throwing Chair :-p
F*cking Query SELECT SQL_CALC_FOUD_ROWS * FROM table WHERE c1 >
1000 OR c2 != 100 ORDER BY c3 LIMIT 10 OFFSET 100000;
http://www.flickr.com/photos/andrew_freese/2200774154/ Operation Engineer
dev> I did nothing..., but....
ops> What did you do? dev> I did nothing..., but....
http://www.flickr.com/photos/jaydoubleyougee/2259260845/ Operation Engineer
I Love Cats
But...., do they only happen in Dev side?
Operation Error!!!!!!!!!!!!!
Operation Mistake # rm -fr /tmp/* # rm -fr /
tmp/*
Capacity Over Sorry, we have no more servers....
...and... Setup Error
server_id = 1212 default_character_set=sjis innodb_buffer_pool_size=2G server_id = 1213 default_character_set=utf8 innodb_buffer_pool_size=3G
Can’t copy the same file to all servers => edit
manually
Can’t copy the same file to all servers => edit
manually M istake!!!
Current servers’ sysctl changed But setup operation did not follow
it...
Current servers’ sysctl changed But setup operation did not follow
it... M istake!!!
http://www.flickr.com/photos/clickflashphotos/3402287993/
Why? Are you a Professional?
Someone has “Setup Procedure Manual” 1. OS install 1.1 IP
address 1.2 nameserver 1.3 OS user 2. SSH Configuration 2.1 ... 3. Install Perl 3.1 ...
Someone have “Setup Procedure Manual” 1. OS install 1.1 IP
address 1.2 nameserver 1.3 OS user 2. SSH Configuration 2.1 ... 3. Install Perl 3.1 ... Operate Manually Check Visually
# ssh $TGT cat /etc/my.cnf | diff -u /etc/my.cnf -
- server_id = 2323 + server_id = 2324 - log_slave_updaets Yes, this difference is OK If $TGT is Master DB this difference is NG
# ssh $TGT cat /etc/my.cnf | diff -u /etc/my.cnf -
- server_id = 2323 + server_id = 2324 - log_slave_updaets Yes, this difference is OK If $TGT is Master DB this difference is NG Impossible
OK, Setup scripts help us!
OK, Setup scripts help us! ✗
Patch, Patch, Patch... patch for a new component many times,
setup script become a spaghetti code...
Script Bug OpsEngi usually not good at programing, the script
will have many many bugs...
Unfollow current change For trouble, servers’ config were changed, but
setup script isn’t changed...
No Test!!!! F*ck!!! usually setup script is simple shell script,
only listed exec commands no test
No Test!!!! F*ck!!! usually setup script is simple shell script,
only listed exec commands no test
Check return code Check status of servers ✗
Describe
Server Correct Status /etc/sysconfig/ /etc/my.cnf /etc/sysctrl.conf /etc/ssh/ssh_config / /etc/cron.d /etc/resolv.conf
/etc/yum.repos.d/ /service/app/run chkconfig ulimit httpd.conf rpms
If you describe the correct status you can also test
current servers!
Correct Status New Server Server Server Server Server Server
Correct Status New Server Server Server Server Server Server Test
Setup
Break Time :) Remain 10min?
Correct Status New Server Server Server Server Server Server Test
Setup
http://www.flickr.com/photos/stocktoc/4539506889/ Touryo ౩ྊNBTUFS DBSQFOUFSPGTFSWFST
Describe
Orchestration Configuration Bootstrapping Server Setup
Orchestration Configuration Bootstrapping Server Setup Data sync, LB configuration OS
install
Orchestration Configuration Bootstrapping Server Setup Configure, Install Middleware
Describe test/setup procedures DSL = blueprint.pl (cron,httpd,mysql,...) test { my
$c = shift; ok ... }; run { my $c = shift; $c->ssh(...) };
server_id = <: $server_id :> default_character_set= <: $char :> Server’s
my.cnf Text::Xslate::render Test example diff
server_id = <: $server_id :> default_character_set= <: $char :> Server’s
my.cnf Text::Xslate::render Run example SSH install
server1 => mbga:web server2 => mbga:db:master server3 => game1:web Describe
role of nodes
Describe blueprints of roles mbga:web => [cron, httpd] mbga:db:master =>
[cron, mysql] game1:web => [cron, httpd]
Describe attribute of roles mbga:web => { cron_hour => 1,
proc_num => 50, } mbga:db:master => {...} game1:web => {...}
How to develop?
Make time
Research
Small snippet
Ask Devs!
Lessons for Manage Tool Development
Maintenance Can you hand over its maintenance?
Touryo blueprint are written by other staffs :)
Simple The UNIX philosophy
Touryo only care about server configuration. Combine other tools ex.
AdminTool(node-role)
Replace Is it able to replace easily?
By Touryo, undocumented procedures have been described. Easier to replace!
Any Questions?
Thanks!
@riywo blog.riywo.com weblog.riywo.com