Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Configuration Management 101
Search
someara
February 04, 2014
Programming
3
630
Configuration Management 101
Given at FOSDEM 2014
someara
February 04, 2014
Tweet
Share
More Decks by someara
See All by someara
Docker Docker Docker Chef
someara
0
340
Hand Crafted Artisinal Chef Resources
someara
0
450
Configuration Management Camp 2015
someara
1
420
Cookbook Reusability
someara
0
420
TDI with ChefDK 0.0.1
someara
2
490
Configuration Management 101 @ Scale12x
someara
2
740
Introduction to Chef - Scale 10x
someara
2
380
Introduction to Chef - NYLUG Jan 2012
someara
2
250
Introduction to Chef - LISA11
someara
10
6k
Other Decks in Programming
See All in Programming
Compose-View Interop in Practice (mDevCamp 2024)
stewemetal
0
170
CDKコントリビュートの最初の壁を越えよう! -簡単issueの見つけ方-
badmintoncryer
3
210
業務ツールとして使うPostman
msys75
0
100
スクラムガイドのスプリントレトロスペクティブを改めて読みかえしてみた / Re-reading the Sprint Retrospective Section in the Scrum Guide
mackey0225
3
480
GitLab CI/CD で C#/WPFアプリケーションのテストとインストーラーのビルド・デプロイを自動化する
hacarus
0
280
新宿ダンジョンを可視化してみた
satoshi7190
3
390
Hanami and htmx
bkuhlmann
0
220
大規模Reactアプリのリアーキテクチャ~8万行のTanStack Query移行の軌跡~
kj455
4
1k
デフォルトにして至高、RubyMineの大好きな所
ruzia
0
770
FigmaとPHPで作る1ミリたりとも表示崩れしない最強の帳票印刷ソリューション
ttskch
43
19k
Implementing Design Systems in Swift
seyfoyun
1
460
Amazon SQSコンシューマー疎結合への旅 - 出張! #DevelopersIO IT技術ブログの中の人が語る勉強会 #3
quiver
0
310
Featured
See All Featured
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
Music & Morning Musume
bryan
41
5.6k
Unsuck your backbone
ammeep
663
57k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
41
4.4k
YesSQL, Process and Tooling at Scale
rocio
165
13k
Agile that works and the tools we love
rasmusluckow
325
20k
The Brand Is Dead. Long Live the Brand.
mthomps
49
29k
A Philosophy of Restraint
colly
197
16k
Six Lessons from altMBA
skipperchong
22
3k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
358
22k
KATA
mclloyd
16
12k
How to train your dragon (web standard)
notwaldorf
75
5.2k
Transcript
Configuration Management 101! FOSDEM 2014
Sean OMeara!
[email protected]
! @someara
Sean OMeara!
[email protected]
! @someara
Sean OMeara!
[email protected]
! @someara
whoami
Part 1
The Dawn of Configuration Management
• What is configuration management?! • Strategies and techniques for
managing configuration and its complexity! • The art of change management
Manual Configuration
• Intuitive! • How we all start out! • Log
into machine, manipulate with fingers! • Make with the clicky clicky! • Long tradition
• Somehow feels the “safest"! • First instinct in emergencies!
• This is an illusion! • Do not do this
• Labor intensive! • Error prone! • Difficult to reproduce!
• Obviously unsustainable
Scripting
• setup.sh! • setup.pl! • setup.py! • setup.rb
• doit.sh! • doit.pl! • doit.py! • doit.rb
• Ad-hoc in nature! • Loss of history! • Lacks
testing methodology! • A step in the right direction
File Distribution
• NFS! • SMB! • AFS! • SSHFS! • GlusterFS
• uucp! • rcp! • ftp! • http! • scp
•Distributed systems! •Shares often managed manually or with scripts! •Package
repositories! •Pull is better than push! •Scp on a cron *
Execution Management
• Image management! • Snapshots and cloning! • Containers
• SSH on a for loop! • Func! • Commands
on message queues! • ISConf
• Loss of history! • Image sprawl! • Easy to
order change across nodes
Convergent Operators! (promises)
None
Tools
• CFEngine! • Bcfg2! • Puppet! • Chef! • Salt!
• Ansible
None
Part 2
Policy http://www.flickr.com/photos/sfllaw/222795669/
• /etc/passwd should be mode 0644! • /etc/shadow should be
mode 0600
• user ‘kermit’ should exist! • user ‘fonzi’ should exist!
• group ‘muppets’ should exist! • group ‘muppets’ should contain kermit and fonzi
• package ‘ntpd’ should be installed! • ntpd should sync
with our AD service! • service ‘ntpd’ should be running
• package ‘httpd’ should be installed! • httpd should be
expose /mnt/software/java! • service ‘httpd’ should be running
• The Java JDK, version 7u45, found on an internally
hosted web server, should be installed into /usr/local/jdk-7u45/
Polices are declarations about the state of things in a
system
Polices are applied repeatedly and repair the system when needed
Policies often change
• package ‘widget-factory’ should be installed at version 1.2.3
• package ‘widget-factory’ should be installed at version 1.3.0
http://www.flickr.com/photos/jakepjohnson/4937767595 Repeatability
Repeatable -> Idempotent -> Convergent
• Scripts are not generally repeatable
None
None
• But they can be!
None
! Idempotent operations can be applied infinite times and will
yield the same result every time
None
Idempotent http://www.flickr.com/photos/ian_munroe/4758240536/
http://www.flickr.com/photos/ian_munroe/4758240536/ Idempotent NOT GOOD ENOUGH
! Convergent operations test state and repair if needed
None
None
! A control loop keeps the system stable and allows
for change when policy is updated
Autonomous agent Policy: The box should be closed
Convergence
None
None
None
Converging with Bash
git clone
[email protected]
:someara/ cbash.git
None
None
None
None
None
None
None
None
None
None
None
None
None
Convergence and Iteration
None
None
None
None
None
None
Does order matter?
YES
None
Promises http://www.flickr.com/photos/nazzen9009/6809694353/
• Agents are autonomous! • A promise is a signal
or message perceived by an observer.! • Promises may or may not be kept.! • Agents can observe other agents! • Agents only have local information *! • Inner workings of agents are assumed to be unknown http://markburgess.org/BookOfPromises.pdf
• Agents have intentions (possible behaviors)! • Agents can make
assessments about other agents http://markburgess.org/BookOfPromises.pdf
• Configuration Management tools embody tenants of Promise Theory intentionally
or not
Domain Specific Languages
! DSLs restrict machine instructions to convergent operations
! DSLs manage ordering
None
type subject intentions
None
type subject intentions
signal
None
type subject intention
observation
None
type subject intentions
None
type intention subject
signal
Intermission
None
Part 3
Composition
None
Recipes
resource one resource two resource three
{ testable intent
recipe[http::server]
recipe[http::server]
recipes supporting files
Types
None
interface implementation
None
intentions parameters
None
new scope intention implementation
new scope intention implementation
Artifacts
metadata
None
metadata
None
http v0.1.0 chef-server api yum v3.0.0
Delivery
• nodes request their own initial run_list
recipe[httpd::server] chef-server api run_list: http v0.1.0
recipe[httpd::server] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1
recipe[ntp::client] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1 recipe[httpd::server]
ntp v1.0.0
• Push vs Pull! • Networking considerations! • Machines down
for maintenance! • Machines that don’t exist yet
Dependencies
None
None
None
None
None
None
recipe[widgetfactory] chef-server api run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0
Integration testing
• Test that a set of agents has achieved their
combined goal
• lsof -i :80! • ps -ef | grep httpd!
• curl localhost 2>&1 > /dev/null
• Berkshelf! • Vagrant! • Kitchen.ci! • Bats! • Serverspec
Environments
• Environments constrain cookbook versions! • Environments can set data
None
None
• Environments can be used to test branches! • Environments
can be used to segregate machines! • Environments can be manipulated programatically
None
None
http v0.1.0 chef-server api http v0.2.0 openssh v1.2.3 postgresql v3.2.1
recipe[widgetfactory] run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: production
recipe[widgetfactory] run_list: http v0.2.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: staging
Part 4
Clusters http://www.flickr.com/photos/youraccount/5938852370/
Typical Cluster
loadbalancer application db-slave db-master
Production httpd 0.1.0
Production Staging httpd 0.1.0 httpd 0.1.0
Production Staging UUID httpd 0.1.0 httpd 0.1.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging httpd 0.2.0 httpd 0.2.0
Production httpd 0.2.0
An Ordering Problem
loadbalancer application
• Take a machine out of the pool! • Drain
the connections! • Modify configuration! • Insert it back into the pool
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
Orchestration
• Conductor showing signals to autonomous agents (creative policy manipulation)!
• External actor controlling sequencing (execution management)! • Application level sequencing (vector clocks, etc)
• Infrastructures are snowflakes! • Solutions are unique to applications
by nature! • Configuration Management 201
• There is no separation between ‘infrastructure’ and ‘application’! •
Distributed systems are hard! • Specialists need to work together
Devops
• Study Promise Theory! • Study distributed systems! • Develop
high quality primitives! • Be excellent to each other
Fin