Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Configuration Management 101
Search
someara
February 04, 2014
Programming
3
620
Configuration Management 101
Given at FOSDEM 2014
someara
February 04, 2014
Tweet
Share
More Decks by someara
See All by someara
Docker Docker Docker Chef
someara
0
330
Hand Crafted Artisinal Chef Resources
someara
0
430
Configuration Management Camp 2015
someara
1
410
Cookbook Reusability
someara
0
390
TDI with ChefDK 0.0.1
someara
2
470
Configuration Management 101 @ Scale12x
someara
2
720
Introduction to Chef - Scale 10x
someara
2
370
Introduction to Chef - NYLUG Jan 2012
someara
2
250
Introduction to Chef - LISA11
someara
10
6k
Other Decks in Programming
See All in Programming
一休.comレストランのRustバックエンド開発の様子
kymmt90
13
7.9k
sbt-assemblyにハマってDB接続できず時間が溶けた話
wakye5815
1
660
CSC308B Lecture 12
javiergs
PRO
0
110
Deno に Web 標準 API を実装する / Implementing Web Standard API to Deno
petamoriken
0
310
20240301_cocone_EMゆるミートアップvol6_LT資料
cocone
0
260
mandaRa: R言語ユーザのための新しい知識共有の場 / mandara_tokyor111
s_uryu
2
410
OpCode目線で眺める PHPコードのカバレッジ
o0h
PRO
2
470
孤独のCTOグルメという やや奇抜な企画をやった目的と効果
shoheimitani
3
830
Learning Ruby
okuramasafumi
5
370
Cloudflare Workersの環境を再現することについて
yusukebe
5
730
Kotlinを用いたDSL的な設計手法と使用上の注意
kohii00
2
490
Learning PHP and Static Analysis with PHP Parser
inouehi
1
220
Featured
See All Featured
Designing for humans not robots
tammielis
247
25k
Happy Clients
brianwarren
91
6.3k
Fashionably flexible responsive web design (full day workshop)
malarkey
397
65k
Building Better People: How to give real-time feedback that sticks.
wjessup
350
18k
The Language of Interfaces
destraynor
150
22k
The Brand Is Dead. Long Live the Brand.
mthomps
48
20k
4 Signs Your Business is Dying
shpigford
174
21k
Typedesign – Prime Four
hannesfritz
36
2k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
657
120k
Why You Should Never Use an ORM
jnunemaker
PRO
50
8.5k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
Building a Scalable Design System with Sketch
lauravandoore
455
32k
Transcript
Configuration Management 101! FOSDEM 2014
Sean OMeara!
[email protected]
! @someara
Sean OMeara!
[email protected]
! @someara
Sean OMeara!
[email protected]
! @someara
whoami
Part 1
The Dawn of Configuration Management
• What is configuration management?! • Strategies and techniques for
managing configuration and its complexity! • The art of change management
Manual Configuration
• Intuitive! • How we all start out! • Log
into machine, manipulate with fingers! • Make with the clicky clicky! • Long tradition
• Somehow feels the “safest"! • First instinct in emergencies!
• This is an illusion! • Do not do this
• Labor intensive! • Error prone! • Difficult to reproduce!
• Obviously unsustainable
Scripting
• setup.sh! • setup.pl! • setup.py! • setup.rb
• doit.sh! • doit.pl! • doit.py! • doit.rb
• Ad-hoc in nature! • Loss of history! • Lacks
testing methodology! • A step in the right direction
File Distribution
• NFS! • SMB! • AFS! • SSHFS! • GlusterFS
• uucp! • rcp! • ftp! • http! • scp
•Distributed systems! •Shares often managed manually or with scripts! •Package
repositories! •Pull is better than push! •Scp on a cron *
Execution Management
• Image management! • Snapshots and cloning! • Containers
• SSH on a for loop! • Func! • Commands
on message queues! • ISConf
• Loss of history! • Image sprawl! • Easy to
order change across nodes
Convergent Operators! (promises)
None
Tools
• CFEngine! • Bcfg2! • Puppet! • Chef! • Salt!
• Ansible
None
Part 2
Policy http://www.flickr.com/photos/sfllaw/222795669/
• /etc/passwd should be mode 0644! • /etc/shadow should be
mode 0600
• user ‘kermit’ should exist! • user ‘fonzi’ should exist!
• group ‘muppets’ should exist! • group ‘muppets’ should contain kermit and fonzi
• package ‘ntpd’ should be installed! • ntpd should sync
with our AD service! • service ‘ntpd’ should be running
• package ‘httpd’ should be installed! • httpd should be
expose /mnt/software/java! • service ‘httpd’ should be running
• The Java JDK, version 7u45, found on an internally
hosted web server, should be installed into /usr/local/jdk-7u45/
Polices are declarations about the state of things in a
system
Polices are applied repeatedly and repair the system when needed
Policies often change
• package ‘widget-factory’ should be installed at version 1.2.3
• package ‘widget-factory’ should be installed at version 1.3.0
http://www.flickr.com/photos/jakepjohnson/4937767595 Repeatability
Repeatable -> Idempotent -> Convergent
• Scripts are not generally repeatable
None
None
• But they can be!
None
! Idempotent operations can be applied infinite times and will
yield the same result every time
None
Idempotent http://www.flickr.com/photos/ian_munroe/4758240536/
http://www.flickr.com/photos/ian_munroe/4758240536/ Idempotent NOT GOOD ENOUGH
! Convergent operations test state and repair if needed
None
None
! A control loop keeps the system stable and allows
for change when policy is updated
Autonomous agent Policy: The box should be closed
Convergence
None
None
None
Converging with Bash
git clone
[email protected]
:someara/ cbash.git
None
None
None
None
None
None
None
None
None
None
None
None
None
Convergence and Iteration
None
None
None
None
None
None
Does order matter?
YES
None
Promises http://www.flickr.com/photos/nazzen9009/6809694353/
• Agents are autonomous! • A promise is a signal
or message perceived by an observer.! • Promises may or may not be kept.! • Agents can observe other agents! • Agents only have local information *! • Inner workings of agents are assumed to be unknown http://markburgess.org/BookOfPromises.pdf
• Agents have intentions (possible behaviors)! • Agents can make
assessments about other agents http://markburgess.org/BookOfPromises.pdf
• Configuration Management tools embody tenants of Promise Theory intentionally
or not
Domain Specific Languages
! DSLs restrict machine instructions to convergent operations
! DSLs manage ordering
None
type subject intentions
None
type subject intentions
signal
None
type subject intention
observation
None
type subject intentions
None
type intention subject
signal
Intermission
None
Part 3
Composition
None
Recipes
resource one resource two resource three
{ testable intent
recipe[http::server]
recipe[http::server]
recipes supporting files
Types
None
interface implementation
None
intentions parameters
None
new scope intention implementation
new scope intention implementation
Artifacts
metadata
None
metadata
None
http v0.1.0 chef-server api yum v3.0.0
Delivery
• nodes request their own initial run_list
recipe[httpd::server] chef-server api run_list: http v0.1.0
recipe[httpd::server] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1
recipe[ntp::client] chef-server api run_list: http v0.1.0 recipe[openssh::server] openssh v3.2.1 recipe[httpd::server]
ntp v1.0.0
• Push vs Pull! • Networking considerations! • Machines down
for maintenance! • Machines that don’t exist yet
Dependencies
None
None
None
None
None
None
recipe[widgetfactory] chef-server api run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0
Integration testing
• Test that a set of agents has achieved their
combined goal
• lsof -i :80! • ps -ef | grep httpd!
• curl localhost 2>&1 > /dev/null
• Berkshelf! • Vagrant! • Kitchen.ci! • Bats! • Serverspec
Environments
• Environments constrain cookbook versions! • Environments can set data
None
None
• Environments can be used to test branches! • Environments
can be used to segregate machines! • Environments can be manipulated programatically
None
None
http v0.1.0 chef-server api http v0.2.0 openssh v1.2.3 postgresql v3.2.1
recipe[widgetfactory] run_list: http v0.1.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: production
recipe[widgetfactory] run_list: http v0.2.0 yum v3.0.0 widgetfactory v1.0.0 chef_environment: staging
Part 4
Clusters http://www.flickr.com/photos/youraccount/5938852370/
Typical Cluster
loadbalancer application db-slave db-master
Production httpd 0.1.0
Production Staging httpd 0.1.0 httpd 0.1.0
Production Staging UUID httpd 0.1.0 httpd 0.1.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging UUID httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
Production Staging httpd 0.2.0 httpd 0.2.0
Production httpd 0.2.0
An Ordering Problem
loadbalancer application
• Take a machine out of the pool! • Drain
the connections! • Modify configuration! • Insert it back into the pool
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
loadbalancer application
Orchestration
• Conductor showing signals to autonomous agents (creative policy manipulation)!
• External actor controlling sequencing (execution management)! • Application level sequencing (vector clocks, etc)
• Infrastructures are snowflakes! • Solutions are unique to applications
by nature! • Configuration Management 201
• There is no separation between ‘infrastructure’ and ‘application’! •
Distributed systems are hard! • Specialists need to work together
Devops
• Study Promise Theory! • Study distributed systems! • Develop
high quality primitives! • Be excellent to each other
Fin