Configuration Management 101

Given at FOSDEM 2014

Transcript

1. Configuration Management 101!
   FOSDEM 2014
   

   View Slide

2. Sean OMeara!
   someara@opscode.com!
   @someara
   

   View Slide

3. Sean OMeara!
   someara@opscode.com!
   @someara
   

   View Slide

4. Sean OMeara!
   someara@getchef.com!
   @someara
   

   View Slide

5. whoami
   

   View Slide

6. Part 1
   

   View Slide

7. The Dawn of Configuration
   Management
   

   View Slide

8. • What is configuration management?!
   • Strategies and techniques for managing
   configuration and its complexity!
   • The art of change management
   

   View Slide

9. Manual Configuration
   

   View Slide

10. • Intuitive!
    • How we all start out!
    • Log into machine, manipulate with fingers!
    • Make with the clicky clicky!
    • Long tradition
    

    View Slide

11. • Somehow feels the “safest"!
    • First instinct in emergencies!
    • This is an illusion!
    • Do not do this
    

    View Slide

12. • Labor intensive!
    • Error prone!
    • Difficult to reproduce!
    • Obviously unsustainable
    

    View Slide

13. Scripting
    

    View Slide

14. • setup.sh!
    • setup.pl!
    • setup.py!
    • setup.rb
    

    View Slide

15. • doit.sh!
    • doit.pl!
    • doit.py!
    • doit.rb
    

    View Slide

16. • Ad-hoc in nature!
    • Loss of history!
    • Lacks testing methodology!
    • A step in the right direction
    

    View Slide

17. File Distribution
    

    View Slide

18. • NFS!
    • SMB!
    • AFS!
    • SSHFS!
    • GlusterFS
    

    View Slide

19. • uucp!
    • rcp!
    • ftp!
    • http!
    • scp
    

    View Slide

20. •Distributed systems!
    •Shares often managed manually or with
    scripts!
    •Package repositories!
    •Pull is better than push!
    •Scp on a cron *
    

    View Slide

21. Execution Management
    

    View Slide

22. • Image management!
    • Snapshots and cloning!
    • Containers
    

    View Slide

23. • SSH on a for loop!
    • Func!
    • Commands on message queues!
    • ISConf
    

    View Slide

24. • Loss of history!
    • Image sprawl!
    • Easy to order change across nodes
    

    View Slide

25. Convergent Operators!
    (promises)
    

    View Slide

26. View Slide

27. Tools
    

    View Slide

28. • CFEngine!
    • Bcfg2!
    • Puppet!
    • Chef!
    • Salt!
    • Ansible
    

    View Slide

29. View Slide

30. Part 2
    

    View Slide

31. Policy
    http://www.flickr.com/photos/sfllaw/222795669/
    

    View Slide

32. • /etc/passwd should be mode 0644!
    • /etc/shadow should be mode 0600
    

    View Slide

33. • user ‘kermit’ should exist!
    • user ‘fonzi’ should exist!
    • group ‘muppets’ should exist!
    • group ‘muppets’ should contain kermit
    and fonzi
    

    View Slide

34. • package ‘ntpd’ should be installed!
    • ntpd should sync with our AD service!
    • service ‘ntpd’ should be running
    

    View Slide

35. • package ‘httpd’ should be installed!
    • httpd should be expose /mnt/software/java!
    • service ‘httpd’ should be running
    

    View Slide

36. • The Java JDK, version 7u45, found on an
    internally hosted web server, should be
    installed into /usr/local/jdk-7u45/
    

    View Slide

37. Polices are declarations
    about the state of things
    in a system
    

    View Slide

38. Polices are applied
    repeatedly and repair the
    system when needed
    

    View Slide

39. Policies often change
    

    View Slide

40. • package ‘widget-factory’ should be
    installed at version 1.2.3
    

    View Slide

41. • package ‘widget-factory’ should be
    installed at version 1.3.0
    

    View Slide

42. http://www.flickr.com/photos/jakepjohnson/4937767595
    Repeatability
    

    View Slide

43. Repeatable -> Idempotent ->
    Convergent
    

    View Slide

44. • Scripts are not generally repeatable
    

    View Slide

45. View Slide

46. View Slide

47. • But they can be!
    

    View Slide

48. View Slide

49. !
    Idempotent operations can be
    applied infinite times and will
    yield the same result every time
    

    View Slide

50. View Slide

51. Idempotent
    http://www.flickr.com/photos/ian_munroe/4758240536/
    

    View Slide

52. http://www.flickr.com/photos/ian_munroe/4758240536/
    Idempotent
    NOT GOOD
    ENOUGH
    

    View Slide

53. !
    Convergent operations
    test state and repair if
    needed
    

    View Slide

54. View Slide

55. View Slide

56. !
    A control loop keeps the
    system stable and allows for
    change when policy is updated
    

    View Slide

57. Autonomous agent
    Policy: The box should be closed
    

    View Slide

58. Convergence
    

    View Slide

59. View Slide

60. View Slide

61. View Slide

62. Converging with Bash
    

    View Slide

63. git clone git@github.com:someara/
    cbash.git
    

    View Slide

64. View Slide

65. View Slide

66. View Slide

67. View Slide

68. View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. View Slide

77. Convergence and Iteration
    

    View Slide

78. View Slide

79. View Slide

80. View Slide

81. View Slide

82. View Slide

83. View Slide

84. Does order matter?
    

    View Slide

85. YES
    

    View Slide

86. View Slide

87. Promises
    http://www.flickr.com/photos/nazzen9009/6809694353/
    

    View Slide

88. • Agents are autonomous!
    • A promise is a signal or message perceived by
    an observer.!
    • Promises may or may not be kept.!
    • Agents can observe other agents!
    • Agents only have local information *!
    • Inner workings of agents are assumed to be
    unknown
    http://markburgess.org/BookOfPromises.pdf
    

    View Slide

89. • Agents have intentions (possible
    behaviors)!
    • Agents can make assessments about
    other agents
    http://markburgess.org/BookOfPromises.pdf
    

    View Slide

90. • Configuration Management tools embody
    tenants of Promise Theory intentionally or
    not
    

    View Slide

91. Domain Specific Languages
    

    View Slide

92. !
    DSLs restrict machine
    instructions to
    convergent operations
    

    View Slide

93. !
    DSLs manage
    ordering
    

    View Slide

94. View Slide

95. type subject intentions
    

    View Slide

96. View Slide

97. type
    subject intentions
    

    View Slide

98. signal
    

    View Slide

99. View Slide

100. type subject
     intention
     

     View Slide

101. observation
     

     View Slide

102. View Slide

103. type
     subject
     intentions
     

     View Slide

104. View Slide

105. type
     intention
     subject
     

     View Slide

106. signal
     

     View Slide

107. Intermission
     

     View Slide

108. View Slide

109. Part 3
     

     View Slide

110. Composition
     

     View Slide

111. View Slide

112. Recipes
     

     View Slide

113. resource one
     resource two
     resource three
     

     View Slide

114. {
     testable
     
     intent
     

     View Slide

115. recipe[http::server]
     

     View Slide

116. recipe[http::server]
     

     View Slide

117. recipes
     supporting files
     

     View Slide

118. Types
     

     View Slide

119. View Slide

120. interface
     implementation
     

     View Slide

121. View Slide

122. intentions
     parameters
     

     View Slide

123. View Slide

124. new scope
     intention implementation
     

     View Slide

125. new scope
     intention implementation
     

     View Slide

126. Artifacts
     

     View Slide

127. metadata
     

     View Slide

128. View Slide

129. metadata
     

     View Slide

130. View Slide

131. http v0.1.0
     chef-server api
     yum v3.0.0
     

     View Slide

132. Delivery
     

     View Slide

133. • nodes request their own initial run_list
     

     View Slide

134. recipe[httpd::server]
     chef-server api
     run_list:
     http v0.1.0
     

     View Slide

135. recipe[httpd::server]
     chef-server api
     run_list:
     http v0.1.0
     recipe[openssh::server]
     openssh v3.2.1
     

     View Slide

136. recipe[ntp::client]
     chef-server api run_list:
     http v0.1.0
     recipe[openssh::server]
     openssh v3.2.1
     recipe[httpd::server]
     ntp v1.0.0
     

     View Slide

137. • Push vs Pull!
     • Networking considerations!
     • Machines down for maintenance!
     • Machines that don’t exist yet
     

     View Slide

138. Dependencies
     

     View Slide

139. View Slide

140. View Slide

141. View Slide

142. View Slide

143. View Slide

144. View Slide

145. recipe[widgetfactory]
     chef-server api
     run_list:
     http v0.1.0
     yum v3.0.0
     widgetfactory v1.0.0
     

     View Slide

146. Integration testing
     

     View Slide

147. • Test that a set of agents has achieved
     their combined goal
     

     View Slide

148. • lsof -i :80!
     • ps -ef | grep httpd!
     • curl localhost 2>&1 > /dev/null
     

     View Slide

149. • Berkshelf!
     • Vagrant!
     • Kitchen.ci!
     • Bats!
     • Serverspec
     

     View Slide

150. Environments
     

     View Slide

151. • Environments constrain cookbook
     versions!
     • Environments can set data
     

     View Slide

152. View Slide

153. View Slide

154. • Environments can be used to test
     branches!
     • Environments can be used to segregate
     machines!
     • Environments can be manipulated
     programatically
     

     View Slide

155. View Slide

156. View Slide

157. http v0.1.0
     chef-server api
     http v0.2.0
     openssh v1.2.3
     postgresql v3.2.1
     

     View Slide

158. recipe[widgetfactory]
     run_list:
     http v0.1.0
     yum v3.0.0
     widgetfactory v1.0.0
     chef_environment: production
     

     View Slide

159. recipe[widgetfactory]
     run_list:
     http v0.2.0
     yum v3.0.0
     widgetfactory v1.0.0
     chef_environment: staging
     

     View Slide

160. Part 4
     

     View Slide

161. Clusters
     http://www.flickr.com/photos/youraccount/5938852370/
     

     View Slide

162. Typical Cluster
     

     View Slide

163. loadbalancer
     application
     db-slave
     db-master
     

     View Slide

164. Production
     httpd 0.1.0
     

     View Slide

165. Production Staging
     httpd 0.1.0 httpd 0.1.0
     

     View Slide

166. Production Staging UUID
     httpd 0.1.0 httpd 0.1.0 httpd 0.2.0
     

     View Slide

167. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

168. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

169. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

170. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

171. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

172. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

173. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

174. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

175. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

176. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

177. Production Staging UUID
     httpd 0.1.0 httpd 0.2.0 httpd 0.2.0
     

     View Slide

178. Production Staging
     httpd 0.2.0 httpd 0.2.0
     

     View Slide

179. Production
     httpd 0.2.0
     

     View Slide

180. An Ordering Problem
     

     View Slide

181. loadbalancer
     application
     

     View Slide

182. • Take a machine out of the pool!
     • Drain the connections!
     • Modify configuration!
     • Insert it back into the pool
     

     View Slide

183. loadbalancer
     application
     

     View Slide

184. loadbalancer
     application
     

     View Slide

185. loadbalancer
     application
     

     View Slide

186. loadbalancer
     application
     

     View Slide

187. loadbalancer
     application
     

     View Slide

188. loadbalancer
     application
     

     View Slide

189. loadbalancer
     application
     

     View Slide

190. loadbalancer
     application
     

     View Slide

191. loadbalancer
     application
     

     View Slide

192. loadbalancer
     application
     

     View Slide

193. Orchestration
     

     View Slide

194. • Conductor showing signals to
     autonomous agents (creative policy
     manipulation)!
     • External actor controlling sequencing
     (execution management)!
     • Application level sequencing (vector
     clocks, etc)
     

     View Slide

195. • Infrastructures are snowflakes!
     • Solutions are unique to applications by
     nature!
     • Configuration Management 201
     

     View Slide

196. • There is no separation between
     ‘infrastructure’ and ‘application’!
     • Distributed systems are hard!
     • Specialists need to work together
     

     View Slide

197. Devops
     

     View Slide

198. • Study Promise Theory!
     • Study distributed systems!
     • Develop high quality primitives!
     • Be excellent to each other
     

     View Slide

199. Fin
     

     View Slide