Make it SOLID. Software Architecture for System Administrators

Gave this talk at OSDC 2014 in Berlin, Germany on April 9th, 2014
Follow me on Twitter: @CodeStars or Sönke @s0enke

Starting with Chef or Puppet as a System Administrator will lead you to a problem where you are not sure what's the best solution of a problem in terms of software architecture. We will give you a brief overview of general well known and battle tested software patterns, which also applies to infrastructure management code. In addition, we'll also show Antipatterns, and best practices.

Transcript

1. Ole Michaelis & Sönke Ruempler | Jimdo Make it SOLID Software Architecture for System Administrators
2. OSDC 2014 Ole Michaelis “Open Source Rockstar” @CodeStars github.com/nesQuick codestars.eu
3. OSDC 2014 Sönke Ruempler “Chief Trolling Officer” @s0enke github.com/s0enke ruempler.eu
4. OSDC 2014 Jimdo
5. OSDC 2014 because infrastructure development is neither a Jenga game! S.O.L.I.D
6. OSDC 2014 a little poll! http://images.fineartamerica.com/images-medium-large/live-long-mr-spock-tobias-woelki.jpg
7. OSDC 2014
8. OSDC 2014 S O L I D
9. OSDC 2014 A dev/ops story Ole as Junior Admin Sönke as Dr. Software
10. OSDC 2014 the basics
11. OSDC 2014 configuration management Idempotent Declarative Convergent Abstract
12. OSDC 2014 software architecture VS. 36 years >2000 years
13. OSDC 2014
14. OSDC 2014 node web-1 { $role = 'web-app-a' $has_apache = true; include php5 } node web-2 { $role = 'web-app-b' $has_fcgi=true include php5 } # has both enabled, but needs a special config node web-3 { $role = 'web-app-c' $has_fcgi=true $has_apache=true include php5 } Web App 1 Web App 2 Web App 3
15. OSDC 2014 class php5 { if ($::has_fcgi) package { 'php5-cli' : ensure => installed } } if ($::has_apache) { package { 'php5-apache' : ensure => installed } } if ($::role == 'web-app-c') package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c' } } } Global variable! Out of Context! No abstraction
16. OSDC 2014 class php5( $has_fcgi = false, $has_apache = false, $role = undef ) { if ($has_fcgi) package { 'php5-cli' : ensure => installed } } if ($has_apache) { package { 'php5-apache' : ensure => installed } } if ($role == 'web-app-c') { package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c' , require => Package[php-fcgi] } }
17. OSDC 2014 node 'web-1' { $role = 'web-app-a' $has_apache = true class { 'php5' : has_apache => true, role => 'web-app-a' } } node 'web-2' { $role = 'web-app-b' $has_fcgi = true class { 'php5' : has_fcgi => true, role => 'web-app-b' } } Injection! Injection!
18. OSDC 2014 Would you solder a lamp directly to the electrical wiring in a wall? Dependency Inversion
19. OSDC 2014
20. OSDC 2014 class php5( $has_fcgi = false, $has_apache = false, $role = undef ) { if ($has_fcgi) package { 'php5-cli' : ensure => installed } } if ($has_apache) { package { 'php5-apache' : ensure => installed } } if ($role == 'web-app-c') { package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c' require => Package[php-fcgi] } } }
21. OSDC 2014 Stuff tends to get big Beware of god classes!
22. OSDC 2014 if ($lighttpd) { package { 'php5-cgi': } configdir { '/etc/php5/cgi': require => Package['php5-cgi'] } configdir { '/etc/php5/cgi/conf.d': require => Package['php5-cgi'] } if ($testserver) { configfile { '/etc/php5/cgi/php.ini': sourcename => 'php5/cgi/php.ini-testserver', require => Package['php5-cgi'] } } else { if ($cms ) { configfile { '/etc/php5/cgi/php.ini': sourcename => 'php5/cgi/php.ini-cms', require => Package['php5-cgi'] } } else { if ($mgmt ) { configfile { '/etc/php5/cgi/php.ini': sourcename => 'php5/cgi/php.ini-mgmt', require => Package['php5-cgi'] } } else { if ($lc ) { configfile { '/etc/php5/cgi/php.ini': sourcename => 'php5/cgi/php.ini-lc. jimdo.com', require => Package['php5-cgi'] } } else { configfile { '/etc/php5/cgi/php.ini': sourcename => 'php5/cgi/php.ini', require => Package['php5-cgi'] } } } } } configfile { '/etc/php5/cgi/conf.d/pdo.ini': sourcename => 'php5/cgi/conf.d/pdo.ini', require => Package['php5-cgi'] } }
23. OSDC 2014
24. OSDC 2014 ?
25. OSDC 2014
26. OSDC 2014 rspec-puppet test-kitchen
27. OSDC 2014
28. OSDC 2014 describe :node => 'web-1' do it { should contain_package ('php5-fcgi') } end describe :node => 'web-2' do it { should contain_package ('php5-apache2' ) } end describe :node => 'web-3' do it { should contain_package ('php5-fcgi') } it { should contain_package ('php5-apache2' ) } end Test php5 class: Install Package ?
29. OSDC 2014 describe :class => 'php5::fcgi' do it { should contain_package('php5-cli') } end describe :class => 'php5::apache' do it { should contain_package('php5-apache2') } end class php5::fgci { package { 'php5-cgi' : ensure => installed } } class php5::apache { package { 'php5-apache' : ensure => installed } }
30. OSDC 2014 Just because you can, doesn’t mean you should! Single Responsibility
31. OSDC 2014 describe :node => 'web-1' do it { should include_class('php5::fcgi') } it { should contain_package('php5-fcgi') } end describe :node => 'web-2' do it { should include_class('php5::apache') } it { should contain_package('php5-apache2') } end node web-1 { include php5::apache } node web-2 { include php5::fcgi } Tests Code
32. OSDC 2014
33. OSDC 2014 “programs […] are changed by adding new code, rather than by changing existing code” Open Close Principle
34. OSDC 2014
35. OSDC 2014 new requirements... now web app 1 needs a second node node web-1 { $role = 'web-app-a' include php5::apache } node web-1a { $role = 'web-app-a' include php5::apache }
36. OSDC 2014
37. OSDC 2014
38. OSDC 2014 class role::web-app-a { include php5::apache } node web-1 { include role::web-app- a } node web-1a { include role::web-app- a } Our new role! And nodes just include it!
39. OSDC 2014 >
40. OSDC 2014
41. OSDC 2014 ? class php5::fcgi($role = undef) { if ($role == 'web-app-c') { package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c', require => Package[php-fcgi] } } }
42. OSDC 2014 Don’t call us, we call you! Hollywood Principle
43. OSDC 2014 class role::web-app-c::special-php-stuff { package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c' } } 1. Split out the special case into its own class.
44. OSDC 2014 class php5::fcgi($include_after_package_install = undef) { package { 'php5-fcgi' : ensure => installed } if ($include_after_package_install) include $include_after_package_install Package['php5-fcgi'] -> Class[$include_after_package_install] } } 2. Make the special case pluggable and reusable. 2a. And name it abstract.
45. OSDC 2014 class role::web-app-c { include php5::apache class { 'php5::fcgi' : include_after_package_install => 'profile::web-app-c::special-php-stuff' } } 3. Pass the special case as dependency in our web-app-c.
46. OSDC 2014
47. OSDC 2014 class profile::web-app-c::special-php-stuff { package { 'php5-web-app-c-special-module' } file { '/etc/php5/php.ini' : source => 'puppet:///modules/php5/php.ini-web-app-c' } } ?
48. OSDC 2014 define php5::specialconfig( $ensure = present, $sapi, $module, $key, $value, $section = undef, $path = '/etc/php5/%s/conf.d/%s.ini' ) { ini_setting { $title: ensure => $ensure, path => sprintf($path, $sapi, $module), section => $section, setting => $key, value => $value, require => Package["php5-${sapi}"] } }
49. OSDC 2014
50. OSDC 2014 ?
51. OSDC 2014 “puppet separate code and data” I’m feeling lucky
52. OSDC 2014 # /etc/puppet/hieradata/web1.yaml --- classes: - profile::web-app-a # /etc/puppet/hieradata/web1a.yaml --- classes: - profile::web-app-a # /etc/puppet/hieradata/web2.yaml --- classes: - profile::web-app-c # /etc/puppet/hieradata/web3.yaml --- classes: - profile::web-app-c puppet-hiera example in YAML
53. OSDC 2014 node default { hiera_include('classes') } node web-1 { include role::web-app-a } node web-1a { include role::web-app-a } node web-2 { include role::web-app-b } node web-3 { include role::web-app-c }
54. OSDC 2014 hiera AWS/Cloudformation LDAP DNS your selfwritten stuff
55. OSDC 2014
56. OSDC 2014
57. OSDC 2014
58. OSDC 2014 What about I and L in SOLID?
59. OSDC 2014 You want me to plug this in, where? Interface Segregation
60. OSDC 2014 If it looks like a duck, quacks like a duck, but need batteries - you probably have the wrong abstraction! Liskov Substitution
61. OSDC 2014 S O L I D
62. OSDC 2014
63. OSDC 2014 Structure your code! Node Profile Profiles Role Profile Modules Resources
64. OSDC 2014 Puppet forge puppet module registry
65. OSDC 2014 Librarian Puppet r10k Berkshelf (chef)
66. OSDC 2014 stdmod https://github.com/stdmod
67. OSDC 2014 sandboxed development https://github.com/mlafeldt/skeleton-cookbook https://github.com/Jimdo/puppet-skeleton
68. OSDC 2014 server spec http://serverspec.org/ beaker (puppet) https://github.com/puppetlabs/beaker
69. OSDC 2014 Thank you!
70. OSDC 2014 @CodeStars ole@jimdo.com @s0enke soenke@jimdo.com Questions?
71. OSDC 2014 *I ’m just a blank slide*
72. OSDC 2014 Sources • http://www.slideshare.net/PuppetLabs/garethrushgrove-puppetconf • http://www.slideshare.net/PuppetLabs/alessandro-franceschi-new • https://github.com/jedi4ever/stop-the-fork • http://lostechies.com/derickbailey/2009/02/11/solid-development-principles-in-motivational- pictures/ • https://speakerdeck.com/jfryman/refactoring-puppet • http://www.clker.com/cliparts/e/2/a/d/1206574733930851359Ryan_Taylor_Green_Tick.svg.med. png • http://www.craigdunn.org/2012/05/239/ • http://cdn.slashgear.com/wp-content/uploads/2012/10/google-datacenter-tech-13.jpg • http://deviq.com/Media/Default/Article/Dont-Call-Us-Well-Call-You-Jun-2013.png • http://www.timeshighereducation.co.uk/Pictures/web/g/q/g/copy_paste_keyboard_button_450.jpg • http://sanremo.com.au/wp-content/uploads/2013/05/lasagna.jpg • http://4.bp.blogspot. com/_9kQQgQD35rY/SaV5p8YBGhI/AAAAAAAAAkg/HOvlhIo7yGI/s400/06_Red_Green_Refactor. JPG • http://www.thelolshop.com/wp-content/uploads/2012/11/20121126-101937.jpg •