The classification problems: challenges and solutions

Df0cf054b01391a0f3113cc8e1048662?s=47 Marco Marongiu
February 01, 2014

The classification problems: challenges and solutions

I have 20 minutes to try to give you an idea of why classification in configuration management is critical, why it is difficult, and how you can make it sane.

Df0cf054b01391a0f3113cc8e1048662?s=128

Marco Marongiu

February 01, 2014
Tweet

Transcript

  1. The classification problem: challenges and solutions Marco Marongiu (@brontolinux)

  2. Acknowledgements All the images and logos used in this presentation

    are shown for didactic purposes only, in the belief that it constitutes a fair use of these resources. I don't claim any right on the images and products shown in this presentation. If you are the owner of any material used in this presentation and you believe that I am abusing it, please let me know and I'll promptly remove it from all my sources.
  3. Agenda ➔ Why classification is critical ➔ Why classification is

    difficult ➔ How we can make it sane ➔ Final take-aways
  4. Part 1: the problem

  5. !!! !!! !!! !!! !!! !!! !!!

  6. Configuration management «Configuration Management is the process of identifying and

    defining the items in the system, controlling the change of these items throughout their life-cycle, recording and reporting the status of items and change requests, and verifying the completeness and correctness of items» -- IEEE Glossary of Software Engineering Terminology (Standard 729-1983)
  7. Configuration management «Configuration Management is the process of identifying and

    defining the items in the system, controlling the change of these items throughout their life-cycle, recording and reporting the status of items and change requests, and verifying the completeness and correctness of items» -- IEEE Glossary of Software Engineering Terminology (Standard 729-1983)
  8. In configuration management you don't apply a configuration to a

    system, you rather apply it to classes of systems!
  9. Part 2: challenges

  10. Exceptions are the rule

  11. Generic settings, the same for all systems Amsterdam DNS server

    NTP server Syslog server SSH configs Oslo Other random exceptions Seattle Iceland
  12. Internal classification doesn't scale definitions explosion difficult reporting «The business

    should define what systems belong in which classes. The Cfengine administrator should build policy. Once the Cfengine administrator is left to manually defining classes within policy, you become a bottleneck» -- M.Svoboda, LinkedIn human bottleneck
  13. Part 3: Solutions

  14. Infrastructure complexity low complexity high complexity Technical skills well versed

    non technical Basic interface & basic backend Basic interface & sophisticated backend Sophisticated interface & sophisticated backend Sophisticated interface & basic backend
  15. The LinkedIn solution M.Svoboda - LinkedIn "Leveraging In-Memory Key Value

    Stores for Large-Scale Operations"
  16. our implementation: hENC power & simplicity config info in plain

    text files module protocol a simple Perl script
  17. None
  18. CFEngine's module protocol +activated_class -cancelled_class =my_var=my value =@my_list={ 'list','of','4','values'} =my_array[element]=value

    @my_array[list]={ 'list','of','4','values'}
  19. #!/bin/sh /bin/egrep -h ^[=@+-] $* 2> /dev/null

  20. #!/usr/bin/perl use strict ; use warnings ; my %class ;

    # classes container my %variable ; # variables container # Silence errors (e.g.: missing files) close STDERR ; while (my $line = <>) { chomp $line ; my ($setting,$id) = ( $line =~ m{^\s*([=\@+-])(.+)\s*$} ) ; # line didn't match the module protocol next if not defined $setting ; # add a class if ($setting eq '+') { # $id is a class name, or should be. $class{$id} = 1 ; } # undefine a class if ($setting eq '-') { # $id is a class name, or should be. $class{$id} = -1 ; } # define a variable/list if ($setting eq '=' or $setting eq '@') { # $id is "variable = something", or should be my ($varname) = ( $id =~ m{^(.+?)=} ) ; $variable{$varname} = $line ; } # discard the rest } # print out classes foreach my $classname (keys %class) { print "+$classname\n" if $class{$classname} > 0 ; print "-$classname\n" if $class{$classname} < 0 ; } # print variable/list assignments, the last one wins foreach my $assignment (values %variable) { print "$assignment\n" ; }
  21. read general defaults read location defaults read environment defaults read

    node settings
  22. oslo_public:: "enc_subdir" policy => "overridable", string => "$(enc_basedir)/pub" ; on_private_net_only::

    "enc_subdir" policy => "overridable", string => "$(enc_basedir)/priv" ; oslo:: "henclist" policy => "overridable", slist => { "$(enc_basedir)/_default_", "$(enc_basedir)/_oslo_", "$(enc_subdir)/_oslo_", "$(enc_subdir)/$(sys.domain)/$(sys.fqhost)", } ; }environment- dependent settings } general defaults } location defaults } environ. defaults
  23. methods: "ENC" comment => "External node classification", usebundle => henc("site.henclist")

    ; vars: "motd_file" string => "$(henc.motd_file)", policy => "overridable" ;
  24. final take-aways smart classification is crucial... ...but doesn't need to

    be complicated... ...unless your infrastructure is. sometimes, even plain text is good enough!
  25. None
  26. comments? @brontolinux mmarongiu@tiscali.it http://syslog.me http://no.linkedin.com/in/marcomarongiu/

  27. THANK you