A3Sec Advanced Deployment System

Managing all your Alienvault's infrastructure with Salt

Transcript

1. A3Sec Advanced Deployment System Managing all your Alienvault's Infrastructure with Salt
2. About the Author • David Gil <dgil@a3sec.com> • Developer, sysadmin, project manager • Really believes in Open Source model • Programming since he was 9 years old • Ossim developer at its early stage • Python lover :-) • Debian package maintainer (a long, long time ago) • Sci-Fi books reader and mountain bike rider
3. Summary • Distribute configuration management • Remote execution platform • Salt Stack • Configuration states • Security deployment approach • Salt architecture • Builtin modules • Targeting • Services
4. Distributed Configuration Management - Why? • Control of all the infrastructure configuration • Ensure configurations are the same on all servers (or on a type of servers) • Auto-restart services on configuration changes • Auto-Install required packages
5. Distributed Configuration Management - Why? • Control of all the infrastructure configuration • Ensure configurations are the same on all servers (or on a type of servers) • Auto-restart services on configuration changes • Auto-Install required packages Don't sysadmin, develop configuration states. Make your life easier!
6. Remote Execution Platform Why? Examples: • I want to restart apache2 on all my frameworks • I don't want to have to shell into every server just to run an apt-get upgrade command • I have {n} servers that I want to check what version of {package} is installed
7. Remote Execution Platform Why? Examples: • I want to restart apache2 on all my frameworks • I don't want to have to shell into every server just to run an apt-get upgrade command • I have {n} servers that I want to check what version of {package} is installed "ssh in a for loop is not a solution" Luka Kanies, Puppet developer
8. What is Salt? • FOSS project for distributed configuration and remote execution • Still young (2011) but active developed • Other tools: ◦ puppet (ruby): fedora, mozilla, sans ◦ chef (ruby): openstack, cloudfoundry • Salt is easier (IMHO) than chef and puppet (steeper learning curve) • Configure states by writing simple lists of items (yaml), more readable for sysadmins than vanilla python or ruby
9. Salt Keys - Just Easy • Simple Architecture ◦ Fits fine with Alienvault's Architecture • Easy Installation ◦ Squeeze debian packages, pip, bootstrap git • Easy Configuration ◦ No need to learn a new programming language • Extensible ◦ Develop Alienvault specific modules is quite easy!
10. Salt Keys - Scalable • Able to manage tens of thousands of servers • ZeroMQ based for messaging • Persistent connections / Parallel execution • MessagePack: fast and small message format (fluentd, redis, etc.)
11. Salt Keys - Secure Secure and encrypted communication transport layer: • Public Key for Master Authentication • 256 bit AES for payload high speed encryption So, VPN configuration is not needed
12. Configuration State example • You have 20 servers with exim4 as default mail transport • You want to get rid of exim4 and install postfix instead • In your 20 servers? One by one?
13. Configuration State example # apt-get install postfix (x20)
14. Configuration State example # apt-get install postfix postfix: pkg: - installed
15. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf (x20) postfix: pkg: - installed
16. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
17. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart (x20) postfix: pkg: - installed /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
18. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
19. Configuration State example # apt-get install postfix # vim /etc/postfix/master.cf # /etc/init.d/postfix restart postfix: pkg: - installed service: - running - watch: - file: /etc/postfix/master.cf /etc/postfix/master.cf: file.managed: - source: salt://postfix/master.cf
20. Security Deployment approach • Always well-known behaviors • Avoid accidental mistakes • All configuration changes stored in a single and unique place (master filesever) • Private Git repository for Knowledge Database (configuration states developed at customers) • Reusable configurations for other deployments! • Just code one time, test it and apply where you need
21. Security Deployment approach A few common and repetitive distribution tasks, for every single server on your infrastructure: • Hostname resolution • Custom plugins distribution • Remote code execution • Snort threshold and rules • Logrotate files • Rsyslog filters • Firewall rules • etc.
22. Security Deployment approach Hostname resolution # address and name management in /etc/hosts file mcafee-database: host.present: - ip: 192.168.0.42 Cron # Clean user crontab management ntpdate-debian > /var/log/syslog: cron.present: - user: root - minute: 7 - hour: 2
23. Security Deployment approach Code fixes and improvements distribution: # patch --forward Detector.py agent_hot_fix.patch /usr/share/ossim-agent/ossim_agent/Detector.py: file.patch: - source: salt://agent_hot_fix.patch - hash: md5=e138491e9d5b97023cea823fe17bac22 # Ensure root login is disabled /etc/ssh/sshd_config: file.sed: - before: 'PermitRootLogin no' - after: 'PermitRootLogin yes'
24. Salt Architecture Master (server) • Approved minion keys for communication • Send commands to minions • Store configurations, files and resources for minions Minon (client) • Connects to Master • Listens for commands • Downloads configurations from Master and apply them (update config states)
25. Salt Architecture Client/Server topology Master Minion #n Minion #2 Minion #1 states modules etc.
26. Salt Architecture Alienvault Deployment Topology AV SIEM AV LOGGER AV SENSOR #2 AV SENSOR #1 AV SENSOR #N Master Minion Minion Minion Minion Minion
27. Salt Architecture Alienvault Deployment Topology AV SIEM AV LOGGER AV SENSOR #2 AV SENSOR #1 AV SENSOR #N Master Minion Minion Minion Minion Minion
28. Salt Architecture Services Model AV SIEM AV LOGGER AV SENSOR #2 AV SENSOR #1 AV SENSOR #N Syndic Minion Minion Minion Minion Master
29. Built-in State Modules (partial list) • cmd • cron • file • git • group • host • locale • mount • mongo* • mysql* • pkg • postgres* • rabbitmq* • service • ssh* • timezone • user Custom modules can be written in Python!
30. Built-in Exec Modules (partial list) • apache • apt • archive • at • cassandra • cluster • cron • disk • file • gem • git • hosts • iptables • keyboard • ldap • locale • network All supported: salt 'node' sys.doc Custom modules can be written in Python!
31. Targeting • Match minions for cli commands and state files • Force different states for every type of server • Alienvault profiles: ◦ SIEM ◦ Logger ◦ Framework ◦ Sensor ◦ Database
32. Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins
33. Targeting base: '*': - core.tools - core.users - core.hosts - core.munin - core.nagios 'av(siem|logger)*': - alienvault.server - alienvault.database 'av(sensor|agent)*': - alienvault.agent - alienvault.plugins $ tree /srv/salt/ /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls `-- top.sls
34. Targeting • By Hostname: Minion_id '*', 'avsensor.*', 'av(siem|logger)', 'web1,web2,web3' • By system information: Grains grains['os'] == 'Debian' and 'Server' in grains ['av_profile'] • By defined groups: Node Groups nodegroups: 'sensors': avsensor*.domain.com, foo.bar 'servers': av(siem|logger).domain.com, bar.foo • Mix combination: Compound 'webserv* and G@os:Debian or E@web-dc1-srv.*'
35. Targeting - Grains Static bits of information that a minion collects about the system (inventory!) lsb_description lsb_os lsb_release manufacturer mem_total nodename num_cpus os oscodename cpu_model cpuarch domain fqdn gpus host kernel kernelrelease lsb_codename osfullname osrelease path productname server_id shell virtual etc.
36. Targeting - Grains • Using grains to define states • Jinja2 template markup apache: pkg.installed: {% if grains['os'] == 'RedHat' %} - name: httpd {% elif grains['os'] == 'Ubuntu' %} - name: apache2 {% endif %}
37. Extending - Custom grains You can write your own custom grains: # Return the host profile for Alienvault minions def av_profile(): alienvault_config = '/etc/ossim/ossim_setup.conf' if not os.path.isfile(alienvault_config): return {} config = ConfigParser.RawConfigParser() config.read(alienvault_config) profile = config.get('root', 'profile') return {'alienvault_profile': profile}
38. Extending - Custom grains And use them: # salt -G 'os:Debian' test.ping avsiem.client01.com: True avlogger.client01.com: True avsensor01.client01.com: True avsensor02.client01.com: True # salt -G 'av_profile:Server' cmd.run 'uname -r' avsiem.client01.com: 2.6.32-5-amd64 avlogger.client01.com: 2.6.31.6
39. Extending - Custom modules Build and use your custom exec modules: # salt '*.alienvault' alienvault.profile 'server01.alienvault': 'Server,Database' 'logger01.alienvault': 'Server,Database' # salt '*.alienvault' alienvault.snort_dbsize 'server01.alienvault': 457Mb 'logger01.alienvault': 0Mb # salt '*.alienvault' alienvault.numalarms 'server01.alienvault': 393 'logger01.alienvault': 0
40. Extending - Custom modules Example: Nagios configuration management # salt ‘nagios01’ nagios.add_host \ “salt-test-host” “127.0.0.1” nagios01: [nagios] Added new host: salt-test-host # salt ‘nagios01’ nagios.add_service \ “salt-test-host” “HTTP” “check_http” nagios01: [nagios] Added new service: HTTP for host salt- test-host
41. Extending - Custom states Custom states for availability monitoring, using exec modules # add new host to nagios ZZZ-salt-test-host: monitoring.add_host: - address: 127.0.0.1 # add new services to nagios HTTP: monitoring.add_service: - checkname: check_http - hostnames: - ZZZ-salt-test-host - YYY-salt-test-host
42. Dynamic Module Distribution • Build your custom grains, exec modules and states • Automatically distributed to the minions with the salt-master file server • Directories are prepended with an underscore /srv/salt/ |-- alienvault | |-- agent.sls | |-- database.sls | |-- plugins.sls | `-- server.sls |-- core | |-- hosts.sls | |-- munin.sls | |-- tools.sls | `-- users.sls |-- _grains |-- _modules |-- _states `-- top.sls
43. User interface
44. Aggregation&Management UI Build services on top of Salt architecture: • MSSPs and SOCs • Monitoring, Deployment and Management off all the infrastructure in a single dashboard • Configurations backup on demand • Patches and fixes distribution • Full Inventory • etc...
45. Aggregation&Management UI
46. Aggregation&Management UI
47. A3Sec http://www.a3sec.com @a3sec México Head Office Avda. Paseo de la Reforma, 389 Piso 10 México DF Tlf. +52 55 5980 3547 Spain Head Office C/ Aravaca, 6 Piso 2 Derecha 28040 Madrid Tlf. +34 915 330 978