Extending operating systems support of Foreman

slides to my talk at cfgmgmtcamp 2016

Transcript

1. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Extending operating systems support of Foreman Michael Moll <mmoll@mmoll.at> 2nd February 2016 1 / 26
2. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Foreman OS support Supported platforms in the quickstart documentation: CentOS, Scientific Linux or Oracle Linux 6 or 7, x86_64 Debian 8 (Jessie), i386/amd64/armhf Debian 7 (Wheezy), i386/amd64 Fedora 19, x86_64 Red Hat Enterprise Linux 6 or 7, x86_64 Ubuntu 14.04 (Trusty), i386/amd64/armhf Ubuntu 12.04 (Precise), i386/amd64 Wait, there’s more? Other operating systems will need to use alternative installation methods (see the manual). 2 / 26
3. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Foreman Provisioning Support 3 / 26
4. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Foreman Building Blocks Foreman provisioning support (RoR and ERB templates) Foreman core (Ruby On Rails + a ton of gems) Foreman proxy (Ruby + very few gems) Foreman installer (Ruby + Puppet + a few gems) Puppet modules (Puppet) Hammer (Ruby + a few gems) Plugins Foreman Packaging (RPM and DEB) for peeking 4 / 26
5. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Why FreeBSD? ZFS, DTrace (thanks SUN / Illumos / OpenZFS!) pf (thanks OpenBSD!) Storage Stack (EMC/Isilon, Spectra Logic, FreeNAS) Network Stack (Netflix, Juniper, pfSense, opnSense) Jails (Solaris Zones and Linux LXC were inspired by it) Linux ABI compatibility layer (Linuxlator) Jails + Linuxlator = Docker on FreeBSD ports collection with rubygems framework ~25000 binary packages available with pkgng 5 / 26
6. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! FreeBSD and Foreman 2013-11-28 FreeBSD provisioning support merged 2014-01-29 foreman-proxy packaged 2014-03-13 hammer packaged 2014-09-27 smart_proxy plugins packaged 2015-09-19 all Puppet modules support FreeBSD 2016-01-15 foreman-installer packaged Only FreeBSD? Having all these changes is making it easier in the future to support other operating systems or Linux distributions. Having Windows support in theforeman-puppet also made it easier to add FreeBSD support. 6 / 26
7. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Ruby Gems What is a Ruby Gem? RubyGems is a package manager for the Ruby programming language that provides a standard format for distributing Ruby programs and libraries (in a self-contained format called a ”gem”). A ”package” for Ruby bundler is used to define sets of Gems Having gems packaged to OS packages seems logical manual transformation (former way to RPMs) Debian: gem2deb Gems framework e.g. in FreeBSD ports 7 / 26
8. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Ruby Gems in FreeBSD PORTNAME= librarian-puppet PORTVERSION= 2.2.1 CATEGORIES= sysutils rubygems MASTER_SITES= RG MAINTAINER= robak@FreeBSD.org COMMENT= Tool for management and bundling Puppet modules LICENSE= MIT LICENSE_FILE= ${WRKSRC}/LICENSE RUN_DEPENDS= rubygem-librarianp>=0.6.3:${PORTSDIR}/devel/rubygem-librarianp \ rubygem-puppet_forge1>=1.0:${PORTSDIR}/sysutils/rubygem-puppet_forge1 \ rubygem-rsync>0:${PORTSDIR}/net/rubygem-rsync USE_RUBY= yes USE_RUBYGEMS= yes RUBYGEM_AUTOPLIST= yes 8 / 26
9. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Hammer Start with something easy. It’s Hammer time! Package all gem dependencies Package the hammer_cli gem Copy the configuration file to the correct place You’re done! Repeat for hammer plugins (e.g. hammer_cli_foreman) 9 / 26
10. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! hammer_cli for FreeBSD PORTNAME= hammer_cli PORTVERSION= 0.5.1 CATEGORIES= sysutils ruby MASTER_SITES= RG MAINTAINER= ruby@FreeBSD.org COMMENT= Provides CLI interfaces for Ruby apps LICENSE= GPLv3 LICENSE_FILE= ${WRKSRC}/LICENSE RUN_DEPENDS= rubygem-apipie-bindings>=0.0.14:${PORTSDIR}/devel/rubygem-apipie-bindings \ rubygem-awesome_print>=1.1.0:${PORTSDIR}/devel/rubygem-awesome_print \ rubygem-clamp>=1.0.0:${PORTSDIR}/devel/rubygem-clamp \ rubygem-fast_gettext>=0.8.1:${PORTSDIR}/devel/rubygem-fast_gettext \ rubygem-highline>=1.6.20:${PORTSDIR}/devel/rubygem-highline \ rubygem-locale>=2.0.6:${PORTSDIR}/devel/rubygem-locale \ rubygem-logging>=1.8.1:${PORTSDIR}/devel/rubygem-logging \ rubygem-table_print>=1.5.1:${PORTSDIR}/devel/rubygem-table_print USE_RUBY= yes USE_RUBYGEMS= yes RUBYGEM_AUTOPLIST= yes post-install: ${MKDIR} ${STAGEDIR}${PREFIX}/etc/hammer/cli.modules.d ${INSTALL_DATA} ${WRKSRC}/config/cli_config.template.yml \ ${STAGEDIR}${PREFIX}/etc/hammer/cli_config.yml.sample 10 / 26
11. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! hammer_cli_foreman for FreeBSD PORTNAME= hammer_cli_foreman PORTVERSION= 0.5.1 CATEGORIES= sysutils ruby MASTER_SITES= RG MAINTAINER= ruby@FreeBSD.org COMMENT= Foreman commands for Hammer CLI LICENSE= GPLv3 RUN_DEPENDS= rubygem-hammer_cli>=0.5.0:${PORTSDIR}/sysutils/rubygem-hammer_cli USE_RUBY= yes USE_RUBYGEMS= yes RUBYGEM_AUTOPLIST= yes post-install: ${MKDIR} ${STAGEDIR}${PREFIX}/etc/hammer/cli.modules.d ${INSTALL_DATA} ${WRKSRC}/config/foreman.yml \ ${STAGEDIR}${PREFIX}/etc/hammer/cli.modules.d/foreman.yml.sample 11 / 26
12. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Proxy Package all gem dependencies Check how packaging is done for RPM and DEB Package the proxy There is a configuration migration script Copy the config files around Check paths in config files Check symlinks You should be all set! 12 / 26
13. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Proxy For FreeBSD Sorry, the font would be much to small 13 / 26
14. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Proxy Plugins bundler_ext is used for proxy plugins Therefore, proxy plugins are gems A file in bundler.d is needed to load the plugin Copy config file Check paths in config file Package gem dependencies Handle other dependencies like salt-master or chef-server You should be all set! 14 / 26
15. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Proxy Plugins for FreeBSD PORTNAME= smart_proxy_dynflow PORTVERSION= 0.0.5 CATEGORIES= sysutils ruby MASTER_SITES= RG MAINTAINER= ruby@FreeBSD.org COMMENT= Plugin to use Dynflow in Foreman Smart Proxy RUN_DEPENDS= foreman-proxy>=1.9.0:${PORTSDIR}/net/foreman-proxy \ rubygem-dynflow>=0.8.4:${PORTSDIR}/devel/rubygem-dynflow \ rubygem-sequel>=0:${PORTSDIR}/devel/rubygem-sequel \ rubygem-sqlite3>=0:${PORTSDIR}/databases/rubygem-sqlite3 NO_ARCH= yes USE_RUBY= yes USE_RUBYGEMS= yes RUBYGEM_AUTOPLIST= yes PLIST_FILES= "@sample etc/foreman-proxy/settings.d/dynflow.yml.sample" \ share/foreman-proxy/bundler.d/dynflow.rb post-install: ${MKDIR} ${STAGEDIR}${PREFIX}/etc/foreman-proxy/settings.d \ ${STAGEDIR}${PREFIX}/share/foreman-proxy/bundler.d ${INSTALL_DATA} ${WRKSRC}/settings.d/dynflow.yml.example \ ${STAGEDIR}${PREFIX}/etc/foreman-proxy/settings.d/dynflow.yml.sample ${INSTALL_DATA} ${WRKSRC}/bundler.d/dynflow.rb \ ${STAGEDIR}${PREFIX}/share/foreman-proxy/bundler.d/dynflow.rb 15 / 26
16. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Puppet Foreman-installer is based on Puppet, so... make sure Puppet (3.x) is working really good on your platform, also when planning to use another configuration management system with Foreman. you will be using puppet apply a lot before even being able to target the installer itself. 16 / 26
17. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Puppet Modules Foreman modules theforeman-* Best to start with theforeman-puppet Modules are designed to also work outside the installer Classical Puppet module development, you’ll see the usual patterns Puppet Labs modules Apache Concat MySQL PostgreSQL Sometimes OS packages have to get modified to be more Puppet friendly locations.ini for mod_passenger on FreeBSD 17 / 26
18. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! kafo kafo is a Ruby gem, taking Puppet modules and generating shiny installers kafo_parsers gem loads Puppet directly, thus not working with Puppet Labs AIO packages currently 18 / 26
19. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Installer Once the proxy is packaged and the needed Puppet modules are ready for your OS, foreman-installer can be targeted. There’s no need to wait for a Foreman core package on your OS. kafo generated installer contains all theforeman-* Puppet modules and their dependencies have a look into DEB/RPM packages for the build process 19 / 26
20. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Installer on FreeBSD PORTNAME= foreman-installer PORTVERSION= 1.10.1 CATEGORIES= sysutils MASTER_SITES= http://downloads.theforeman.org/foreman-installer/ BUILD_DEPENDS= asciidoc>=0:${PORTSDIR}/textproc/asciidoc \ docbook-xsl>=0:${PORTSDIR}/textproc/docbook-xsl \ gsed>=0:${PORTSDIR}/textproc/gsed \ libxslt>=0:${PORTSDIR}/textproc/libxslt \ rubygem-kafo>=0.6.12:${PORTSDIR}/devel/rubygem-kafo RUN_DEPENDS= rubygem-kafo>=0.6.12:${PORTSDIR}/devel/rubygem-kafo USE_RUBY= yes USE_RAKE= yes do-build: cd ${WRKSRC} && \ ${RAKE_BIN} build PREFIX=${LOCALBASE} SBINDIR=${LOCALBASE}/sbin \ SYSCONFDIR=${LOCALBASE}/etc \ KAFO_MODULES_DIR=${LOCALBASE}/lib/ruby/gems/${RUBY_VER}/gems/kafo-0.6.12/modul do-install: cd ${WRKSRC} && \ ${RAKE_BIN} install PREFIX=${STAGEDIR}${LOCALBASE} \ SBINDIR=${STAGEDIR}${LOCALBASE}/sbin \ SYSCONFDIR=${STAGEDIR}${LOCALBASE}/etc && \ ${INSTALL_MAN} ${WRKSRC}/_build/${PORTNAME}.8 ${STAGEDIR}${MAN8PREFIX}/man/man8 20 / 26
21. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Foreman Core Foreman itself is a Ruby on Rails application, with dependencies onto dozens of Gems with specific versions. It’s much easier (especially mid/long-term) to have the Gems for Foreman core seperated from the rest of the system Gems. On Red Hat based systems this is done by using a Ruby from a SCL (software collection), making it also easy to update the Ruby version used. On Debian systems, all needed Gems are coming with the Foreman binary package by using bundle package. Some Gems will bring native extensions, often untested on non-Linux/OSX. JavaScript engines used by execjs (like libv8 or nodejs) will often require porting work, not only OS but also CPU architecture wise. 21 / 26
22. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Provisioning Support 1/2 Have a look into existing implementations: Kickstart (PXELinux) Preseed (PXELinux) Solaris 8-10 (DHCP options) . . . FreeBSD (talk by Martin Matuška) CoreOS (talk by Nils Domrose and Johannes Scheuermann) 22 / 26
23. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Provisioning Support 2/2 know your target OS installer very good some installers are not PXE friendly some installers are not really unattended manually set up PXE provisioning for that OS at least one time to know the DHCP specifics and file contents know PXELinux and how to use it to work around OS installer shortcomings FreeBSD provisioning via PXELinux memdisk and pxelinux.cfg re-usage via tftp to parametrize the install script 23 / 26
24. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Further Goals Foreman core package on FreeBSD Dragonfly BSD (should already be covered by FreeBSD work) NetBSD, OpenBSD OpenSUSE, SLES (OBS project? converge RPM sources?) Solaris AI provisioning Puppet module support Packages NixOS VMware ESX as PXE installable OS XenServer as PXE installable OS setup.exe with proxy and installer for Windows Re-use Puppet Labs Ruby? 24 / 26
25. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Katello Katello as repository server for OpenSUSE and SLES Pulp work already done zypper can use yum repositories yum is already packaged for Spacewalk / SUSE Manager katello-agent and friends yet to be done (stalled OBS project) Katello as repository server for Debian pulp_deb (WIP) package/port katello-agent and friends Katello as repository server for FreeBSD pulp_pkgng (N/A) package/port katello-agent and friends Katello itself on Debian Would be a whole lot of packaging work, but no technical reasons against it Current way to package Foreman core plugins on Debian might interfere 25 / 26
26. Intro Hammer Proxy Installer Foreman Core Further Goals Katello Thanks! Thanks! Questions? Discussion! 26 / 26