Using RUDDER plugins to package tools and configuration policies

🎥 https://www.youtube.com/watch?v=VFQox6_RrBs
🧑 Félix Dallidet
📅 Configuration Management Camp 2020

In RUDDER 5.0 we have introduced a plugins’ ecosystem to make RUDDER more flexible and adaptable to user needs. Plugins aim to bring new functionalities to RUDDER: to plug it with other tools or simply package re-usable policy sets for example. Even though the currently available plugins cover a large range of functionalities, you may need to create new ones or extend current ones to meet your specific needs.

This talk will go through the process of plugin creation and maintaining, focusing on the ones involving proper configuration elements. This will let us see the current possibilities to import, export, share and maintain subsets of configuration policies between distinct RUDDER environments.

Transcript

1. USE RUDDER PLUGIN TO PACKAGE TOOLS AND POLICIES  1
2. Who am I? Félix Dallidet • Rudder developer • Consultant • • felix@rudder.io @Fdall on IRC and Gitter •  2
3. RUDDER PLUGINS  3
4. Plugins types Classic plugins: whatever you want (o en script to link Rudder to other tools) • Advanced features plugins: extend the webapp capabilities • Policy packages: set of Rudder policies •  4
5. Rudder-pkg Plugins are managed by the rudder package cli • Local json based database under /var/rudder/packages/index.json • Keep track of the unpacked files • Create a dedicated per plugin folder in /opt/rudder/share/plugins •  5
6. Rudder-pkg index { "name": "rudder-plugin-scale-out-relay", "build-date": "2019-12-08T03:46:37+01:00", "files": [ "/opt/rudder/share/plugins/", "/opt/rudder/share/plugins/scale-out-relay/", "/opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar" ], "content": { "files.txz": "/opt/rudder/share/plugins" }, "type": "plugin", "build-commit": "6ca3b2ee3e89f03cdb4873623b3405bc5c84ea67", "jar-files": [ "/opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar" ], "version": "6.0-1.2" }  6
7. PLUGIN STRUCTURE  7
8. Plugin format ar archive called rpkg and containing: A metadata file in JSON format named medatata • A tarball file in txz format name scripts.txz that contains package control files • One or more tarball files in txz format that contain the package files •  8
9. Metadata { "type": "plugin", # it is always plugin "name": "myplugin", "version": "4.1-1.0", "jar-files": [ "test.jar" ], # if any "depends": { "binary": [ "zip" ] # binary base dep }, # the plugin content (mandatory) "content": { # archive source -> install dir "files.txz": "/opt/rudder/share", "var_rudder.txz": "/var/rudder" } }  9
10. Control files The file.txz should contains: Should be unpacked in /opt/rudder/share/plugins. preinst • prerm • postinst • postrm • and their respective dependencies •  10
11. Versioning Based on the schema: <Rudder major version>-<Plugin version> • 6.0-1.0 • 5.0-1.2 •  11
12. Dependencies No plugin dependencies at the moment • Binary based dep will look for the binary with a distutils.spawn.find_executable • "depends": { "binary": [ "zip" ] } Other keyword will only be displayed at install time and will not try to detect their fulfillment • "depends": { "dpkg": [ "python-requests" ], "rpm": [ "python-requests" ] } # Will result in the following dipslay at install time This package depends on the following on dpkg : python-requests on rpm : python-requests It is up to you to make sure those dependencies are installed  12
13. Basic tree root@server:ar t mydemo_plugin-6.0-1.0.rpkg metadata files.txz scripts.txz  13
14. Removal Prerm execution • Removal of all the files indexed by rudder package • Removal of the control files in /var/rudder/packages/<plugin name> •  14
15. Upgrade remove → install •  15
16. IN GENERAL  16
17. Common tools For all Rudder API related stuff: • • https://github.com/Normation/rudder-api-client now packaged as rudder-api-client •  17
18. POLICY PLUGINS  18
19. Standard techniques and elements  19
20. Packaging policies  20
21. Methods No API available • Commit the files under /var/rudder/configuration- repository/ncf/30_generic_methods/ • You can use intermediate folders to organize it • . └── 30_generic_methods └── partition_check ├── partition_check_mounted.cf └── partition_check_options.cf  21
22. Techniques Depends of methods, everything via API • No documented API but it is public and used by the technique editor when using import|export|save|delete • Under the endpoint /api/techniques • Can be created under technique categories starting 6.0.3 •  22
23. Techniques categories Created via the technique creation API adding a key in the technique JSON at creation time via API • "category": "CIS_plugin", Category must already exist • To create a category just create a folder containing a category.xml under /var/rudder/configuration- repository/techniques •  23
24. Rudder- synchronize Cli to import/export techniques/directives/rules starting 6.0 • python based, depends on the package rudder-api- client • bring it in the plugin, use it in postinst • https://repository.rudder.io/tools/rudder-synchronize  24
25. Directives Depends of techniques, everything via API • • https://docs.rudder.io/api/#api-Directives- createDirective  25
26. Rules Depends of directives, everything via API • • https://docs.rudder.io/api/#api-Rules-createRule  26
27. Removal/Upgr ade All packaged files are removed at uninstallation You need to remove manually all non-packaged elements created by the plugin in prerm  27
28. Removal API created objects Keep a local base of the object id created via the API to remove them a erward • Remove them in the reverse import order •  28
29. Methods removal issues End user can have created techniques from imported methods • curl -k -H "X-API-Token: $(cat /var/rudder/run/api-token)" -X GET https://127.0.0.1/ncf/api/techniques? path=/var/rudder/configuration-repository/ncf | jq '.data|.techniques[] | select(.method_calls | contains([{"method_name": "<method name to find>"}])) | .bundle_name'  29
30. Techniques removal issues End user can have: Created directives from the imported techniques • • https://docs.rudder.io/api/#api-Techniques- listTechniquesDirectives Modified the imported techniques •  30
31. THANK YOU  31