Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using RUDDER plugins to package tools and configuration policies

February 04, 2020

Using RUDDER plugins to package tools and configuration policies

🎥 https://www.youtube.com/watch?v=VFQox6_RrBs
🧑 Félix Dallidet
📅 Configuration Management Camp 2020

In RUDDER 5.0 we have introduced a plugins’ ecosystem to make RUDDER more flexible and adaptable to user needs. Plugins aim to bring new functionalities to RUDDER: to plug it with other tools or simply package re-usable policy sets for example. Even though the currently available plugins cover a large range of functionalities, you may need to create new ones or extend current ones to meet your specific needs.

This talk will go through the process of plugin creation and maintaining, focusing on the ones involving proper configuration elements. This will let us see the current possibilities to import, export, share and maintain subsets of configuration policies between distinct RUDDER environments.


February 04, 2020

More Decks by Rudder

Other Decks in Programming


  1. Plugins types Classic plugins: whatever you want (o en script

    to link Rudder to other tools) • Advanced features plugins: extend the webapp capabilities • Policy packages: set of Rudder policies •  4
  2. Rudder-pkg Plugins are managed by the rudder package cli •

    Local json based database under /var/rudder/packages/index.json • Keep track of the unpacked files • Create a dedicated per plugin folder in /opt/rudder/share/plugins •  5
  3. Rudder-pkg index { "name": "rudder-plugin-scale-out-relay", "build-date": "2019-12-08T03:46:37+01:00", "files": [ "/opt/rudder/share/plugins/",

    "/opt/rudder/share/plugins/scale-out-relay/", "/opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar" ], "content": { "files.txz": "/opt/rudder/share/plugins" }, "type": "plugin", "build-commit": "6ca3b2ee3e89f03cdb4873623b3405bc5c84ea67", "jar-files": [ "/opt/rudder/share/plugins/scale-out-relay/scale-out-relay.jar" ], "version": "6.0-1.2" }  6
  4. Plugin format ar archive called rpkg and containing: A metadata

    file in JSON format named medatata • A tarball file in txz format name scripts.txz that contains package control files • One or more tarball files in txz format that contain the package files •  8
  5. Metadata { "type": "plugin", # it is always plugin "name":

    "myplugin", "version": "4.1-1.0", "jar-files": [ "test.jar" ], # if any "depends": { "binary": [ "zip" ] # binary base dep }, # the plugin content (mandatory) "content": { # archive source -> install dir "files.txz": "/opt/rudder/share", "var_rudder.txz": "/var/rudder" } }  9
  6. Control files The file.txz should contains: Should be unpacked in

    /opt/rudder/share/plugins. preinst • prerm • postinst • postrm • and their respective dependencies •  10
  7. Dependencies No plugin dependencies at the moment • Binary based

    dep will look for the binary with a distutils.spawn.find_executable • "depends": { "binary": [ "zip" ] } Other keyword will only be displayed at install time and will not try to detect their fulfillment • "depends": { "dpkg": [ "python-requests" ], "rpm": [ "python-requests" ] } # Will result in the following dipslay at install time This package depends on the following on dpkg : python-requests on rpm : python-requests It is up to you to make sure those dependencies are installed  12
  8. Removal Prerm execution • Removal of all the files indexed

    by rudder package • Removal of the control files in /var/rudder/packages/<plugin name> •  14
  9. Common tools For all Rudder API related stuff: • •

    https://github.com/Normation/rudder-api-client now packaged as rudder-api-client •  17
  10. Methods No API available • Commit the files under /var/rudder/configuration-

    repository/ncf/30_generic_methods/ • You can use intermediate folders to organize it • . └── 30_generic_methods └── partition_check ├── partition_check_mounted.cf └── partition_check_options.cf  21
  11. Techniques Depends of methods, everything via API • No documented

    API but it is public and used by the technique editor when using import|export|save|delete • Under the endpoint /api/techniques • Can be created under technique categories starting 6.0.3 •  22
  12. Techniques categories Created via the technique creation API adding a

    key in the technique JSON at creation time via API • "category": "CIS_plugin", Category must already exist • To create a category just create a folder containing a category.xml under /var/rudder/configuration- repository/techniques •  23
  13. Rudder- synchronize Cli to import/export techniques/directives/rules starting 6.0 • python

    based, depends on the package rudder-api- client • bring it in the plugin, use it in postinst • https://repository.rudder.io/tools/rudder-synchronize  24
  14. Removal/Upgr ade All packaged files are removed at uninstallation You

    need to remove manually all non-packaged elements created by the plugin in prerm  27
  15. Removal API created objects Keep a local base of the

    object id created via the API to remove them a erward • Remove them in the reverse import order •  28
  16. Methods removal issues End user can have created techniques from

    imported methods • curl -k -H "X-API-Token: $(cat /var/rudder/run/api-token)" -X GET path=/var/rudder/configuration-repository/ncf | jq '.data|.techniques[] | select(.method_calls | contains([{"method_name": "<method name to find>"}])) | .bundle_name'  29
  17. Techniques removal issues End user can have: Created directives from

    the imported techniques • • https://docs.rudder.io/api/#api-Techniques- listTechniquesDirectives Modified the imported techniques •  30