Group plugins - hub.group

Group plugins are capable of organizing and manipulating the return data of an idem run before it is rendered.

Finding Group Plugins

Use idem doc to list all the available group plugins:

$ idem doc group | grep -o "^group.\w*"

You can also use jq to list plugins next to their doc strings:

$ idem doc group --output=json | jq -r '. | keys[] as $k | "\($k): \(.[$k].doc)"'

At the time of writing, with no extra plugins installed, this is the result of that command:

group.duration.apply: Sort the output by the total seconds of the state's run time
group.init.apply: Apply all group plugins specified in config
group.number.apply: Sort the data by run number
group.omit_noop.apply: Remove states that reported success without changes
group.sort.apply: Reorganize the data by sorting by each state's unique tag

Using Group Plugins

Group plugins are specified in a single string and are separated by a pipe. Each group plugin will be run in the order that it is defined.

Group plugins can be specified on the cli with the “–group” flag.

$ idem state my_state.sls --group="number|omit_noop"

Group plugins can also be specified in the idem config.

# idem.cfg
idem:
  group: number|omit_noop

Creating a Group Plugin

First create a directory at my_project_root/my_provider/group. In your project’s conf.py, extend idem’s namespace with your “group” directory.

# my_project_root/my_provider/conf.py
DYNE = {"group": ["group"]}

Now create a plugin in your “source” directory.

The plugin simply needs to implement the apply function. This function receives the hub, and data arguments. These will be passed into the function.

data contains the full results of a state run. Here are the contents of data for a run containing a single state:

{
    "resource_ref_|-state_block_name_|-state_name_|-function": {
        "tag": "resource_ref_|-state_block_name_|-state_name_|-function",
        "name": "state_name",
        "changes": {},
        "new_state": {},
        "old_state": {},
        "comment": None,
        "rerun_data": None,
        "result": True,
        "esm_tag": "resource_ref_|-state_block_name_|-state_name_|-",
        "__run_num": 1,
        "start_time": "2022-08-15 09:39:33.608291",
        "total_seconds": 0.001207,
        "sls_meta": {"SLS": {}, "ID_DECS": {}},
    },
}

This is what a basic group plugin looks like:

# my_project_root/my_provider/group/my_plugin.py
from typing import Any
from typing import Dict


def apply(hub, data: Dict[str, Any]) -> Dict[str, Any]:
    """
    Re-organize/filter the state runtime results from "data"
    """
    # initialize an ordered dictionary for the return (All dictionaries are ordered after python 3.7)
    ret = {}
    # iterate over the state return data in the order you want to add it to the return
    for tag in data:
        # retrieve the result of a single state
        state_ret = data[tag]
        # Break the state tag into it's component parts
        comps = tag.split("_|-")
        state = comps[0]
        id_ = comps[1]
        fun = comps[3]
        # "result" is True if the state ran successfully, otherwise it is False
        result = state_ret.get("result")
        # Any comment(s) from the running state
        comment = state_ret.get("comment")
        # An empty dictionary if there were no changes, else a comparison of "new" and "old" state of the resource
        changes = state_ret.get("changes", {})
        # The state of the resource before the function ran
        old_state = state_ret.get("old_state", {})
        # The state of the resource after the function ran
        new_state = state_ret.get("new_state", {})

        # omit this state ret from the return data based on any of the previous information
        if not True:
            continue

        # Copy the state ret from the input data to the return data
        ret[tag] = data[tag]

    return ret