JMESpath

idem describe is able to filter it’s results using a tool called JMESpath. JMESpath is a query language for json.

When the --filter option is used with idem describe, the sls data gets changed into a format that is easy to use with jmespath.

For example, a traditional sls state in json format looks like this:

{
    "Description of test.succeed_with_comment": {
        "test.succeed_with_comment": [
            {"name": "succeed_with_comment"},
            {"comment": None},
        ]
    },
}

When performing a JMESpath search on the data, it first gets transformed to look like this:

[
    {
        "name": "Description of test.succeed_with_comment",
        "ref": "test.succeed_with_comment",
        "resource": [{"name": "succeed_with_comment"}, {"comment": None}],
    },
]

The data has been flattened into an list of dictionaries and the keys “name”, “ref”, and “resource” have been added for easy filtering. Don’t worry, the end result is turned back into the sls form unless you supply the –output=jmespath flag.

You can always run idem describe --output=jmespath without --filter to see what the internal jmespath structure looks like.

Practicing with Static Data

Gathering data from the cloud can take a long time. When you are learning how to write JMESpaths, try writing a small script like this one to practice on static data:

# my_filter.py
import jmespath
import pprint
import sys

# In this example, "data" is the output of "idem describe test --output=jmespath"
data = [
    {
        "name": "Description of test.anop",
        "resource": [{"name": "anop"}],
        "ref": "test.anop",
    },
    {
        "name": "Description of test.configurable_test_state",
        "resource": [
            {"name": "configurable_test_state"},
            {"changes": True},
            {"result": True},
            {"comment": ""},
        ],
        "ref": "test.configurable_test_state",
    },
    {"name": "Description of test.describe", "resource": [], "ref": "test.describe"},
    {
        "name": "Description of test.fail_with_changes",
        "resource": [{"name": "fail_with_changes"}],
        "ref": "test.fail_with_changes",
    },
    {
        "name": "Description of test.fail_without_changes",
        "resource": [{"name": "fail_without_changes"}],
        "ref": "test.fail_without_changes",
    },
    {
        "name": "Description of test.mod_watch",
        "resource": [{"name": "mod_watch"}],
        "ref": "test.mod_watch",
    },
    {
        "name": "Description of test.none_without_changes",
        "resource": [{"name": "none_without_changes"}],
        "ref": "test.none_without_changes",
    },
    {
        "name": "Description of test.nop",
        "resource": [{"name": "nop"}],
        "ref": "test.nop",
    },
    {
        "name": "Description of test.succeed_with_changes",
        "resource": [{"name": "succeed_with_changes"}],
        "ref": "test.succeed_with_changes",
    },
    {
        "name": "Description of test.succeed_with_comment",
        "resource": [{"name": "succeed_with_comment"}, {"comment": None}],
        "ref": "test.succeed_with_comment",
    },
    {
        "name": "Description of test.succeed_without_changes",
        "resource": [{"name": "succeed_without_changes"}],
        "ref": "test.succeed_without_changes",
    },
    {
        "name": "Description of test.treq",
        "resource": [{"name": "treq"}],
        "ref": "test.treq",
    },
    {
        "name": "Description of test.update_low",
        "resource": [{"name": "update_low"}],
        "ref": "test.update_low",
    },
]

search_path = sys.argv[1]
pprint.pprint(jmespath.search(search_path, data))

Examples

Now for some examples of filtering with JMESpath. I will use the format of idem describe test --filter="<JMESpath>" in the following examples. If you called the little script we wrote above my_filter.py then the following two commands are equivalent. Keep that in mind as you move your one-off experiments to idem describe:

# Equivalent commands
my_filter.py "<JMESpath>"
idem describe test --output=pretty --filter="<JMESpath>"

Return only the states that use test.update_low

idem describe test --filter="[?ref=='test.update_low']"

output: .. code-block:: yaml

Description of test.update_low:

test.update_low: - name: update_low

Return only the states that start with “test.succeed”

idem describe test --filter="[?starts_with(ref, 'test.succeed']"

output: .. code-block:: yaml

Description of test.succeed_with_changes:

test.succeed_with_changes: - name: succeed_with_changes

Description of test.succeed_with_comment:

test.succeed_with_comment: - name: succeed_with_comment - comment: null

Description of test.succeed_without_changes:

test.succeed_without_changes: - name: succeed_without_changes

Return only tests that have “changes” in the state name:

idem describe test --filter="[?contains(name, 'changes')]"

output:

Description of test.fail_with_changes:
  test.fail_with_changes:
  - name: fail_with_changes
Description of test.fail_without_changes:
  test.fail_without_changes:
  - name: fail_without_changes
Description of test.none_without_changes:
  test.none_without_changes:
  - name: none_without_changes
Description of test.succeed_with_changes:
  test.succeed_with_changes:
  - name: succeed_with_changes
Description of test.succeed_without_changes:
  test.succeed_without_changes:
  - name: succeed_without_changes

Return only states that have “succeed_with_comment” in the “name” parameter

idem describe test --filter="[?resource[?name=='succeed_with_comment']]"

output:

Description of test.succeed_with_comment:
  test.succeed_with_comment:
  - name: succeed_with_comment
  - comment: null

Learn More

https://jmespath.org/tutorial.html

https://jmespath.org/examples.html

https://jmespath.org/specification.html

https://pypi.org/project/jmespath

https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_jmespath.html

https://docs.microsoft.com/en-us/cli/azure/query-azure-cli

https://www.azurecitadel.com/cli/jmespath/