SLS Structure#

An Idem structured layer state (SLS) file contains plain data that represents the desired state of a system. You can write layered SLS data in any format: JSON, YAML, XML, or even a programming language. SLS makes it easy to write Idem data in whatever way works best for you.

Core Components#

SLS files are represented in YAML by default. Unlike other YAML systems, SLS has a finite depth, making it easy to read, write, and understand.

To learn the data structure, begin with the ID declaration, path reference, and argument components, which are central to all SLS files.

Some_Desired_State:        # ID Declaration
  cloud.instance.present:  # Path Reference
    - option: value        # Argument

ID Declaration#

The ID declaration is the top-level state identifier for the data under it. The ID declaration is also used by the state function as the name unless you specify a separate name argument.

Path Reference#

The path reference specifies the underlying function called in order to enforce an idempotent state at the target system. The dot delimited path corresponds to how plugins are loaded into Idem using POP, and is a literal reference to a location on the hub inside Idem.

The hub contains all of the code that Idem runs, making the path reference a pointer to the code location hub.idem.states.<path-reference>. This gives you direct insight into where code that is being called resides, which simplifies development and debugging. If an Idem state is exhibiting a problem, you know exactly where to look.

Path Reference Components#

The path reference consists of a state ref and function ref separated by the final dot. Everything before the final dot is the state ref, followed by the function ref after the final dot.

For example, the path reference cloud.network.present includes state ref cloud.network and function ref present.

Arguments#

Idem data is somewhat self-documenting. Since the path reference is a path to a function, the arguments are almost always arguments for that specific function.

Be aware that some arguments are applicable to all state definitions though; such as name, order, and requisites.

Name and Names#

Every state has a name as its primary identifier. If you don’t supply a name argument, Idem defaults to using the ID declaration as the name.

The plural names lets you easily work with multiple, identical resources. You can supply a list of names as shown in the following example, which creates nine identical cloud instances named web1 through web9.

Some_machines:
  cloud.instance.present:
    - names:
      - web1
      - web2
      - web3
      - web4
      - web5
      - web6
      - web7
      - web8
      - web9

Order#

By default, Idem executes an SLS file in the order that it is written, from top to bottom. You can affect the order with the following strategies.

  • The order keyword

    The order keyword is a simple number sequence where 1 is first, 2 is next, and so on.

    To make an SLS operation come last regardless of how many operations preceded it, set the order to -1. Second-to-last would be -2, and so on.

State_A:
  cloud.instance.present:
    - order: -1  # Make it last
State_B:
  cloud.instance.present:
    - order: 1  # Make it first

  • Parallel runtime

    Parallel runtime simultaneously executes every operation in an SLS file regardless of order written or the presence of order keywords. In the previous example, both instances would be created at the same time.

    The only operations not run in parallel are those that are bound by requisites.

  • Requisites

    Requisites are dependencies between operations as described in the next section.

Requisites#

Requisites set the relationships that states have with each other. They help Idem enforcement run faster and more reliably, and can be used to set up interdependent tasks. Relationships are evaluated at runtime.

A requisite is passed to a state as an argument. In the following example, State_A will only run after State_B has completed successfully.

State_A:
  cloud.instance.present:
    - require:
      - cloud.instance: State_B

State_B:
  cloud.network.present:

Referencing uses the state ref component of the path reference, followed by the name or ID of the state with which to create a relationship.

Requisite Ins

You can append _in to a requisite to reverse the call direction of a dependency. Think of a standard requisite as I require you and a reverse _in requisite as you need to require me.

In the following standard requisite, State_A is saying I require State_B.

State_A:
  cloud.instance.present:
    - require:
      - cloud.network: State_B

State_B:
  cloud.network.present:

A reverse _in requisite achieves the same effect by having State_B say that State_A needs to require me..

State_A:
  cloud.instance.present

State_B:
  cloud.network.present:
    - require_in:
      - cloud.instance: State_A

Top Level Keys#

Idem supports top level keys that allow your current SLS file to run content from other SLS files, and to optionally make local adjustments to that content prior to the run.

Include#

To run the contents from another SLS file in your current Idem run, place an include at the top of the current SLS file, followed by references to the other SLS files.

include:
  - aws.instances
  - azure.networks

In an SLS file, an include block can exist along with traditional states.

include:
  - aws.instances
  - azure.networks

aws.resource-1:
    aws.resource-1.present:
        . . .

References to other SLS files are evaluated relative to the current file location.

  • Directories are dot delimited and relative to the directory of the current file.

  • The final entry in a dot delimited path can be an SLS file with or without its extension, or a directory.

  • An assumed init.sls file after the final path entry is supported.

    An init.sls file is analogous to an assumed index.html file at the end of a URL.

In the following example, Idem looks for the included SLS file in the order listed.

include:
  - aws.instances

  1. Idem first checks for an aws/instances.sls file.

  2. Idem then checks for an aws/instances/init.sls file.

Included files don’t need to be in subdirectories under the current location. For relative referencing, you can add preceding dots to the path.

The following include appears in file aws/my-states.sls.

include:
  - .ec2.vpcs

In the example, Idem goes up one directory to look for the included SLS file.

  1. Idem first checks for an aws/ec2/vpcs.sls file.

  2. Idem then checks for an aws/ec2/vpcs/init.sls file.

Extend#

The extend keyword lets you modify a state that comes from an included SLS file.

An extend is useful if you’re managing multiple, nearly identical resources. You include SLS files that specify certain values, then use extend to overwrite values as needed.

include:
  - gcp.networks

extend:
  Network_1:
    gcp.network.present:
      - ip_range: 10.10.57

In the preceding example, the Network_1 ID coming from the included gcp.networks SLS file is overwritten with a new IP range value.

Rules to remember when extending states:

  • Don’t forget to include the SLS being extended.

  • An extend block is a top-level declaration. You cannot have more than one extend block in the same SLS file.

  • Multiple IDs can be added for extension under the single extend block.

  • For requisites watch and require, an extend can only append to the value. It can’t overwrite the value like it does in all other cases.

The order of execution can be effectively nullified when using the parallel runtime. Idem can execute using either a serial, or a parallel runtime. The parallel runtime will evaluate all requisites and then run everything that is not encumbered by a requisite at the same time. This means that if you are using the parallel runtime, the order keyword will have no effect.

If you are using the serial runtime, then each state is executed one after another. This means that the order keyword can be used to change the order in which things are executed. Use the order keyword and pass in a number. The default ordering defined by Idem will add numbers based on the highest order value passed in. This means that if you pass order: 1 then that state will be evaluated first. Similarly, you can pass order: -1 and start with negative numbers to ensure that states are executed LAST.

State_A:
  cloud.instance.present:
    - order: -1  # Make it last
State_B:
  cloud.instance.present:
    - order: 1  # Make it first

Again, remember, that if you are using the parallel runtime, then both of these instances would be created at the same time.

Requisites#

The requisite system inside of Idem is very powerful at determining the relationships that states have with each other. Being able to define requisites can make your enforcement significantly faster, more reliable, and can be used to create tasks.

These relationships are evaluated at runtime and can handle dynamic situations within your state definitions. A requisite is passed to a state as an argument:

State_A:
  cloud.instance.present:
    - require:
      - cloud.instance: State_B

State_B:
  cloud.network.present:

Is this case, State_A will only run once State_B has completed successfully. The referencing works by taking the state ref component of the path ref followed by the name or ID of the desired state to create a relationship with.

Requisite Ins#

The requisites come in two flavors - requisites, and requisite ins. Every requisite that exists can be appended with an _in to specify that the direction changes.

A standard requisite states “I require you”. For instance, this state is a standard requisite:

State_A:
  cloud.instance.present:
    - require:
      - cloud.network: State_B

State_B:
  cloud.network.present:

State_A is saying “I require State_B”

A requisite in simply says “They require me”. This means that we can get the same effect as the requisite code above with this requisite in:

State_A:
  cloud.instance.present

State_B:
  cloud.network.present:
    - require_in:
      - cloud.instance: State_A

So in this case, we are saying “I am State_B, State_A needs to require me”.

Top Level Keys#

Idem has a number of top level keys that can be used to include additional SLS files or to exclude specific IDs. You can also modify ID declarations from another file with the extend keyword.

Include#

Include simply allow you to include information from another sls file in your run. Using include is simple, at the top of your SLS file just add include followed by a list of SLS references you wish to include in your run:

include:
  - foo.bar
  - azure.networks

The include statement evaluates SLS paths. You can easily execute Idem against a single SLS file, but Idem supports having a file tree. References to locations on the file tree are dot delimited and reference directories.

For instance, the SLS file you execute is assumed to be at the root of the tree. So if you execute an SLS file called start.sls, and it has the include statement:

include:
  - aws.instances
  - azure.networks

Then Idem will look for these files in a few locations. Idem will check the aws directory for a file called instances.sls, and if it does not find that file, it will check for a directory called aws/instances/init.sls.

Note that an SLS file can have include block along with states. Here is a possible aws/instances.sls:

include:
  - .ec2.vpcs

aws.resource-1:
    aws.resource-1.present:
        ....

An SLS file aws/ec2/vpcs.sls or aws/ec2/vpcs/init.sls is expected.

Include references need to be relative to an sls source For example, if “/srv/idem” is in my sls-sources, then my include statements need to be relative to /srv/idem.

Consider the files:

  • /srv/idem/init.sls

  • /srv/idem/foo/bar/init.sls

This is how /srv/idem/init.sls can include /srv/idem/foo/bar/init.sls:

include:
  - foo.bar

Notice that I used just “foo.bar”. “init.sls” is a special name. If the sls reference is a directory, idem automatically infers that there is an “init.sls” file in the directory.

Now consider the files:

  • /srv/idem/foo/init.sls

  • /srv/idem/foo/bar/init.sls

With the same sls-root at “/srv/idem”, this is how “/srv/idem/foo/init.sls” can include “/srv/idem/foo/bar/init.sls”:

include:
  - .bar

This is known as a “relative include”. “.bar” is not relative to an sls root. The “.” indicates that the include is relative to the parent of the file defining the include, not the sls root.

Extend#

The extend keyword allows for diving into the state compiler and modifying a state from another included SLS file. This allows you to modify an external state. This can be useful if you are using your Idem code to manage multiple clouds that are NEARLY identical. So you can include the SLS files that define some external ID Declarations, then overwrite the options passed to them:

include:
  - gcp.networks

extend:
  Network_1:
    gcp.network.present:
      - ip_range: 10.10.57

In this case, the Network_1 ID from the gcp.networks SLS file will be overwritten with a new option.

RULES TO EXTEND BY#

There are a few rules to remember when extending states:

  1. Always include the SLS being extended with an include declaration

  2. Requisites (watch and require) are appended to, everything else is overwritten

  3. extend is a top level declaration, like an ID declaration, cannot be declared twice in a single SLS

  4. Many IDs can be extended under the extend declaration