HomeTechnologyWrite your own Red Hat Ansible Tower inventory plugin

Write your own Red Hat Ansible Tower inventory plugin

Write your own Red Hat Ansible Tower inventory plugin

Ansible is an engine and language for automating many different IT tasks, such as provisioning a physical device, creating a virtual machine, or configuring an application and its dependencies. Ansible organizes these tasks in playbook files, which run on 1 or more remote goal hosts. Inventory files maintain lists of these hosts and are formatted as YAML or INI documents. For example, a simple inventory file in INI format follows:

[web]
web1.example.com
web2.example.com

Ansible inventories can be static (stored in a file and managed in a source code repository) or dynamic (retrieved from an external web resource, such as through a RESTful API). Dynamic inventories are generated on-demand using inventory scripts or inventory plugins, consisting of code that Ansible runs to get a list of hosts to goal when executing playbooks.

Red Hat Ansible Tower, also known as AWX (the name of its upstream community project), is a entrance-end to Red Hat Ansible Engine that simplifies operations on large IT infrastructures. Operators can log into the Ansible Tower web interface and create single jobs or complex workflows using Ansible Engine building blocks such as tasks, roles, and playbooks. Enterprises typically handle assets in a configuration management database (CMDB), such as NetBox, which Ansible Tower connects to using a specially written script or plugin.

This article shows you how to use Ansible Tower to create dynamic inventories. We’ll start with a sample inventory script, then transform the script into a plugin. As you’ll see, inventory plugins can accept parameters, which gives them an advantage over plain scripts.

Note: Inventory scripts are deprecated in Ansible Tower, so they will be removed in a prospective version. There’s a fine reason: Source code is properly managed in a version control system, where developers and operators can track and review changes to its corpus.

A sample inventory script

Inventory scripts are organized in a single executable file, written in a scripting language such as Python or Bash. The script should return its data in JSON format. For instance, the following output provides the Ansible playbook with a list of hosts and related data:

{
    "all": {
        "hosts": ["web1.example.com", "web2.example.com"]
    },
    "_meta": {
        "hostvars": {
            "web1.example.com": {
                "ansible_user": "root"
            },
            "web2.example.com": {
                "ansible_user": "root"
            }
        }
    }
}

The following Bash code is an inventory script that generates the output just proven:

#!/usr/bin/env bash
# id: scripts/trivial-inventory-script.sh

cat << EOF
{
    "all": {
        "hosts": ["web1.example.com", "web2.example.com"]
    },
    "_meta": {
        "hostvars": {
            "web1.example.com": {
                "ansible_user": "rdiscala"
            },
            "web2.example.com": {
                "ansible_user": "rdiscala"
            }
        }
    }
}
EOF

Here, an Ansible command runs the inventory script and compares the actual output to the anticipated output:

$ ansible -m ping -i scripts/trivial-inventory-script.sh all
web1.example.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": untrue,
    "ping": "pong"
}
web2.example.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": untrue,
    "ping": "pong"
}

The output shows that Ansible properly interpreted the information given in the hostvars section and used my username rdiscala to connect via SSH to the server hosts.

Note: The sample script is intentionally brief and omits a detail: Ansible invokes these scripts with the --list option if a list of hosts needs to be produced, as it does in our case. Alternatively, Ansible provides the --host=NAME option when it needs the variables of a specific host, identified by its NAME. To make the script fully compliant, you would need to implement logic to handle these options.

Making scripts work in Ansible Tower

Scripts are defined in the Inventory Scripts section of Ansible Tower’s web interface. Alternatively, you can write a script in any scripting language supported on the Ansible Tower host. As proven in Figure 1, you can paste the script we’ve just written immediately into the CUSTOM SCRIPT field and use it to sync an inventory inside Ansible Tower.

Ansible Tower's Inventory Scripts screen contains a text field named CUSTOM SCRIPT, where an administrator can insert an inventory script.

Figure 1: You can plug a pre-written script into Ansible Tower’s Inventory Scripts section.

We can now use this new script as an inventory source in any Ansible Tower inventory. An inventory source provides information about hosts to Ansible Tower on demand. When the source syncs, the script will run, fetch the data, and format it as proven previously so that Ansible Tower can import it into its own host database. The complete list of hosts will show up in the HOSTS desk, as proven in Figure 2.

Ansible Tower's Inventory Scripts screen contains a text field named CUSTOM SCRIPT, where an administrator can insert an inventory script.

Figure 2: Find the complete list of hosts in the HOSTS desk.

Create an inventory plugin with Ansible Galaxy

The newer and counseled way to distribute and consume Ansible content is to create an inventory plugin and package it as an Ansible collection. An inventory plugin is considered a module when packaged in a collection.

You can kickstart your effort by using the Ansible Galaxy command-line program to create the basic structure for a collection:

$ ansible-galaxy collection init zedr.blog_examples
- Collection zedr.blog_examples was created successfully
$ tree .
.
└── zedr
    └── blog_examples
        ├── docs
        ├── galaxy.yml
        ├── plugins
        │   └── README.md
        ├── README.md
        └── roles

Let’s start with galaxy.yml, the manifest file describes this collection:

namespace: zedr
name: blog_examples
version: 1.0.0
readme: README.md
authors:
  - Rigel Di Scala <[email protected]>

We will create our plugin as a Python script named example_hosts.py inside the plugins/inventory folder. Placing the script in this location lets Ansible detect it as an inventory plugin. We can delete the docs and roles folders to focus on the minimal viable set of files desired to implement our collection. We should end up with a folder structure like this 1:

$ tree .
.
└── zedr
    └── blog_examples
        ├── galaxy.yml
        ├── plugins
        │   └── inventory
        │       └── example_hosts.py
        └── README.md

Important: Always specify the full namespace of the collection (for instance, zedr.blog_examples) when referring to assets contained inside it, such as roles and plugins.

We can now copy over, clean up, and populate the basic boilerplate code for an inventory plugin:

from ansible.plugins.inventory import BaseInventoryPlugin

ANSIBLE_METADATA = {
    'metadata_version': '',
    'status': [],
    'supported_by': ''
}

DOCUMENTATION = '''
---
module:
plugin_type:
short_description:
version_added: ""
description:
options:
author:
'''


class InventoryModule(BaseInventoryPlugin):
    """An example inventory plugin."""

    NAME = 'FQDN_OF_THE_PLUGIN_GOES_HERE'

    def verify_file(self, path):
        """Verify that the source file can be processed properly.

        Parameters:
            path:AnyStr The path to the file that needs to be verified

        Returns:
            bool True if the file is legitimate, else False
        """

    def parse(self, inventory, loader, path, cache=True):
        """Parse and populate the inventory with data about hosts.

        Parameters:
            inventory The inventory to populate
        """
        # The following invocation helps Python 2 in case we are
        # nonetheless relying on it. Use the more handy, pure Python 3 syntax
        # if you don't need it.
        super(InventoryModule, self).parse(inventory, loader, path, cache)

About the code

You’ll note that this boilerplate defines 2 methods: verify_file() and parse(). Use verify_file() when the host list you want to process comes from a file, such as a CSV document, on a filesystem at a given path. This method is used to validate the file quickly before passing it to the more expensive parse() method. Normally, verify_file() ensures that the file is legitimate incoming JSON and matches a predefined schema. (Note that the verify_file() method is currently empty and should be stuffed in.)

Note: The verify_file() method can return True when enter comes from a source other than a file, such as when calling a remote HTTP API. But it could also verify the incoming JSON.

The parse() method does most of the work of processing the source data to filter and format it properly. However, instead of immediately constructing the payload’s dict namespace, as we did in the inventory script, we will rely on the instance attribute, self.inventory, which is a special object with its own methods. The attribute offers add_host() and set_variable() methods to construct a data object suitable for Ansible to consume. (The parse() method is currently empty except for a call to the superclass’s function.)

Additionally, note that the module-level attributes ANSIBLE_METADATA and DOCUMENTATION are required, and that the NAME attribute should have the plugin’s fully certified domain name, including the namespace.

Invoking the plugin

When the plugin is invoked in Ansible from the command line, the following chain of events occurs:

  1. The conventional name InventoryModule is imported from the chosen inventory module (zedr.blog_example.example_hosts.py).
  2. An instance of InventoryModule is created.
  3. The instance method InventoryModule.verify_file() is called to perform an initial validation of the file (when applicable) and is anticipated to return a truthy value to proceed.
  4. The instance method InventoryModule.parse() is called to populate the InventoryModule.inventory object.
  5. The InventoryModule.inventory object is introspected to retrieve the host data that Ansible will consume.

We can now rewrite the script logic as follows:

from ansible.plugins.inventory import BaseInventoryPlugin

ANSIBLE_METADATA = {
    'metadata_version': '1.0.0',
    'status': ['preview'],
    'supported_by': 'community'
}

DOCUMENTATION = '''
---
module: example_hosts
plugin_type: inventory
short_description: An example Ansible Inventory Plugin
version_added: "2.9.13"
description:
    - "A very simple Inventory Plugin created for demonstration purposes only."
options:
author:
    - Rigel Di Scala
'''

class InventoryModule(BaseInventoryPlugin):
    """An example inventory plugin."""

    NAME = 'zedr.blog_examples.example_hosts'

    def verify_file(self, path):
        """Verify that the source file can be processed properly.

        Parameters:
            path:AnyStr The path to the file that needs to be verified

        Returns:
            bool True if the file is legitimate, else False
        """
        # Unused, always return True
        return True

    def _get_raw_host_data(self):
        """Get the raw static data for the inventory hosts

        Returns:
            dict The host data formatted as anticipated for an Inventory Script
        """
        return {
            "all": {
                "hosts": ["web1.example.com", "web2.example.com"]
            },
            "_meta": {
                "hostvars": {
                    "web1.example.com": {
                        "ansible_user": "rdiscala"
                    },
                    "web2.example.com": {
                        "ansible_user": "rdiscala"
                    }
                }
            }
        }

    def parse(self, inventory, *args, **kwargs):
        """Parse and populate the inventory with data about hosts.

        Parameters:
            inventory The inventory to populate

        We ignore the other parameters in the prospective signature, as we will
        not use them.

        Returns:
            None
        """
        # The following invocation helps Python 2 in case we are
        # nonetheless relying on it. Use the more handy, pure Python 3 syntax
        # if you don't need it.
        super(InventoryModule, self).parse(inventory, *args, **kwargs)

        raw_data = self._get_raw_host_data()
        _meta = raw_data.pop('_meta')
        for group_name, group_data in raw_data.items():
            for host_name in group_data['hosts']:
                self.inventory.add_host(host_name)
                for var_key, var_val in _meta['hostvars'][host_name].items():
                    self.inventory.set_variable(host_name, var_key, var_val)

Note that we have ignored facilities related to grouping and caching to hold things simple. These facilities are worth looking into to organize the host list better and optimize the synchronization process’s performance.

Build, install, and test the plugin

The next step is to construct the Ansible collection package, install it regionally, and test the plugin:

$ cd zedr/blog_examples
$ mkdir construct
$ ansible-galaxy collection construct -f --output-path construct
Created collection for zedr.blog_examples at /home/rdiscala/blog/ansible-tower-inventory-plugin/collections/zedr/blog_examples/construct/zedr-blog_examples-1.0.0.tar.gz
$ ansible-galaxy collection install construct/zedr-blog_examples-1.0.0.tar.gz
Process install dependency map
Starting collection install process
Installing 'zedr.blog_examples:1.0.0' to '/home/rdiscala/.ansible/collections/ansible_collections/zedr/blog_examples'

Next, we need to enable our plugin by adding a native galaxy.cfg file in our current working directory. The contents are:

[inventory]
enable_plugins = zedr.blog_examples.example_hosts

To check whether the native installation was successful, we can attempt to display the documentation for our inventory plugin, using its fully certified domain name:

$ ansible-doc -t inventory zedr.blog_examples.example_hosts
> INVENTORY    (/home/rdiscala/.ansible/collections/ansible_collections/zedr/blog_examples/plugins/inventory/example_hosts.py)

        An example Inventory Plugin created for demonstration purposes only.

  * This module is maintained by The Ansible Community
AUTHOR: Rigel Di Scala <[email protected]>
        METADATA:
          status:
          - preview
          supported_by: community

PLUGIN_TYPE: inventory

We can also list the available plugins to verify that ours is detected properly. Note that for this to work with the Ansible collection, you will need Ansible version 3.0 or higher:

$ ansible-doc -t inventory -l
advanced_host_list                                 Parses a 'host list' with ranges
amazon.aws.aws_ec2                                 EC2 inventory source
amazon.aws.aws_rds                                 rds instance source
auto                                               Loads and executes an inventory plugin specified in a YAML config

(...)

zedr.blog_examples.example_hosts                   A trivial example of an Ansible Inventory Plugin

Finally, we can test the plugin regionally by operating it using an inventory configuration file. Create a file named inventory.yml with the following content:

plugin: "zedr.blog_examples.example_hosts"

Here is the command to invoke the plugin and generate the inventory data:

$ ansible-inventory --list -i inventory.yml
{
    "_meta": {
        "hostvars": {
            "web1.example.com": {
                "ansible_user": "rdiscala"
            },
            "web2.example.com": {
                "ansible_user": "rdiscala"
            }
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
    "ungrouped": {
        "hosts": [
            "web1.example.com",
            "web2.example.com"
        ]
    }
}

Ansible has generated 2 “virtual” groups: ungrouped, with our list of hosts, and all, which includes ungrouped. We have verified that the plugin is working properly.

Making the plugin work in Ansible Tower

Ansible Tower can automate a collection’s installation, making its roles and plugins available to initiatives and job templates. To make it work, we need the following:

  • A place to provide the package file that we constructed for our collection. We’ll use a Git repo hosted on GitHub, but it could also be revealed on Ansible Galaxy.
  • A repo for the project files containing the requirements.yml file that references our collection and the inventory.yml configuration file we used previously.
  • An Ansible Tower project that points to the project files repo.
  • An Ansible Tower inventory.
  • An Ansible Tower inventory source for our inventory.

The following events will be triggered when Ansible Tower executes a job that uses this inventory:

  1. The job triggers a project update (the internal project_update.yml playbook).
  2. The project syncs with its associated Git repo.
  3. If necessary, the project installs any desired dependencies, which should be listed in the collection/requirements.yml file.
  4. The project update triggers an inventory update.
  5. The inventory update triggers an inventory source sync.
  6. The inventory source sync reads the inventory file inventory.yml and runs our plugin to fetch the host data.
  7. The host data populates the inventory.
  8. The job runs the associated playbook on the inventory host list using the provided hostnames and variables.

Figure 3 shows this workflow.

Visualizing the inventory-update process workflow just described.

Figure 3: The workflow for populating a host list using an inventory plugin.

Now, let’s create the components required to make the plugin work.

Note: The following example was examined on Ansible Tower 3.7.1.

Create a Git repo for the collection

To start, we’ll create a new repo on Github and push the collection files we created earlier. A sample repo is available on GitHub.

Ansible cannot clone a repository and construct the collection by itself, so we need to construct the package and make it available as a downloadable tar.gz file. As an example, from the Releases page.

Note: At the time of writing, Ansible Tower cannot fetch the package as an authenticated user, so you will need to allow anonymous clients.

If you are using GitHub, you can set up a GitHub Actions workflow to fully automate this process:

# id: .github/workflows/main.yml

name: CI

# Only construct releases when a new tag is pushed.
on:
  push:
    tags:
      - '*'

jobs:
  construct:
    runs-on: ubuntu-latest

    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can entry it
      - uses: actions/[email protected]

      # Extract the version from the tag name so it can be used later.
      - name: Get the version
        id: get_version
        run: echo ::set-output name=VERSION::${GITHUB_REF#refs/tags/}

      # Install a recent version of Python 3
      - name: Setup Python
        uses: actions/[email protected]
        with:
          python-version: 3.7

      # Install our dependencies, e.g. Ansible
      - name: Install Python 3.7
        run: python3.7 -m pip install -r requirements.txt

      - name: Build the Ansible collection
        run: |
          mkdir -p construct
          ansible-galaxy collection construct -f --output-path construct

      - name: Create a Release
        id: create_a_release
        uses: actions/[email protected]
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ steps.get_version.outputs.VERSION }}
          release_name: Release ${{ steps.get_version.outputs.VERSION }}
          draft: untrue

      - name: Upload a Release Asset
        uses: actions/[email protected]
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ steps.create_a_release.outputs.upload_url }}
          asset_path: construct/zedr-blog_examples-${{ steps.get_version.outputs.VERSION }}.tar.gz
          asset_name: "zedr-blog_examples-${{ steps.get_version.outputs.VERSION }}.tar.gz"
          asset_content_type: "application/gzip"

Create a Git repo for project files

Next, we need another Git repo for the files that the Ansible Tower project will source. Here is the folder structure:

$ tree .
.
├── collections
│   └── requirements.yml
└── inventory.yml

Note that collections/requirements.yml will contain a reference to our Ansible collection package so that Ansible Tower can download, install, and use it when the inventory is synced. Additionally, the inventory.yml is the same file we created earlier, containing the plugin’s fully certified domain name. See the example repo for more details.

Create a new Ansible Tower project

Next, sign in to your Ansible Tower instance, create a new project, and fill in the following fields and checkboxes:

  • Name: My Project.
  • Organization: Default (or whatever you prefer).
  • SCM Type: Git.
  • SCM URL: https://github.com/zedr-automation/example_project.git (or the Git repo URL of your project).
  • SCM Branch/Tag/Commit: master.
  • SCM Update Options: select Clean, Delete On Update, and Update Revision on Launch.

Figure 4 shows the ensuing form.

This form creates the Ansible Tower project.

Figure 4: Creating the Ansible Tower project.

Create a new Ansible Tower inventory

There are just 2 fields to create a new inventory in Tower: For the Name field, enter My Inventory. For the Organization, you can select the default or whatever you previously entered. Figure 5 shows the ensuing form.

This form creates the Ansible Tower inventory.

Figure 5: Creating the Ansible Tower inventory.

Create a new inventory source for the inventory

Finally, create a new inventory source for the inventory. Fill in the fields and checkboxes as follows:

  • Name: My inventory source.
  • Source: Sourced from a project.
  • Project: My project.
  • Inventory File: inventory.yml.
  • Update Options: Select Overwrite, Overwrite Variables, and Update on Project Update.

Save the form and then click the Start sync process button for the new inventory source you just created. If the process finishes properly, your inventory’s HOSTS page will display the 2 example hosts, as proven in Figure 6.

The two hosts just created appear in the hosts list in the Ansible Tower inventory.

Figure 6: Viewing the HOSTS list in the Ansible Tower inventory.

Final thoughts

The inventory plugin we’ve created is basic, but it’s a fine foundation for implementing more complex ones that can query external sources of data, perhaps using third-party libraries. Being modules, inventory plugins can also accept parameters, giving them an advantage over plain scripts. For more information, see the official Ansible documentation on plugin configuration. Also, note that if you choose to use a third-party library not present in Python’s standard library, such as Requests, you will need to install it manually in the appropriate Python virtual environment inside Ansible Tower.

Happy developing!

.

Most Popular