Red Hat Compliance service and the Red Hat Insights API

In an earlier blog I walked you through the process of using the Red Hat Enterprise Linux (RHEL) and Red Hat Insights Compliance service to:

• Create compliance policies

• Prepare your hosts for the compliance service

• Assign hosts to a compliance policy

• Run a compliance scan

• View a compliance report

The Red Hat Insights Compliance service, which works with OpenSCAP, is a significant step to managing your regulatory compliance requirements at scale. The steps I demonstrated can be done through the UI without much difficulty. But what if you could automate the process of assigning hosts to specific compliance policies and running compliance scans during the host provisioning process?

This blog takes you through the steps to add the compliance policy to your overall Standard Operating Environment (SOE) process. Doing this step as part of your SOE will allow your deployed hosts to be assigned to their targeted compliance policies.

Using the Red Hat Ansible Automation Platform I will demonstrate how to:

• Create an Ansible Automation Platform Workflow that uses an Ansible Smart Inventory as part of the process

• Create an Ansible Playbook that uses the Insight API to add your hosts to an existing compliance policy

Overall scenario

Before getting into the details of the process, I want to make sure that I set up the higher level scenario. In my fictional organization we have three types of regulatory compliance policies that we must adhere to:

• PCI-DSS v3.2.1 Control Baseline for Red Hat Enterprise Linux 8

• Health Insurance Portability and Accountability Act (HIPAA)

• Protection Profile for General Purpose Operating Systems

The diagram below depicts the policies that I have set up in the Insights Compliance service within the Hybrid Cloud Console.

My goal with this blog is to demonstrate how to assign a host to one of these policies through an automation process using the Red Hat Insights API.

Provisioning process

When I provision a host or hosts into my environment using Ansible Automation Platform one of the options in my provisioning process is to select the Compliance profile to which this host (or set of hosts) should be assigned.

To accomplish this I use the survey functions within Ansible Automation Platform as described below.

Selecting one of these compliance options will set the compliance_type variable which is used in my playbook.

Specifically, to identify which compliance policy this host (or set of hosts) should be assigned, I place each host in a group that can then be used by an AAP Inventory.

This has the result of creating a group in an inventory in the Ansible Automation Platform (AAP) that can be used by any job template.

The screenshot below shows three groups created in my inventory specifically for compliance reasons.

Playbook overview

The playbook I created consists of two plays. One play operates against the entire inventory while the second play is performed strictly by localhost.

Play 1: Create host file for Insights API

The goal of this play is to create a list of specific hosts that can be passed to the Red Hat Insights API.

Let’s break each piece of the playbook down prior to showing the entire playbook

The first step is to pull the hosts into the playbook run. The key point to note here is "hosts=all". In order for this to work in the AAP Controller, your job template will need to be configured to target the correct inventory.

- name: capture hosts adding to compliance policies hosts: all
gather_facts: yes
remote_user: cloud-user

Once the hosts are captured in the playbook run, I created a series of files that parsed the hosts into separate files based on their compliance type label. This was done using a jinja2 template.

The template shown below creates a file that contains the hosts from the inventory in a format that can be manipulated by subsequent plays.

Template:

{% for host in groups[ ‘label_compliance_policy_standard’ ] %} {{ hostvars[host][‘ansible_hostname’] }}.dumont-lab.lan {% endfor %}

Playbook:

- name: create standard policy host list template: src: templates/STD_host_file_template.j2 dest: files/std_compliance_hosts delegate_to: localhost when: compliance_type == ‘label_compliance_policy_standard’

Please note that this task is delegated to localhost.

The output of this task is stored in files/std_compliance_hosts and is formatted as a list of hosts as below.

vm-2022-03-01-20-23-04-1.dumont-lab.lan vm-2022-03-01-01-24-06-1.dumont-lab.lan

**
Play 2 - Assign hosts to compliance profile**

Up to this point, the workflow gathered all of the information required to make calls to the Red Hat Insights API to assign hosts to the correct Compliance profile.

This final Job Template consists of a single playbook that will use the Red Hat Insights API to update (PATCH) the desired compliance profile with the list of hosts that need to be applied to this policy.

This playbook will call the “patch compliance profile api” to apply the hosts listed in the file above to the Compliance profile called "Protection Profile for General Purpose Operating Systems".

To accomplish this, two key pieces of information are required:

• The profile id of the compliance profile - in this example “Protection Profile for General Purpose Operating Systems”

• The UUIDs of each of the hosts that will be applied to this compliance profile

Once that information is known, “patch compliance profile api” call will need to contain a properly formatted body in JSON format.

The format of that body that is shown below:

{ "data": { "relationships": { "hosts": { "data": [ { "id": "<host UUID 1>", "type": “host” }, { "id": "<host UUID 2>", "type": “host” }, … ] } } } }

To accomplish the above, I will breakdown the tasks in my playbook as follows:

• Obtain an access token for the playbook run

• Call the Red Hat Insights API to pull the Compliance profiles

• Extract a specific Policy ID from the Compliance profiles using a json_query

• Call the Insights API to pull the Host list

• Extract the hosts UUIDs using a json_query

• Use a jinja2 template to create the body for the Red Hat Insights API call to update the Compliance profile

• Call the Red Hat Insights API to apply the Hosts to the Compliance profile

Please note three playbook level attributes have been set:

• This playbook is set up to run on the local host using “hosts: localhost”

• We will not be gathering facts for the localhost as it is unnecessary, so “gather_facts: no”

• For this demonstration playbook I have set a variable in the AAP Job Template to the specific profile name that I want to apply to my hosts. In this example I set the variable

profile_name: ‘Protection Profile for General Purpose Operating Systems’

Obtain an access token

Red Hat APIs use OAuth 2.0 for authorization. To obtain a token and access the APIs, you will need the following pieces of information:

• Offline token generated on the API Tokens page

• Client ID = rhsm-api

• Token URL = https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token

• Installed jq to process JSON objects:

  • jq is a command-line JSON processor that can be installed using the yum command: sudo yum install jq

This process is documented in the Getting started with Red Hat APIs knowledgebase article.

A sample playbook for this process is provided below.

- name: get access token uri: url: “https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token” method: POST return_content: True body_format: form-urlencoded headers: accept: application/json body: grant_type: refresh_token client_id: rhsm-api refresh_token: “{{ my_offline_token }}” validate_certs: False

Call the Red Hat Insights API to pull the Compliance profiles

The specific Red Hat Insights API call I used to pull a list of all the compliance profiles in my account is located here.

The code used to pull the list of profiles is shown below. Please note that a login to your account in the Hybrid Cloud Console is required.

- name: Get Compliance List-URI uri:
url: https://console.redhat.com/api/compliance/profiles method: GET
headers:
Content-Type: “application/json” accept: application/json Authorization: “Bearer {{ token }}” body_format: form-urlencoded status_code: 200
return_content: yes validate_certs: false force_basic_auth: yes register: profiles

Extract a specific Policy ID from the Compliance profiles using a json_query

The Red Hat Insight API call used to PATCH the compliance profile requires the profile_id.

Now that I have all of the details of the compliance profiles in a register variable, I can create an Ansible fact and then parse that Ansible fact with a JSON query to store the profile_id that I will need.

- name: set profile list set_fact: profile_list: “{{ profiles.json }}”

- name: profile_id set_fact: profile_id: “{{ profile_list | json_query(get_profile_id) }}”
vars: get_profile_id: “data[?attributes.name == ‘{{ profile_name }}’].id”

The stanza above creates the profile_id variable by piping the profile_list through a json_query using the get_profile_id variable.

The get_profile_id variable (when applied to the profile list will search the data object for an attribute.name that matches the variable set in the profile_name and captures the profile_id

Call the Red Hat Insights API to pull the Host list and create the UUIDs

The Red Hat Insights API uses the UUIDs of the hosts as the unique identifier to determine which hosts are applied to which compliance profile. Therefore, to assign the hosts to a compliance profile, the UUID of each of the hosts is required.

The first step in this process is to pull the list of hosts that is contained in my Red Hat Insights Inventory.

The specific Red Hat Insights API call I used to pull a list of all the hosts in my account is located here.

- name: Get Host List uri: url: https://cloud.redhat.com/api/inventory/v1/hosts method: GET
headers: Content-Type: “application/json” accept: application/json Authorization: “Bearer {{ token }}” body_format: form-urlencoded status_code: 200 validate_certs: false force_basic_auth: yes return_content: yes register: host_list

Now that the entire list of hosts that Red Hat Insights has in its inventory is included in the playbook, I set the host_data variable using Ansible facts.

- name: host_data fact set_fact: host_data: “{{ host_list.json }}”

The next step in the play leverages the file (files/std_compliance_hosts) created in the previous play to set the file_contents fact.

- name: set file_contents fact set_fact: file_contents: “{{ item }}” with_file: - “files/std_compliance_hosts”

An important point to note is that the file_contents variable will be created with ‘\n’ newline delimiter as follows:

'host-1.dumont-lab.lan\nhost-2.dumont-lab.lan\nhost-3.dumont-lab.lan`

Finally, the play extracts UUIDs only for the hosts that will be added to the compliance profile I loop through the host list contained in the file_contents variable as follows:

• Set the uuid fact by piping the host_data fact through a json_query

• The get_uuid fact when looped using the file_contents fact scans the results object searching for the fqdn of the hosts contained in the file contents variable.

• Notice that the loop pipes the file_contents variable through the jinja2 split filter using the ‘\n’ delimiter

- name: get uuid of host set_fact: uuid: “{{ uuid + [ host_data | json_query(get_uuid)] }}” vars: uuid: [ ] get_uuid: “results[?fqdn == ‘{{ item }}’].id” loop: “{{ file_contents | split(‘\n’) }}”

Use a jinja2 template to create the body for the Insights API call to update Insights

We are now ready to build the body for the Red Hat Insights API call that will update (PATCH) the Compliance profile. This body must be in a very specific format according to the Red Hat Insights API as documented here.

In addition to the static content contained the in documentation, the body of the API call will use the two variables I created in the previous steps:

• profile_id — the profile id of the compliance profile being updated

• uuid — the UUID of each of the hosts that are need to be assigned to the compliance policy

To build this body I used the jinja2 template identified below. This template does the following:

• Adds the profile_id created earlier in the playbook to the body of the API call

• Uses the jinja2 “for” loop to add the uuid of the hosts contained in the uuid variable created earlier

{ "data":{ "id": "{{ profile_id[0] }}", "relationships": { "hosts":{ "data":[ {% for var in uuid %} { "id": "{{ var[0] }}", "type": “host” } {% endfor %} ] } } } }

This template is called using the template module and creates a file that I will use in the API call.

- name: create lookup file template: src=templates/compliance_host.j2 dest=compliance_host.json mode=0644

The resulting JSON file looks like this:

{ "data":{ "id": "52f66bfd-e5c6-464d-9542-6d6edd6ed7e6", "relationships": { "hosts":{ "data":[ { "id": "a5dbc59c-07bc-4260-a786-8bf9628a0b46", "type": “host” }, { "id": "6491e323-c671-4052-bbcb-18b4dd87a4e6", "type": “host” }, ] } } } }

Call the Red Hat Insights API to apply the Hosts to the Compliance Profile

The last step in the process is to call the Red Hat Insights API and update or PATCH the Compliance profile using the JSON body created in the compliance_host.json file.

• Calls the Insights API with a specific profile_id

• The method: PATCH indicates that the JSON object should be updated

• Use the lookup plugin with the file argument to add the body formatted exactly how you wanted it formatted as setup in the compliance_host.json file

- name: Update Compliance Host List uri:
url: “https://cloud.redhat.com/api/compliance/profiles/{{ profile_id[0] }}” method: PATCH
headers:
Content-Type: “application/json” accept: application/json Authorization: “Bearer {{ token }}”
body_format: json
body: “{{ lookup(‘file’,’compliance_host.json’) }}” status_code: 200
return_content: yes
validate_certs: false
force_basic_auth: yes
register: new_profile

Validation

To validate that the Compliance profile now has the desired hosts assigned, login to the Hybrid Cloud Console and click to Compliance -> SCAP policies -> Protection Profile for General Purpose Operating Systems.

This list will contain the hosts that were tagged with label_compliance_policy_standard during the provisioning process.

Compliance scanning

Now that your hosts are assigned to the desired compliance policies the next step is to determine how aligned your hosts are to the assigned policy.

The most direct way to determine this is to run a compliance scan against the targeted hosts. The playbook below is what I use to determine my host’s adherence to the compliance policy.

—

• name: run insights compliance scan hosts: all become: true

  tasks:

  • name: run insights compliance import_role: name: redhat.insights.compliance tasks_from: run

Once you run the compliance scan a Compliance report will be generated and made available in the Hybrid Cloud Console as depicted below.

As you can see my hosts have a compliance score of 40%.

Please join me next time as I show you the best way to remediate the failed rules using Red Hat tooling.

Code

The code contained in this document is available in my github repository at: https://github.com/bdumont01/insights_api_project

Closing

The goal of this blog was to show you the details of how you can use the Red Hat Insights API in combination with Red Hat Ansible Automation Platform to automate the process of putting your hosts into the desired compliance policy.

If you are not already using Insights, give it a try! Go to the Getting Started page to learn more.

I encourage you to engage with me in discussing this approach that I have described in this blog.