Lab 8: Config Remediation

Scenario

It’s important for a system’s configuration to comply with industry standards, regulatory requirements, and organizational policies. However, configuration drift can happen when there are manual updates or unexpected modifications over time. The IT manager and compliance team want to enforce baseline configurations that can automatically be applied to all virtual machines when drift occurs. You’ll learn how to achieve automatic configuration remediation using Automation Config.

Task A. Baseline Config - Hosts File

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/config-management/linux/base-config/hosts.sls
4. Click hosts.sls and examine its SaltStack State File YAML

   {% if grains.os == 'Ubuntu' %}
   
   {% set IP_ADDRESS = salt.grains.get('ip4_interfaces:ens160:0', 'NOTFOUND') %}
   {% set FQDN = grains.fqdn %}
   {% set HOST = grains.host %}
   
   file_hosts:
       file.managed:
           - name: /etc/hosts
           - contents:
               - '### Warning: The contents of this file is centrally managed by Aria Automation Config ###'
               - '127.0.0.1 localhost'
               - '127.0.1.1 {{ FQDN }} {{ HOST }}'
               - '{{ IP_ADDRESS }} {{ FQDN }} {{ HOST }}'
           - user: root
           - group: root
   
   {% endif %}
   

5. Running this state file will overwrite /etc/hosts file with above contents on the target minions, but we are NOT going to run it now. We will apply this state collectively as automatic configuration remediation

Task B. Baseline Config - OpenSSH Configuraiton

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/config-management/linux/base-config/sshd.sls
4. Click sshd.sls and examine its SaltStack State File YAML

   {% if grains.os == 'Ubuntu' %}
   
   configure_sshd:
       file.managed:
       - name: /etc/ssh/sshd_config
       - source: salt://POV/config-management/linux/base-config/sshd_config
       - makedirs: True
   
   reload_sshd:
       service.running:
       - name: sshd
       - enable: True
       - reload: True
       - watch:
           - configure_sshd
   
   {% endif %}
   

5. Running this state file will overwrite OpenSSH configuration file on the target minions with /POV/config-management/linux/base-config/sshd_config in the file server, but we are NOT going to run it now. We will apply this state collectively as automatic configuration remediation

Task C. Baseline Config - Enable Open VM Tools Service

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/config-management/linux/base-config/vmtools.sls
4. Click vmtools.sls and examine its SaltStack State File YAML

   {% if grains.os == 'Ubuntu' %}
   
   manage_vmtools:
       service.running:
           - name: open-vm-tools
           - enable: True
           - reload: True
   
   {% endif %}
   

5. Running this state file will enable Open VM Tools service on the target minions, but we are NOT going to run it now. We will apply this state collectively as automatic configuration remediation

Task D. Create Baseline Configuration (Optional)

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/config-management/linux/base-config/main.sls
4. Click main.sls and examine its SaltStack State File YAML

   include:
       - pov.config-management.linux.base-config.hosts.sls
       - pov.config-management.linux.base-config.sshd.sls
       - pov.config-management.linux.base-config.users.sls
       - pov.config-management.linux.base-config.vmtools.sls
   

5. Running this state file will apply all above states collectively as a baseline configuration, but we are NOT going to run it now. Instead, we will create beacon to detect drifts and define reactors to trigger individual state file execution once subscribed event is triggered

Task E. Create Beacon Config - Subscribe Events

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/drift-remediation/linux/config-remediation/files/beacons.conf
4. Click beacons.conf and examine its SaltStack State File YAML

   beacons:
       files_to_watch:
           - files:
               /etc/hosts:
                   mask:
                       - modify
                       - delete
                       - delete_self
                       - moved_from
   
               /etc/ssh/sshd_config:
                   mask:
                       - modify
                       - delete
                       - delete_self
                       - moved_from
   
           - beacon_module: inotify
           - disable_during_state_run: True
   
       service:
           - services:
               open-vm-tools:
                   onchangeonly: True
   

5. Locate /POV/drift-remediation/linux/config-remediation/deploy_beacon.sls
6. Click deploy_beacon.sls and examine its SaltStack State File YAML

   install_pip3_package:
       pkg.installed:
           - name: python3-pip
   
   install_inotify_prerequsite:
       pip.installed:
           - name: pyinotify
   
   configure_beacon:
       file.managed:
           - name: /etc/salt/minion.d/beacons.conf
           - source: salt://POV/drift-remediation/linux/config-remediation/files/beacons.conf
           - makedirs: True
   
   # # CentOS 8 may not restart by the usual method
   # restart_salt-minion-for-CentOS-8:
   #     service.running:
   #         - name: salt-minion
   #         - enable: True
   #         - watch:
   #             - file: /etc/salt/minion.d/beacons.conf
   
   restart_salt-minion:
       service.running:
           - name: salt-minion
           - enable: True
           - watch:
               - configure_beacon
   

7. Running this state file will deploy the beacons.conf, install dependencies and restart the salt-minion service
8. Next, we will create a job to run this state file

Task F. Create Job - Deploy Beacon for Linux

1. Click Configurations > Jobs in the side menu
2. Click Create Job button
3. Input the following details for the new job and save the job
   • Name = Deploy Beacon for Linux
   • Description = Beacon that detects hosts file, sshd config & open-vm-tools service drift
   • Command = salt
   • Targets = Ubuntu
   • Function = state.apply
   • Environments = base
   • States = POV.drift-remediation.linux.config-remediation.deploy_beacon
   • Arguments = (Blank)
   • Job Inputs = (Blank)
4. Click SAVE button
5. Now that you’ve created a job

Task G. Run Job - Deploy Beacon for Linux

1. In the side menu, click Targets to open the Targets workspace and view your targets
2. Open Linux minions target
3. Locate the first Ubuntu server, click the button besides the first column, and select Run job
4. In the Run Job dialog box
   • Input the following details
     • Job = Deploy Beacon for Linux (the job you have created in the previous task)
     • States = (Unchanged)
     • Run as Test = false
     • Environments = base
     • Options = (Unchanged)
5. Click RUN NOW button
6. Click Activity > Completed in the side menu
7. The job will appear on the list once it is done. If not, Refresh the page in your browser
8. Check if this job is completed successfully without error
   • Check Success is 1 or higher, and Failed is 0

Task H. Create Reactor Config - Define Remediations

1. Click Configurations > File Server from the side menu
2. Expand base folder in the file server folder tree
3. Locate /POV/drift-remediation/linux/config-remediation/files/reactor.conf
4. Click reactor.conf and examine its SaltStack State File YAML

   reactor:
       - 'salt/beacon/*/files_to_watch//etc/hosts':
           - salt://srv/reactor/remediate_hosts.sls
       - 'salt/beacon/*/files_to_watch//etc/ssh/sshd_config':
           - salt://srv/reactor/remediate_sshd.sls
       - 'salt/beacon/*/service/open-vm-tools':
           - salt://srv/reactor/remediate_vmtools.sls
   

5. Locate /POV/drift-remediation/linux/config-remediation/deploy_reactor.sls
6. Click deploy_reactor.sls and examine its SaltStack State File YAML

   configure_reactor.conf:
       file.managed:
           - name: /etc/salt/master.d/reactor.conf
           - source: salt://POV/drift-remediation/linux/config-remediation/files/reactors.conf
           - makedirs: True
   
   configure_sshd_remdediation:
       file.managed:
           - name: /srv/reactor/remediate_sshd.sls
           - source: salt://POV/drift-remediation/linux/config-remediation/files/remediate_sshd.sls
           - makedirs: True
   
   configure_host_remdediation:
       file.managed:
           - name: /srv/reactor/remediate_hosts.sls
           - source: salt://POV/drift-remediation/linux/config-remediation/files/remediate_hosts.sls
           - makedirs: True
   
   configure_vmtools_remdediation:
       file.managed:
           - name: /srv/reactor/remediate_vmtools.sls
           - source: salt://POV/drift-remediation/linux/config-remediation/files/remediate_vmtools.sls
           - makedirs: True
   
   restart_salt_master_service:
       service.running:
           - name: salt-master
           - watch:
               - configure_reactor.conf
   

7. Running this state file will deploy the reactor.conf and remediation state files to the Salt Master
8. Next, we will create a job to run this state file

Task I. Create Job - Deploy Reactor for Salt Master

1. Click Configurations > Jobs in the side menu
2. Click Create Job button
3. Input the following details for the new job and save the job
   • Name = Deploy Reactor for Salt Master
   • Description = Reactor that remediate hosts file, sshd config & open-vm-tools service drift
   • Command = salt
   • Targets = Linux
   • Function = state.apply
   • Environments = base
   • States = POV.drift-remediation.linux.config-remediation.deploy_reactor
   • Arguments = (Blank)
   • Job Inputs = (Blank)
4. Click SAVE button
5. Now that you’ve created a job

Task J. Run Job - Deploy Reactor for Salt Master

1. In the side menu, click Targets to open the Targets workspace and view your targets
2. Open Linux minions target
3. Locate the saltmaster server, click the button besides the first column, and select Run job
4. In the Run Job dialog box
   • Input the following details
     • Job = Deploy Reactor for Salt Master (the job you have created in the previous task)
     • States = (Unchanged)
     • Run as Test = false
     • Environments = base
     • Options = (Unchanged)
5. Click RUN NOW button
6. Click Activity > Completed in the side menu
7. The job will appear on the list once it is done. If not, Refresh the page in your browser
8. Check if this job is completed successfully without error
   • Check Success is 1 or higher, and Failed is 0

Task K. Check Results - Baseline Configuration

1. Log on to one of the target Linux minions
2. Check if /etc/hosts will be rolled back to baseline configuration after modification
3. Check if SSHD Configuration will be rolled back to baseline configuration after modification
4. Check if Open VM Tools will be re-enabled after disabling it

Summary

Automation Config can automatically apply baseline configuration upon configuration drift:

• Automation Config’s Event-Driven Automation technology provides a light-weight and effortless solution to remediate configuration drift to enforce continuous baseline configuration
• Beacon is used to listen for events at target minions, such as file, config or service status changes
• Triggered events will be sent to the Salt Master for further processing
• Reactor is to be deployed to Salt Master, its configuration will determine what remediation actions to be taken
• Remediation state files are securely stored in Salt Master
• Alternatively, remediation actions could be pointed to a State File in the File Server using salt:// path