The Operation Validation Framework: Test your infrastructure using Pester

Knowing if your IT infrastructure is operating as it should is a fundamental task for any IT administrator. Plethoras of tools and products are available in the market to accomplish this task. I want to talk about another option that is free, simple, and uses PowerShell and Pester to perform the heavy lifting. This simple module is called the Operation Validation Framework.

Author
Recent Posts

Brandon Olin

Brandon is a Cloud Architect and veteran Systems Engineer with a penchant for PowerShell, DevOps processes, and open-source software. You can reach Brandon at devblackops.io or on Twitter @devblackops.

Why Pester for infrastructure testing?

Developers typically use Pester to test that PowerShell scripts and modules operate as intended. Operation validation is similar, but instead of testing scripts and modules, we're testing the behavior of our real-world infrastructure.

Are the application's services running on this server? Do I have enough disk space? Is this TCP port responding? These are the kinds of tests I mean when I talk about operation validation. Pester can test these scenarios as well. It is a versatile framework and has some useful capabilities for infrastructure testing.

Keep everything in source control

Another benefit to using Pester as the basis for infrastructure testing is that you can store your tests in source control right alongside your application source code or configuration management artifacts.

It makes sense to keep everything about an application close together, including infrastructure tests. This helps ensure that as the application configuration changes over time, you can easily modify your infrastructure tests along with it, so everything stays in sync. This is immensely harder to do when using third-party monitoring systems that are not easily automatable, have no notion of configuration management, or are administered by another person or group.

Structure of an OVF module

OVF modules are just standard PowerShell modules with Pester tests in a defined folder structure. OVF is set up this way so it can discover infrastructure tests by inspecting the PowerShell modules installed on the system and searching for Pester tests under this folder structure. OVF will return any tests it finds, and you can then choose to execute all or some of them.

Let's look at a sample OVF module that will test the operation of an application called ACME. We'll call this module ACME.OVF. We store this module in a directory included in $env:PSModulePath so OVF can discover the module.

OVF\
- Diagnostics\
  - Comprehensive\
    - tests.ps1
  - Simple\
    - tests.ps1
    - tests.ps1
  - OVF.psd1

The only requirement to call this PowerShell module an OVF module is that it has a valid module manifest (.psd1) and a Diagnostics folder at the root of the module. This Diagnostics folder will then have Simple and Comprehensive subfolders. You'll store your Pester tests inside these subfolders.

You can use the Simple folder for tests that are quick and low impact to run. Think of these as smoke tests that validate the basic operation of the application. Comprehensive is where you'd put longer running tests that may impact the running system. Simple tests would run more frequently and Comprehensive ones less frequently.

OVF has no opinions on what other PowerShell functions or components are in this module. You could even have custom Desired State Configuration (DSC) resources to configure the application and a Diagnostics folder that contains the operation tests. DSC would be responsible for configuring the application, and OVF would be responsible for testing the operation of it—all in one module!

Example OVF tests

Let's take a closer look at the services and ports tests under the Simple directory. For this fictional ACME application, the Pester tests are validating common Windows services and ports. If this were a real application, you'd have these test services and ports specific to that application.

When we execute Pester against these files manually, you can see our normal Pester output, but instead of unit testing code, we're testing the operation of the system.

describe 'ACME Ports' {
    context 'Listening ports' {
        135, 445 | Foreach-Object {
            it "Port [$_] is listening" {
                $portListening = (Test-NetConnection -Port $_ -ComputerName localhost).TcpTestSucceeded
                $portListening | Should -Be $true
            }
        }
    }
}

Running TCP port test with Pester

param(
    $Services = @(
        'DHCP', 'DNSCache','Eventlog', 'PlugPlay', 'RpcSs', 'lanmanserver',
        'LmHosts', 'Lanmanworkstation', 'MpsSvc', 'WinRM'
    )
)

describe 'ACME Services' {
    context 'Service Availability' {
        $Services | ForEach-Object {
            it "[$_] should be running" {
                (Get-Service $_).Status | Should -Be running
            }
        }
    }
}

Running a services test with Pester

Test discovery

First off, we need to install OVF. In the PowerShell Gallery it is called OperationValidation, and you can install it with the below command.

Install-Module OperationValidation -Respository PSGallery -Scope CurentUser

Now that we have our Pester tests packaged into a module and installed in $env:PSModulePath and the OVF module itself installed from the PowerShell Gallery, let's use OVF to discover these tests. We'll use the Get-OperationValidation function included in the OperationValidation module.

We can include a few parameters to narrow down the tests returned. In this example, we'll specify the module to query and the test type, which in this case is simple. We can also specify a module version or tests with certain Pester tags applied, or even exclude tests based on tag.

Getting available tests from the ACME.OVF module using OperationValidation

Test execution

Now that we have a collection of tests, we can execute these with Invoke-OperationValidation. We also have the option of displaying the Pester output as well. Imagine having your monitoring system running the simple script below and throwing alerts if any of the Pester tests have failed.

Running OVF tests

Advanced tests

You may have noticed that in the services.tests.ps1 file above, we have a parameter called Services where we set the default collection of services to test.

param(
    $Services = @(
        'DHCP', 'DNSCache','Eventlog', 'PlugPlay', 'RpcSs', 'lanmanserver',
        'LmHosts', 'Lanmanworkstation', 'MpsSvc', 'WinRM'
    )
)

Pester and OVF can override these parameters at test execution time. This means you can write a generic OVF module that tests an application a certain way yet provides the ability to override parameters to modify how it executes the tests. This could be a useful way to tailor the tests for a development environment versus a production environment as an example.

Along with the tests packaged into PowerShell modules, this capability allows publishing generic OVF modules to the PowerShell Gallery.

Imagine dozens or hundreds of OVF modules out there that test common applications or operating system components. These would have default settings you could override. Just download the module and execute it, overriding parameters as necessary.

The command below is a simple example of overriding parameters in the test. Instead of testing the default collection of services, we're overriding the parameter and testing the print spooler service.

Subscribe to 4sysops newsletter!

$simpleTests | Where-Object ($_.Name -eq 'ACME Services') | Invoke-OperationValidation ‑Overrides @(Services = 'Spooler')

Running an OVF test and overriding test parameters

Conclusion

Using Pester and the Operation Validation Framework is a simple, free, and effective way to test your infrastructure. As DevOps practices grow in popularity with traditional IT administrators, it's becoming the standard to define everything in code and automate critical tasks. We test our code—shouldn't we be testing our infrastructure too?