Monitor network connections and listening ports with SCOM

The following article briefly explain the components of a SCOM management tool I published on GitHub that allows you to monitor network connections and listening ports with the help of netstat and PowerShell.

Author
Recent Posts

Ruben Zimmermann

Ruben is an infrastructure specialist with strong backround in Active Directory, PKI, and System Center Operations Manager. He automates in VBS, PowerShell and C#.
You can follow him on Twitter @Ruben8Z.

Latest posts by Ruben Zimmermann (see all)

Monitoring Microsoft 365 with SCOM and the NiCE Active 365 Management Pack - Tue, Feb 7 2023
SCOM.Addons.MailIn: Monitor anything that can send email with SCOM - Mon, May 25 2020
Display a user’s logged-on computer in Active Directory Users and Computers (ADUC) - Mon, Jan 21 2019

System Center Operation Manager (SCOM) TCP port monitoring allows you to identify whether a port on a target machine responds. However, cases exist where you don't want to or can't establish a connection to the remote machine. The management pack discussed in this post uses netstat and PowerShell to monitor active connections and listening ports.

Diagram view showing monitored listening ports and TCP connections

To ensure the management pack also runs on Windows Server 2008 R2, it is compatible with PowerShell version 2.

SCOM connections and ports discovery

SCOM first has to discover the connections or ports that need monitoring. The file named monitoredTcpConnects.csv stores the connection and has this structure:

remoteIP,remoteName,remotePort,procName,comment
10.1.11.83,,80,CcmExec,sccm
,linvmas146,5723,HealthService
194.69.46.72,,40936,powershell

It needs to store the ports in a file named monitoredListeningPorts.csv:

ipProtocol,localIP,localPort,procName,comment
udp,127.0.0.1,49740,dfsrs
udp,,161,snmp
tcp,,10115,endpoint,perfdata

Listing connections and listening ports

Running netstat -ano lists all established connections and listening ports including the process identification number (PID):

Listing all connections and listening ports with netstat

The PowerShell function below runs netstat, stores the result in a file, and then converts the file into a list of objects for further processing. A parameter decides whether the list will contain "listening port objects" or "established connection objects." Note that piping the output of netstat ‑ano into a file that it then reads is faster than directly storing the result in a variable.

#Retrieving computer name and IP addresses for later use
$localComputerName   = $env:COMPUTERNAME
$localIPAddresses    = ([System.Net.Dns]::GetHostAddresses($localComputerName)) | Where-Object { $_.AddressFamily -eq 'interNetwork' } | `
											 Select-Object -ExpandProperty IPAddressToString 
Function Format-NetstatData {
	param(
[Parameter(Mandatory=$true)][object]$netstatInPut,
		[Parameter(Mandatory=$true)][string]$qryType,				
		[Parameter(Mandatory=$true)][ref]$nestatIPData 
	)
	#Retrieving all processes to map PID in netstat to executable name
	$allProcesses    = Get-Process | Select-Object -Property Name, id
	$netStatConnects = New-Object -TypeName System.Collections.Generic.List[object]
	$netStatArr      =  $netstatInPut -split "`r`n"						
	$netStatArr | ForEach-Object {
		$netStatItm = $_
		if ($netStatItm -match "\d") {       
			#Split the line by using 'more than 2 white spaces' as delimitation
			$netStatItmParts = [Regex]::Split($netStatItm,"\s{2,}")							
			if ($qryType -eq 'tcpConnection') {
			
				$proto           = $netStatItmParts[1]					
				$localIP         = ($netStatItmParts[2] -split ':')[0]
				$localPort       = ($netStatItmParts[2] -split ':')[1]					
				$remoteIP        = ($netStatItmParts[3] -split ':')[0]
				$remotePort      = ($netStatItmParts[3] -split ':')[1]
				$connectState    = $netStatItmParts[4]
				$procId          = $netStatItmParts[5]
				
				$procInfo        = $allProcesses | Where-Object { $_.id -eq $procId }
				$procName        = $procInfo.Name				
				if ($localIPAddresses -contains $localIP) {
					$localName = $localComputerName
				}					
			
				#Filtering records to only contain connections to remote systems
				if (($localIp -match $regIpPat -and $remoteIp -match $regIpPat) -and ($remoteIP -notmatch '0.0.0.0|127.0.0.1') ) {
					$myNetHsh = @{'proto' = $proto}
					$myNetHsh.Add('localIP', $localIP)
					$myNetHsh.Add('localName', $localName)						
					$myNetHsh.Add('remoteIP', $remoteIP)
					$myNetHsh.Add('remotePort', $remotePort)
					$myNetHsh.Add('connectState', $connectState)
					$myNetHsh.Add('procId', $procId)
					$myNetHsh.Add('procName', $procName)
					
					$myNetObj = New-Object -TypeName PSObject -Property $myNetHsh
					$null     = $netStatConnects.Add($myNetObj)    
				}
			} else {
				$proto               = $netStatItmParts[1]		
				if ($proto -ieq 'TCP') {
					$localIP         = ($netStatItmParts[2] -split ':')[0]
					$localPort       = ($netStatItmParts[2] -split ':')[1]					
					$remoteIP        = ($netStatItmParts[3] -split ':')[0]
					$remotePort      = ($netStatItmParts[3] -split ':')[1]
					$connectState    = $netStatItmParts[4]
					$procId          = $netStatItmParts[5]					
				} else {
					$localIP         = ($netStatItmParts[2] -split ':')[0]
					$localPort       = ($netStatItmParts[2] -split ':')[1]					
					$remoteIP        = ($netStatItmParts[3] -split ':')[0]
					$remotePort      = ($netStatItmParts[3] -split ':')[1]
					$connectState    = '-'
					$procId          = $netStatItmParts[4]					
				}				
							
				$procInfo = $allProcesses | Where-Object { $_.id -eq $procId }
				$procName = $procInfo.Name					
				if ($localIPAddresses -contains $localIP) {
					$localName = $localComputerName
				}					
				if (($localIp -match $regIpPat) -and ($remoteIP -match '\*|0.0.0.0|127.0.0.1') ) {
					$myNetHsh = @{'proto' = $proto}
					$myNetHsh.Add('localIP', $localIP)
					$myNetHsh.Add('localName', $localName)												
					$myNetHsh.Add('localPort', $localPort)						
					$myNetHsh.Add('connectState', $connectState)
					$myNetHsh.Add('procId', $procId)
					$myNetHsh.Add('procName', $procName)
					
					$myNetObj = New-Object -TypeName PSObject -Property $myNetHsh
					$null     = $netStatConnects.Add($myNetObj)    
				} 
			} # END if ($qryType -eq 'tcpConnect')										
		} #END if ($netStatItm -match "\d") 
	} #END $netStatIpArr | ForEach-Object {} 
	If ($netStatConnects.count -gt 0) {
		$rtn = $true
		$nestatIPData.Value = $netStatConnects
	} else {
		$rtn = $false	
	}
	$rtn
} #END Function Format-NetstatIPData		
Invoke-Expression "C:\Windows\System32\netstat.exe -ano" | Out-File -FilePath $netStatIpFile
$netStatIp = Get-Content -Path $netStatIpFile | Out-String		
$netStatIPConnects = New-Object -TypeName System.Collections.Generic.List[object]
Format-NetstatData -netstatInPut $netStatIp -qryType $discoveryItem -nestatIPData ([ref]$netStatIPConnects)

Interpreting output and initiating reaction

To check whether a defined connection is active or a port is listening, we compare "should and is." I only show the script for the connections below because the code for listening ports is very similar. You can download the entire solution from GitHub.

<insert script 2>if($MonitorItem -eq 'tcpConnection') {
	$monitoredTcpConnectsFilePath = $filePath + '\' + 'monitoredTcpConnects.csv'
	
	if (Test-Path -Path $monitoredTcpConnectsFilePath) {				
				
		$monitoredTcpConnects = Import-Csv -Path $monitoredTcpConnectsFilePath		
		
		#Working through all connections mentioned in the file
		foreach ($tcpConnect in $monitoredTcpConnects) {
			$remoteIP        = ''
			$remoteName      = ''
			$remotePort      = ''
			$comment         = ''
			$procName        = ''
			$connectDetails  = ''
			$connectionState = ''
			$remoteIP        = $tcpConnect.remoteIP
			$remoteName      = $tcpConnect.remoteName
			$remotePort      = $tcpConnect.remotePort
			$comment         = $tcpConnect.comment
			$procName        = $tcpConnect.procName			
			
			#Resolving remote IP if remote name was mentioned			
			if ($remoteName -and ([String]::IsNullOrEmpty($remoteIP))) {
				$remoteIP = [system.net.dns]::Resolve($remoteName).AddressList | Where-Object { $_.AddressFamily -eq 'interNetwork' } | Select-Object -ExpandProperty IPAddressToString
			} 
			
			if ($remotePort -and $remoteIP) {	
				
				#Checking if the mentioned connection is currently active plus retrieving additional information
				$connectDetails = $netStatIPConnects | Where-Object { $_.remotePort -eq $remotePort -and $_.remoteIP -eq $remoteIP }							
				
				#If connection is not active sending back 'Red' which will be interpreted as critical alert
				if ([string]::IsNullOrEmpty($connectDetails) -or [string]::IsNullOrWhiteSpace($connectDetails)) {
					
					$localIP         = $localIPAddresses										
					$Key             = "tcpConnectOn$($localComputerName)For$($procName)To$($remoteIP):$($remotePort)"
					$connectionState = 'No active connection found.'									
				
					$state           = 'Red'					
					$localPort       = 'NA'										
					$supplement      = "localIP: $($localIP)`t localPort: $($localPort)`n procName: $($procName)`n ConnecionState: $($connectionState)`n"
					$supplement     += "remoteIP: $($remoteIP)`t remotePort: $($remotePort)`n"								
					
					#A 'property bag' is sent back to inform SCOM of the state of the particular object						
					$bag = $api.CreatePropertybag()								
					$bag.AddValue("Key",$key)		
					$bag.AddValue("State",$state)				
					$bag.AddValue("Supplement",$supplement)		
					$bag.AddValue("TestedAt",$testedAt)			
					$bag	
					continue
					
				} #END if ([string]::IsNullOrEmpty($connectDetails) -or [string]::IsNullOrWhiteSpace($connectDetails))							
				
				#Looping through the mentioned active connections 
				foreach ($connDetail in $connectDetails) {								
					$connectionState = ''
					$supplement      = ''
					$localIP         = $connDetail.localIP															
					#Resolve hostname if only IP is available
					if ([String]::IsNullOrEmpty($remoteName)) {
						$tmpName = [system.net.dns]::Resolve($remoteIP).HostName
						if ($tmpName -ne $remoteIP) {
							$tmpName    = $tmpName -replace $localComputerDomain,''
							$tmpName    = $tmpName -replace '\.',''
							$remoteName = $tmpName          
						} else {
							$remoteName = 'No reverse record in DNS.'
						}
					}
					
					#Resolve hostname if hostname is an IP
					if ($remoteName -match '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}') {
						$tmpName = [system.net.dns]::Resolve($remoteName).HostName					
						if ($tmpName -ne $remoteIP) {
							$tmpName    = $tmpName -replace $localComputerDomain,''
							$tmpName    = $tmpName -replace '\.',''
							$remoteName = $tmpName
						} else {
							$remoteName = 'No reverse record in DNS.'
						}
					}
					
					$Key             = "tcpConnectOn$($localComputerName)For$($procName)To$($remoteIP):$($remotePort)"
					$connectionState = $connDetail.connectState
					$supplement      = "localIP: $($localIP)`t  `n procName: $($procName)`t `n ConnecionState: $($connectionState)`n"
					$supplement     += "remoteIP: $($remoteIP)`t remotePort: $($remotePort)`n"						
					#If the connection is ESTABLISHED, 'green' is healthy state, if it is 'TIME_WAIT' then 'yellow' indicates a warning state, 
					#If none of the above applies the connection state is most likely CLOSE_WAIT where 'Red' returns critical state
					if ($connectionState -eq 'ESTABLISHED') {
						$state       = 'Green'						
					} elseif ($connectionState -eq 'TIME_WAIT') {						
						$state       = 'Yellow'
						$supplement += 'TIME_WAIT = Local endpoint (this computer) has closed the connection.'
					} else {
						$state       = 'Red'
						$supplement += 'CLOSE_WAIT = Remote endpoint (this computer) has closed the connection.'
					}																						
					
					#A 'property bag' is sent back to inform SCOM about the state of the particular object						
					$bag = $api.CreatePropertybag()								
					$bag.AddValue("Key",$key)		
					$bag.AddValue("State",$state)				
					$bag.AddValue("Supplement",$supplement)		
					$bag.AddValue("TestedAt",$testedAt)			
					$bag										
						
				} #END foreach ($connDetail in $connectDetails)
						
			} else {
				$foo = 'No details this time, not sending to inventory.'
			} # END	if ($connectDetails)					
		
		} #END foreach($tcpConnect in $monitoredTcpConnects)

	} else {
		$api.LogScriptEvent('Monitor NetStatWatcher Three State.ps1',3002,1,"NetStatWatcherMon MonitorItem $($MonitorItem) - File not found in $($monitoredTcpConnectsFilePath)")
	}
}

Management pack components

Classes

Everything in SCOM that has a health state is an object. Instead of checking all Windows computers whether those files exist, we define a dedicated computer class.

<ClassType ID="Network.Windows.Computer.NetstatWatcher.Computer" Accessibility="Public" Abstract="false" Base="Windows!Microsoft.Windows.ComputerRole" Hosted="true" Singleton="false" Extension="false">
  <Property ID="FilePath" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
  <Property ID="NodeName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
</ClassType>

We also require a class for TCP connections and listening ports:

<ClassType ID="Network.Windows.Computer.NetstatWatcher.TcpConnection" Accessibility="Public" Abstract="false" Base="System!System.LogicalEntity" Hosted="false" Singleton="false" Extension="false">
    <Property ID="ComputerName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="Key" Type="string" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="localIP" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="localName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="remoteIP" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="remoteName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="remotePort" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="procName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="512" MinLength="0" Required="false" Scale="0" />
    <Property ID="comment" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="1024" MinLength="0" Required="false" Scale="0" />
</ClassType>
<ClassType ID="Network.Windows.Computer.NetstatWatcher.ListeningPort" Accessibility="Public" Abstract="false" Base="System!System.LogicalEntity" Hosted="false" Singleton="false" Extension="false">
    <Property ID="ComputerName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="Key" Type="string" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="localIP" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="localPort" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="ipProtocol" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
    <Property ID="procName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="512" MinLength="0" Required="false" Scale="0" />
    <Property ID="comment" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="1024" MinLength="0" Required="false" Scale="0" />
</ClassType>
To create a relation between computer and it’s monitored tcp-connections or listening-ports two additional classes are required:
<RelationshipType ID="Network.Windows.Computer.NetstatWatcher.ComputerHostsTcpConnection" Accessibility="Public" Abstract="false" Base="System!System.Containment">
    <Source ID="Source" MinCardinality="0" MaxCardinality="2147483647" Type="Network.Windows.Computer.NetstatWatcher.Computer" />
    <Target ID="Target" MinCardinality="0" MaxCardinality="2147483647" Type="Network.Windows.Computer.NetstatWatcher.TcpConnection" />
</RelationshipType>
<RelationshipType ID="Network.Windows.Computer.NetstatWatcher.ComputerHostsListeningPort" Accessibility="Public" Abstract="false" Base="System!System.Containment">
    <Source ID="Source" MinCardinality="0" MaxCardinality="2147483647" Type="Network.Windows.Computer.NetstatWatcher.Computer" />
    <Target ID="Target" MinCardinality="0" MaxCardinality="2147483647" Type="Network.Windows.Computer.NetstatWatcher.ListeningPort" />
</RelationshipType>

Discoveries

"Discovery" is the the mechanism of creating objects that match the definition and storing them in the SCOM database. There are different types of discoveries, starting from matching registry values over results of a Windows Management Instrumentation (WMI) query to scripts that can cover everything. Targets define on which component the discovery will run.

First, we use the discovery Discovery.NetstatWatcher.Computer to find computer objects. It targets all Windows computers (which SCOM already monitors).

The FilteredRegistryDiscoveryProvider discovery scans the registry, and if the key HKLM\ SOFTWARE\ABCIT\NetstatWatcher exists, it will create the object. The interval is daily.

Also discovered here is the FilterPath, which defines the path in the file system where we'll find both text files.

The second discovery Discovery.NetstatWatcher.listeningPorts finds listening ports reading out monitoredListeningPorts.csv. It targets the previously discovered …NetstatWatcher.Computer computer objects.

The TimedPowerShell.DiscoveryProvider triggers the DiscoverNetstatWatcherItems.ps1 PowerShell script that does the logic (see above: Preparing raw data). The interval is hourly.

The third discovery Discovery.NetstatWatcher.tcpConnections finds listening ports reading out monitoredTcpConnects.csv. It targets the previously discovered …NetstatWatcher.Computer computer objects.

The fourth and fifth discoveries Discovery.NetstatWatcher.ComputerHostsTcpConnections / …ComputerHostsListeningPorts create the relation between computers and the monitored objects.

The TimedPowerShell.DiscoveryProvider also triggers DiscoverNetstatWatcherItemRelations.ps1 script. The interval is hourly.

Monitors

Monitors are for finding out which health state an object has.

tcpConnection targets all objects of the class Network.Windows.Computer.NetstatWatcher.TcpConnection.
listeningPort targets all objects of the class Network.Windows.Computer.NetstatWatcher.ListeningPort.

This monitor here uses the PowerShell script MonitorNetstatWatcherItems.ps1 to determine the state of the object (see above: Interpreting output and initiating reaction). The interval is every five minutes.

Views

State views make all discovered objects and their health states visible.

State view showing listening ports

State View showing TCP connections

The system creates alerts for non-listening ports or lost connections. The NetstatWatcher Alerts view shows these.

Conclusion

You can download the management pack with the .xml or. mpb extensions from GitHub. I published the entire software under the GNU General Public License GitHub. Feel free to use it without costs or obligations. The software is provided "as is" without express or implied warranty.

If you don't like the naming used, feel free to change the text in the XML file. Make sure you search with case sensitivity. I used Visual Studio 2015 with the Authoring Extensions add-on for this management pack.