When you need to run PowerShell commands against a large set of computers with the PowerShell cmdlet Invoke-Command, you often have to deal with offline computers. In this post, you will learn how to deal with unresponsive machines.
Avatar

This cmdlet can easily reach out to hundreds of computers with only one line of code. But running code on remote machines can be a bit of a minefield if you have computers spread across many locations or multiple domains. Today, I will show you how can handle three common challenges you may encounter when trying to use Invoke-Command to connect to multiple computers on your network:

  • Avoiding connecting to offline computers
  • Connecting to computers requiring different credentials
  • Compensating for computers that may respond slowly

These situations and others can make running PowerShell remote commands a challenge. Follow along as I share some tips you can use to handle these situations.

What is PowerShell Remoting?

PowerShell Remoting allows you to connect to multiple computers on your network from a PowerShell command prompt. Remoting makes a secure connection behind the scenes to each computer you specify and then executes the commands on the remote machine(s). When the commands finish processing, it sends the results back to your machine. It differs from connecting via Windows Management Instrumentation (WMI) or using some other tool like PsExec or Remote Desktop Protocol (RDP). PowerShell Remoting connects to multiple machines in bulk and has security features built in to make the connections secure.

This will be a deeper dive on remoting and challenges you may encounter. If you need a primer on getting started with PowerShell remoting, refer to the 4sysops wiki. I have also discussed remoting configurations and security options extensively on my site and recorded a video presentation on security and logging options. In addition, you can read up on remoting by typing Help About_Remoting at your PowerShell cmd prompt.

Avoiding connecting to unreachable computers

Invoke-Command can connect to many machines at once, but you can encounter problems when the cmdlets can't connect to a remote PC. There are many reasons a remote connection can't connect. It could be a DNS issue, or the PC may be offline. It could also be a bad IP address, the ports may not be open, or a firewall could block the connection attempt.

When a connection isn't possible, Invoke-Command will throw an error. The error messages Invoke-Command provides can be useful, but if you want to run a query against a large set of computers, these error messages get in the way. If you attempt to connect to a computer that isn't reachable, Invoke-Command can take a long time to complete waiting for each query to timeout.

In the example below, I am querying the date on four computers. Two of the computers are real, and two are fake names that will not resolve. I attempt to connect to each PC with Invoke-Command and measure how long it takes for the process to complete.

Long timeouts for bad computer names

Long timeouts for bad computer names

The first thing you probably noticed is the error text returned for the two failed computers, server1 and server2. These error messages are helpful, but if I were parsing the output, the errors would cause problems for me when tallying results. Look at how long it took to resolve those four names: 21 seconds! Ninety percent of the time spent was waiting for the two fake names to time out.

So how can we avoid the errors and the long wait? Invoke-Command excels at executing code against remote computers and managing the data returned. It has no options available to check the status of computers. That means we need to test computers before we execute the command. To make a remote connection, port 5985 must be open on the remote computer, and the WinRM service must be running. Domain-joined computers running Windows 8 or above will have the WinRM service set to autostart.

Knowing this, we could build a simple test of each computer by checking whether port 5985 is open, and if not, we skip connecting to that computer. An easy way to do this is with an If statement and the Test-NetConnection cmdlet. Test-NetConnection is a more advanced version of ping, and it returns objects instead of text. We're not interested in time to live (TTL) or hops for this task, just whether the computer is online or offline, and Test-NetConnection can report back only online or offline status.

Test-NetConnection computername -port 5985 -InformationLevel Quiet

This syntax will return True (reachable) or False (unreachable). We're testing port 5985 specifically because our Invoke-Command cmdlet will use this port. We want to test each computer, and if it's online, save the result to a new list.

In my test below, I have saved five computer names to a variable called $Computers, and then I build a test of each computer. Two of the computers are offline. The code tests each PC, and if it's online, it adds the computer name to a new variable called $Online.

$online = @()
$Computers = "svr01","svr02","svr03","svr04","svr05"

Foreach ($PC in $Computers){
    If((Test-NetConnection $PC -InformationLevel Quiet)) { $online += $PC }
}

The result is that we now have a list that contains only the computers that are online and reachable on port 5985. Now we can run Invoke-Command and make successful connections to all online computers.

Subscribe to 4sysops newsletter!

$results = Invoke-Command $online -ScriptBlock {Get-Date}
$results | Select-Object PSComputerName, Date

PSComputerName      Date
--------------      ----
Svr04               Tuesday, December 24, 2019 11:26:16 AM
Svr01               Tuesday, December 24, 2019 11:26:10 AM
Svr05               Tuesday, December 24, 2019 11:26:11 AM

Summary

This is the first in series on how to deal with situations that can cause problems when connecting to remote computers using Invoke-Command. make sure to visit the other posts for great tips. Please use the comments section below to reach out with your questions and ideas.

avataravataravataravatar
Articles in seriesInvoke-Command: Common challenges
16 Comments
  1. Avatar
    Jeff Hicks 4 years ago

    You could also use the -CommonTCPPort parameter in case you can't remember what the WSMan port is.

    Test-Netconnection srv1 -commontcpport winrm -informationlevel quiet

    But you would need the port if you were using remoting over SSL, in which case you would test port 5986. One other thing to keep in mind is that this command is merely testing if the port is responding. Theoretically, the port might be responding for some other reason. This is where using Test-WSMan might be better because it will verify if the remoting service is actually running on the remote server. Plus, you can specify an alternate credential that might be useful if your Invoke-Command is also using an alternate credential.

    avataravataravataravataravatar
    • Avatar
      Leos Marek (Rank 4) 4 years ago

      Have to agree with Jeff. Testing the port might not provide relevant answer. Test-WSMan should be used instead.

      • Avatar Author

        I agree with both Jeff and yourself. This is one of those scenarios where there more than one way to testdepending on what results you need back and how well you know your computers. 

        However, I have a small issue with Test-WSman. For the record, I use Test-Wsman all the time and find it very useful. However, the output has to be parsed in order to make a determination in a script that the machine is configured correctly. 

        Contrast this with Jeff's first example or mine where the result is true or false. I agree Test-WSMan is the more complete test. But if you know your work environment well then maybe Test-WSMan is overkill. Also consider that in a domain situation, win8, win10, srv2012, 2016, and 2019 machines all have remoting on by default; so maybe test-WSMan is the better test, but is it necessary? 

        I think it depends on the task you are working on… 
         

  2. Avatar
    PowerMe! (Rank 3) 4 years ago

    Thank you Mike.

    I was reading about the TCP ports used PoweShell remoting. PS-remoting uses HTTP and/or HTTPs. The common ports being TCP 80 and TCP 443, respectively.

    It seems like, due to security concerns, these ports have been moved to TCP 5985 (HTTP) and 5986 (HTTPs). Also note that if you'd like to use IIS 80 and 443 must be available. The above is an interesting article on the port-security. I prefer -useSSL as HTTP is plain-text.

    • Avatar Author

      A few things to consider here some of which Jeff already mentioned:

      • The ports for remoting are 5985 and 5986 with 5986 only coming into use when certificates are configured and can be resolved meaning you need an internal PKI infrastructure.
      • All remoting traffic is encrypted by default with a 256 bit key; even over HTTP.
      • Remoting only occurs when you use the correct commands. WMI is not considered remoting.
      • SSL verifies the computer. Remoting is authenticated and takes care of resolving the user.
      • Not everyone can use remoting. You need to have admin priv's on a box or be a member of the remote mgmt users group. The other option is JEA is in place but that's a higher level discussion.
      • Remoting is very safe. if you have security concerns, it should not be with remoting. Chances are the user security on your client's is the security weakness. 
      • The article you referenced is 10 years old. Microsoft moved away from 80 and 443 because there's enough on those ports already, but doesnt imply there are security concerns with remoting. 
      • IT shops can AND should be using remoting. it is one of the safest methods for interacting with a remote PC and it the equivalent of SSH for Linux/mac clients.

  3. Avatar
    Jeff Hicks 4 years ago

    While I would still recommend configuring remoting with SSL in a corporate environment, even if you use HTTP and 5985 the traffic is still encrypted by default. 

    get-item WSMan:\localhost\Service\AllowUnencrypted

    Using SSL with remoting provides an additional layer of security. 

    avatar
    • Avatar

      I think Microsoft didn't add HTTPS support as an "additional layer of security." PowerSell remoting via HTTP is only encrypted if both computers are Active Directory members. For standalone computers, PowerShell remoting is not encrypted by default which I find amazing! At least it was like this when I looked into this last time and I doubt that Microsoft really cares. So you need to configure HTTPS "manually" if you work with standalone machines. This is the reason why Microsoft added HTTPS support.

      • Avatar Author

        Michael,

        I wouldn;t say your wrong but to me… remoting between WORKGROUP computers is not the norm. Since there is no central spot to handle the encryption and decryption with workgroup PCs it sort of makes sense that SSL is not on by default on a standalone PC.

        Most readers of this blog are dealing with domains and the default config for domain joined computers differs greatly from workgroup pc's. 

        • Avatar

          Mike, this argument could be from a Microsoft official. 😉 The main problem is that most IT pros are unaware of the fact that PowerShell remoting is not encrypted on standalone computers because it is kind of crazy to even support this and even worse to use this as the default setting. Nobody expects such irresponsible behavior nowadays.

          Standalone machines might not be the norm in on-premisses networks. However, in the cloud you often don't need Active Directory. AD was not really made for the cloud and gets in the way if you do "cattle computing."

          I also don't understand why Microsoft simply doesn't copy the SSH way. Messing around with SSL certificates just to get a secure remoting connection is kind of silly.

          avataravatar
      • Avatar
        PowerMe! (Rank 3) 4 years ago

        I did a lab to understand PS-Remoting. Michael has an interesting article on the latter.

        • I used a Windows Server 2019 as my target and Windows 10 as the source of PS-Remoting.
        • They are WorkGroup hosts (no Domain).
        • I have tested both HTTP and HTTPS methods with the following script where I changed the port number to 5985 for HTTP.
        $myhost = 'winS19'
        $winrmPort = '5986'
        $cred = $null
        $cred = Get-Credential -UserName Administrator -Message "Provide Password"
        
        if(Test-NetConnection $myhost -port $winrmPort){
        
            $sessOptions = New-PSSessionOption -SkipCACheck
            $sess = New-PSSession -ComputerName $myhost -UseSSL -Credential $cred -SessionOption $sessOptions
            Enter-PSSession $sess
            }
        

        Encryption in PS-Remoting

        1. In both the cases- HTTP and HTTPS- the PS-Remoting traffic are encrypted.
        2. Encryption in HTTP is provided by NTLM Security Support Provider (NTLMSSP) which I think uses 128 bit encryption with Challenges. I was reading about encryption in HTTP based PS-Remoting. In a domain-environment it uses Kerberos, otherwise (workgroup) it uses NTLMSSP.
        3. HTTPS based PS-Remoting uses TLS1.2 with much secure encryption (AES 256, SHA 384 …) 
        4. Considering the vulnerabilities with NTLM, I would be more inclined towards HTTPS and remove an HTTP option as suggested by the article from Michael.

        I took packet captures in Wireshark. I have some screenshots that I can add to the post if useful.

        avatar
        • Avatar

          This is what Microsoft writes about encryption in WinRM:

          Under no circumstances should unencrypted HTTP requests be sent through proxy servers. When data must pass through a proxy server before being sent to the destination server, the following security issues are very important:

          • It is possible that a malicious proxy server could examine every request/response pair, including credentials.
          • If the TCP/IP connections are not strongly mapped between both the WinRM client and the proxy server and between the proxy server and the destination server, an unauthorized client could connect to the destination server by using the same authenticated connection from the proxy server to the destination server. The destination server might allow the unauthenticated client access to data. If encryption is enforced, the destination server sends an access denied message to the unauthenticated client.

          Using encryption would mitigate these potential security issues.

          This is about WinRM 2.0. I am unsure if things are different WinRM 3.0.

          avatar
          • Avatar
            PowerMe! (Rank 3) 4 years ago

            Nicely summarized. 

            Certificate based- SSL or SSH – not only encrypt data but also protect the identities of the 2 hosts involved in WinRM – little room for a Man-in-the-Middle attack.

            avatar
        • Avatar

          Also read this about the role of NTLM here. The password might no be transmitted in clear text, but this wouldn't really stop a skilled attacker. And the ongoing communication is a different matter anyway.

          avatar
  4. Avatar
    Denis 4 years ago
    if ( Test-Connection $hostname -Count 2 -Quiet:$true ) {
        $session = New-PSSession -ComputerName $hostname -Name $hostname -ErrorAction SilentlyContinue
        if ( $session.Availability -eq 'Available') {
            Invoke-Command -Session $session -ScriptBlock {
                
            }
            Remove-PSSession $session
        }
        else {
            Write-Host "$hostname Connect WinRM - failed" 
        }
    }
    else {
        Write-Host "$hostname offline." 
    }
    avatar
  5. Avatar
    Curtis Dove 3 years ago

    I'm needing to do something similar to what is being discussed and for me, iterating through each server with foreach ($Server in $Servers) to test whether or not the computer is alive and responsive, to me seems inefficient, the larger the set of computers, the longer the test will take. 

    Scenario, I have 20 Servers to upgrade software on, hypothetically 25% of these are offline at any given time.  How can I test all these Servers at the same time, versus each one in iteration? 

    Test-Connection Server1 ……..times out +30 second delay

    Test-Connection Server2……..times out +30 second delay, etc…..

    If I can test both of these at the same time, I lose 30 seconds, not 60… can we use Powershell 7, parallel parameter for this or a way to invoke-command against all of them at once, to figure out which ones are on and responsive, so I can run my script against only the ones that are alive and responsive? Saving me the most amount of time.  Any help would be greatly appreciated.

    avatar
    • Avatar
      Leos Marek (Rank 4) 4 years ago

      Something like this should work

      workflow Ping-Subnet3 {
          [CmdletBinding()]
          Param (
              
              [Parameter(Mandatory=$true,
                         ValueFromPipelineByPropertyName=$true,
                         Position=0)]
              [string[]]$comps
          )
      
      
      
          foreach -parallel ($computer in $comps)
          {
              Test-Connection -Count 1 -ComputerName $computer -ErrorAction SilentlyContinue
          }
      }

Leave a reply

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2023

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account