In PowerShell, you can download a file via HTTP, HTTPS, and FTP with the Invoke-WebRequest cmdlet
Latest posts by Michael Pietroforte (see all)

Download with SMB

If you are working in a hybrid IT environment, you often need to download or upload files from or to the cloud in your PowerShell scripts. If you only use Windows servers that communicate through the Server Message Block (SMB) protocol, you can simply use the Copy-Item cmdlet to copy the file from a network share:

Copy-Item -Source \\server\share\file -Destination C:\path\

This assumes that you have a VPN solution in place so that your cloud network virtually belongs to your intranet. Things get a bit more complicated if we are leaving the intranet and have to download from an extranet or the Internet.

Download in PowerShell 2

The next simple case is where you have to download a file from the web or from an FTP server. In PowerShell 2, you had to use the New-Object cmdlet for this purpose:

$WebClient = New-Object System.Net.WebClient
$WebClient.DownloadFile("https://www.contoso.com/file","C:\path\file")

As of PowerShell 3, we have the Invoke-WebRequest cmdlet, which is more convenient to work with. It is PowerShell’s counterpart to GNU wget, a popular tool in the Linux world, which is probably the reason Microsoft decided to use its name as an alias for Invoke-WebRequest. This is perhaps an understatement; Invoke-WebRequest is more powerful than wget because it allows you to not only download files but also parse them. But this is a topic for another post.

Download with Invoke-WebRequest

To simply download a file through HTTP, you can use this command:

Invoke-WebRequest -Uri "http://www.contoso.com" -OutFile "C:\path\file"

In the example, we just download the HTML page that the web server at www.contoso.com generates. Note that, if you only specify the folder without the file name, as you can do with Copy-Item, PowerShell will error:

Invoke-WebRequest : Could not find a part of the path

The shorter version for the command line is:

wget "http://www.contoso.com" -outfile "file"

If you omit the local path to the folder, Invoke-WebRequest will just use your current folder. The -Outfile parameter is always required if you want to save the file. The reason is that, by default, Invoke-WebRequest sends the downloaded file to the pipeline.

However, the pipeline will then not just contain the contents of the file. Instead, you will find an object with a variety of properties and methods that allow you to analyze text files. If you send a binary file through the pipeline, PowerShell will treat it as a text file and you won’t be able to use the data in the file.

To only read the contents of the text file, we need to read the Content property of the object in the pipeline:

Invoke-WebRequest "http://www.contoso.com" | Select-Object -ExpandProperty Content | Out-File "file"

This command does the same thing as the previous one. The -ExpandProperty parameter ensures that the header (in this case, “Content”) won’t be stored in the file.

If you want to have the file in the pipeline and store it locally, you have to use -PassThru parameter:

Invoke-WebRequest "http://www.contoso.com" -OutFile "file" -PassThru | Select-Object -ExpandProperty Content

This command stores the web page in a file and displays the HTML code.

Download and display file

Download and display file

Authenticating at a web server

If the web server requires authentication, you have to use the -Credential parameter:

Invoke-WebRequest -Uri https://www.contoso.com/ -OutFile C:"\path\file" -Credential "yourUserName"

Note that, if you omit the -Credential parameter, PowerShell will not prompt you for a user name and password and will throw this error:

Invoke-WebRequest : Authorization Required

You have to at least pass the user name with the -Credential parameter. PowerShell will then ask for the password. If you want to avoid a dialog window in your script, you can store the credentials in a PSCredential object:

$Credentials = Get-Credential
Invoke-WebRequest -Uri "https://www.contoso.com" -OutFile "C:\path\file" -Credential $Credentials

You can use the -UseDefaultCredentials parameter instead of the -Credential parameter if you want to use the credentials of the current user. To add a little extra security, you might want to encrypt the password. Make sure to always use HTTPS instead of HTTP if you have to authenticate on a remote server. If the web server uses basic authentication, your password will be transmitted in clear text if you download via HTTP.

Note that this method only works if the web server manages authentication. Nowadays, most websites use the features of a content management system (CMS) to authenticate users. Usually, you then have to fill out an HTML form. I will explain in one of my next posts how you can do this with Invoke-WebRequest.

Downloading files through FTP works analogous to HTTP. You also shouldn’t use this protocol if security matters. To download multiple files securely, you had better work with SFTP or SCP. Invoke-WebRequest doesn’t support these protocols. However, third-party PowerShell modules exist that step into the breach.

In my next post I will show you can use Invoke-WebRequest to parse HTML pages and scrape content from websites.

avataravataravatar
85 Comments
  1. Shane 4 years ago

    Hi Michael – great article.

    I am running a script on a scheduled basis (daily) to download a .csv file. However the uri changes every month, so I was wondering if the uri destination value can be set based on a value in a reference file as opposed to hard coding it, if so how?

    Cheers

     

  2. Shane 4 years ago

    Thanks Michael – worked perfectly!

    avatar
  3. Rof 4 years ago

    Hi Michael

    i am downloading a  zip file from a website using the PowerShell, however the issue  is that i have to filter by date to download that zip file.

    Thank you.

    • Author

      Is the date on the website? Then use Invoke-WebRequest to read and then adapt the script to get the right URL of the zip. Will take some coding.

  4. Billy Madison 4 years ago

    Great tips, can you tell me how you would apply this same concept in powershell to download all files from a web folder? Thank you in advance.

     

    Example: https://www.foo.com/folder/*

    avatar
    • Leos Marek (Rank 4) 4 years ago

      Im not sure whether this is possible. You would somehow need to enumerate the content of the folder and then download it. That is normally forbidden by webservers. 

      If you have a webserver where directory browsing is allowed, I guess you could use invoke-webrequest/invoke-restmethod to that folder which would list available files. Then you could parse the output and ask for specific files to be downloaded (or all of them). But I dont see any straight-forward way.

  5. Marc@4s 3 years ago

    In case you don't want to specify the destination filename:

    If filename is part of the url, you could use

    $filename = [System.Uri]::UnescapeDataString((Split-Path -Leaf $strDownloadURL))

    (e.g. "Test%20123.pdf" –> "Test 123.pdf")

    If it's not: Use function "Get-RedirectedUrl" from https://stackoverflow.com/questions/25125818/powershell-invoke-webrequest-how-to-automatically-use-original-file-name/25127597#25127597

  6. Matt 3 years ago

    I'm trying to get a list of log file paths for a cleanup script.

    $r = Invoke-WebRequest -Uri http://10.7.58.99/logpaths.txt
    $logpaths = $r.Content

    This works fine but I cannot step through this content. When I put this content through a foreach loop it dumps every line at once. If I save it to a file then I can use System.IO.File::ReadLines to steps through line by line but that only works if I download the file. How can I accomplish this without downloading the file?

    • Author

      You can’t parse text files with Invoke-WebRequest. If the text file is unstructured you can parse it with regex. More information about using regex in PowerShell can be found here and here

  7. Andrew Cirel 2 years ago

    WebRequest gives memory errors when trying to use it to download large files.

  8. Ken 2 years ago

    Hello, I am having an issue with webrequest and downloading files.

    I am trying to download files from a site, sadly they are be generated to include the Epoch Unix timestamp in the file name.
    example:
    Upload_Result_20210624_1624549986563.txt
    system_Result_20210624_1624549986720.csv

    Also all the files are being kept in single folder, such as Upload or Result.

    Now as I am unable to replace the Epoch Unix timestamp portion of the file name with a wild card, I was wondering if there was a way to do the download based on the date modified field of the file ?

  9. Sumit Mukherjee 2 years ago

    Hi,

    Thanks for this. I plan to use this along with task scheduler to download a fresh file every week. However, the new file overwrites the older one. Is there a way to preserve the older file as well?

    Regards.

  10. @Ken – You should be able to relatively easily – however, you’d have to download it first, since you can’t get the file properties until you download it. You could download it to a temp location, grab the LastWriteTime stamp and parse it to create your new name.

    @Sumit – You have a similar situation. Download your file to a temporary location, and then copy/rename it with a timestamp in the name, and then you’ll keep a running list. You’ll also need to manage the old copies so you don’t fill up your disk. 🙂

    David F.

  11. Marta V 2 years ago

    hi, I’m a bit new to powershell and trying to figure out if it is possible to add either a powershell script or a function inside a powershell script to your path. Basically at the moment I have a problem and have to open powershell in the directory where the powershell script is, or navigate to the directory. From there I then have to execute my script. I want to just be able to execute script from anywhere on the shell without messing about with specifying the path to the tool, is than even possible?

    • Author

      Marta, you can append the PATH environment variable. The PATH variable determines where the OS searches for executables.

      You could change $PATH in your PowerShell profile to make it permanent:

      $env:Path += ";C:\myScripts\"

  12. What you should probably do is just either put the scripts you want in your $profile script or just dot source them from there. Having them in your path doesn’t necessarily do much for you.

    If you just dot source them from your profile, then you can easily call your functions from anywhere..

    David F.

    • Author

      David, that works if you just have a few tiny scripts. However, in general you will want to organize your scripts in folders. Why do you think that working with the path variable won’t work?

  13. Polo Qaz 2 years ago

    Hey. I am trying to write a script to automate file download in PowerShell from Microsoft website:
    https://teams.microsoft.com/downloads/desktopurl?env=production&plat=windows&arch=x64&managedInstaller=true&download=true
    But all functions are failing to download… any idea?

  14. Hello All,

    I’ve written a smile script to download a test file from my github and it works fine on my laptop powershell, however it throws below error when i run the same script on Windows Server 2012. The only thing i noticed is that version of Powershell is different only Laptop and the Server.

    Exception calling “DownloadFile” with “2” argument(s): “An exception occurred during a WebClient request.”
    At line:3 char:1
    + $WebClient.DownloadFile(“$Link”,”$env:C:\Test_powershell\BackupCI.xls …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException

    any suggestions/help on this is highly appreciated

  15. FB 8 months ago

    Hello,
    Great Article.

    But I need help :
    I would like to recover a file with a .tok extension in PS.
    I try with 2 different methods : Invoke-WebRequest and Invoke-RestMethod
    And It is not OK.

    Today, my only solution is to convert the .toc file in .txt file.
    But I want to keep the original extension.
    Do you have an idea?
    Thank you.

    For exemple with Invoke-WebRequest :

    PS C:\Temp>Invoke-WebRequest -Uri "http://server/Test.txt" -Outfile "C:\Temp\Test.txt"       => Ok
    
    PS C:\Temp>Invoke-WebRequest -Uri "http://server/Test.tok" -Outfile "C:\Temp\Test.tok"
    => 
    Invoke-WebRequest : Server Error
    404 - File or directory not found.
    The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.
    Au caractère Ligne:1 : 1
    + Invoke-WebRequest -Uri "http://dxcl6008/NVIDIA/test.tok" -Outfile "C: ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation : (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
        + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

  16. FB 8 months ago

    Michael,
    The file is on the server.
    Did you ever work with this kind of file ?

  17. Author

    Maybe the web server is configured not to serve this file type. Try to open the URL in a web browser and see it what happens. Invoke-WebRequest is doing the same as any browser.

  18. FB 8 months ago

    I cannot paste a picture but when I open the url in a web browser, I see my 2 files .txt and .tok.

Leave a reply

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2023

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account