- AccessChk: View effective permissions on files and folders - Thu, Apr 13 2023
- Read NTFS permissions: View read, write, and deny access information with AccessEnum - Wed, Mar 29 2023
- Kill Windows a process with Tskill and Taskkill - Mon, Mar 13 2023
Over the past 25 years, since Robocopy first became available as part of the Windows NT4 resource pack, it has not changed much. Some features, such as SMB compression, were added, but the usage is still the same. As Robocopy has many great features, such as the ability to copy NTFS ACLs (permissions), it is one of the best free tools for file copy/mirror operations. This Wikipedia post is a good starting point for getting more information about robocopy.
Before we get to the Robocopy examples, you should know the main limitation of robocopy—it cannot copy open or locked files, as it does not use VSS (Volume Shadow Copy Service).
Help and default options
Robocopy itself includes very detailed help, with a description of each option. Simply type robocopy /? to get help displayed.
Robocopy help
The basic syntax for Robocopy is:
robocopy c:\temp\source c:\temp\destination
This will copy all files, excluding subfolders, from the specified source (c:\temp\source) to the specified destination (c:\temp\destination) with a bunch of default options.
Note: If you have a folder with spaces in the name, simply put quotes around the path (“c:\temp\source with spaces”).
Robocopy default options
Let's take a closer look at what the options mean:
- *.*—Defines a file filter. The first asterisk (*) represents the filename part, and the second asterisk represents the extension. In this case, the command copies all filenames with all extensions.
- /DCOPY:DA—Defines what to copy for directories. D stands for "data" and A stands for "attributes."
- /COPY:DAT—Defines what to copy for files. The extra T stands for "timestamps;" DA is the same as for directories.
- /R:1000000—Defines the number of retries on failed copies, for example, if a file is locked.
- /W:30—Defines the number of seconds to wait between retries.
As you can see, the number of retries is very large (one million), which is not practical in most cases. If there is a single locked file inside the directory, robocopy will do a million retries and will wait 30 seconds between them, amounting to almost 350 days. I guess this is not what you normally want.
Generally speaking, there are three things you will do using robocopy: copy, move, and synchronize folders and files.
Copy examples
The easiest way to copy a folder with all files and subfolders is to run this command:
robocopy c:\temp\source c:\temp\destination /E /DCOPY:DAT /R:10 /W:3
The /E switch tells Robocopy to copy all subfolders, including empty ones. If you don't want to copy empty subfolders, use the /S switch. Note that I have specified /DCOPY:DAT (default is DA), as I usually want the folder timestamps to be the same as the source. I also limit the number of retries and wait times. If there is a file locked, it won't usually be unlocked quickly, and it's easier for me to check the erred files manually later rather than waiting ages for the whole command to complete.
If you are about to copy a large number of files, it might be more useful to save the output to a log file so you can review the results later, even if the command prompt is closed.
robocopy c:\temp\source c:\temp\destination /E /DCOPY:DAT /R:100 /W:3 /LOG:C:\temp\robocopylog.txt
Will give you a text log file with all the information you would normally see on screen.
Redirecting the output to a log file
Note: If the destination directory (c:\temp\destination) already exists, you will see one skipped folder in the results.
In many (if not most) cases, you will need to create a completely identical copy of the source, including the NTFS owner and auditing information. For that, I add /COPYALL to the command, which is equivalent to /COPY:DATSOU.
robocopy c:\temp\source c:\temp\destination /E /COPYALL /DCOPY:DAT /R:100 /W:3
Note that this command has to be run from an elevated command prompt, unless you have turned off UAC, otherwise you'll receive this error message:
ERROR : You do not have the Manage Auditing right.
Error when using /COPYALL without elevation
Move examples
Robocopy offers two options for moving files and folders:
- /MOV—Moves files only, leaving the empty folder structure at the source
- /MOVE—Moves files and folders
Use the command below to move all files and folders, including empty ones, with all attributes. Note that the source folder will also be deleted.
robocopy c:\temp\source c:\temp\destination /E /COPYALL /DCOPY:DAT /MOVE /R:100 /W:3
Synchronize examples
As with move, there are two options to synchronize folders:
- /PURGE—Deletes the files and folders that are no longer present in the source but without subfolders.
- /MIR—Is equivalent to /PURGE /E. /MIR also copies all files in subfolders, including empty ones.
Be careful with this command, especially with source and destination folders. If you make a mistake here, you might get files lost in the specified folder.
robocopy c:\temp\source c:\temp\destination /MIR /COPYALL /DCOPY:DAT /R:100 /W:3
Filtering examples
Sometimes you might want to copy only specific files, for example, Word documents. This can be done easily by adding the *.docx filter after the destination folder before any other options.
robocopy c:\temp\source c:\temp\destination *.doc
You might also want to exclude some files or folders from the operation. This is done using the /XF and /XF options. To exclude all .doc files, use the command below.
robocopy c:\temp\source c:\temp\destination /XF *.doc
To exclude multiple subfolders, use spaces in the command:
robocopy c:\temp\source c:\temp\destination /XD Subfolder EmptySubfolder
Robocopy includes many more filtering options, which you can find in help under the File Selection Options section.
Networking examples
Another great feature of Robocopy is that it works well with UNC paths and can resume operation in case of a network interruption. The command below will copy the files in restartable mode.
robocopy c:\temp\source \\MYSERVER\myshare /Z
Backup mode
The last, but definitely not the least, option I will cover today is the backup mode option /B. This option allows administrators to copy files in a special backup mode. This is extremely useful if you are asked to copy folders to which you don't have access. The backup mode uses the Backup and Restore security privilege to bypass the NTFS permissions and copy the files. This again requires a command prompt to be elevated. In addition, your account needs at least SeBackupPrivilege, which is usually granted to the Administrators and Backup Operators groups.
robocopy c:\temp\source c:\temp\destination /E /COPYALL /DCOPY:DAT /B
Final words
Robocopy is a very powerful tool for performing various file and folder operations. Many administrators use it on a daily basis for profile migrations and other similar operations. The big advantage is that it's part of the operating system itself. There are dozens of other options that can be used with Robocopy, but to cover them all would be beyond the scope of this post.
While robocopy is one of the most powerful ways to sync drives, it does fail spectacularly in certain events–dst changes, file systems other than ntfs, and just missing files. It is important to check on it periodically to make sure the results are what you want them to be. Another great alternative to robocopy is its non-ms twin, xxcopy.
I dont think that DST has any impact since NTFS timestamps are stored in UTC format. Of course, robocopy was created mainly for NTFS filesystem.
It actually does between non-NTFS file systems/nas units. There is even the /DST switch to help with the issue.
I have never used robocopy on any other system than NTFS. Yes there is that switch which somehow filters the file selection. No idea how it works and the description in help is not much useful. 🙂
A lot of sites have more details on the switches, but it is also hit and miss on how well it works depending on the source and destination.
For one backup operation I had to make a ‘fall’ and ‘spring’ set since when dst kicked in, robocopy would copy everything again and not just the changed files.
For backups in general I prefer more sophisticsted tools, like free and great Cobian Backup with its own service, schedule, compressions etc..
Backup programs can also add their own additional point of failure though if a restore has to be done through the backup program. This is why I’ve stuck with straight file copies for backups.
Thats the beauty of Cobian – it produce flat files copy or a simple zip archives that can be extracted by anything. Im preparing a post about Cobian, you will be able to read it soon if you like.
Can’t wait to read about Cobian–seems like it’s pretty neat!
Robocopy worked once copying shared folders from Windows to TrueNAS. After that, attempting to do it again for testing prior to committing to putting it in a batch file for Scheduled Tasks, it became completely worthless. TrueNAS now renders an “unable to open the file” error, even thougbh i have the correct switches and permissions on the TrueNAS share have not changed.
Either it’d s Robocopy iissur or it’s TrueNAS, which frankly I’m about out of patience with that as well.
Doing something as simple as syncing files between two different machines shiould not be so senselessly complicated. There just HAS to be a better way.
At this point I’m about to dump TrueNAS and just do Proxmox VMs and containers. I really wanted the bare metal NAS install, but not if I can’t easily sync my data folders between my machines and the NAS and back.
I can get close to what I want with Proxmox, but didn’t really want to be tied to everything in a VM, since it’s really not a NAS. My use case is more running apps in containers, but another critical use case for me is synchronozation between multiple machines.
Maybe I’ll just build a multi-node Proxmox cluster and be done with it.
Cheers,
-=Cameron
Strange that you’re running into this. Just as an experiment, will xcopy /d update the copy? I know this won’t delete any files deleted on the source like robocopy /mir, but if xcopy works, this might just be a robocopy limitation and you can use xxcopy instead which is pretty much the same as robocopy.
Similar to rclone?
Not using rclone but as I checked, its a similar tool. Robocopy is built-in in Windows and its not cross platform.
It may be worth mentioning that the various Robocopy exit codes for successes and failures. 0 through 7 are all successes, 8 through 16 are failures.
It might also be worth mentioning a few tricks for copying and comparison.
For copying from one source to multiple destinations, using the ‘start’ command in a ‘for’ loop can start the copies simultaneously and allow the 100% cache hit to effectively double the copy speed since the original is basically only being read once and written twice. I’ve used this for up to 4 destinations simultaneously and it works best with similar destinations. Even when the destinations are not similar and the individual robocopy sessions become ‘off sync’ from each other, the latency of hitting the disk usually brings them back into alignment because one of the sessions has a cache hit while the other is still going to disk.
For comparison, MS’s own windiff is a pretty good way to compare entire tree branches. However, if one needs to compare a single source to multiple destinations, winmerge does a better job since it has the ability to actually compare a single source to 2x destinations. And being a portable program that runs on the various windows platforms (just like windiff), it’s another handy tool that can check the work of a robocopy session for bit by bit accuracy.
You are welcome to share such tricks with specific commands, readers would appriciate that.
It’s pretty simple actually. If your source is T:\SUBDIR and destinations are \\10.10.10.1\VOLUME, \\10.10.10.2\VOLUME, \\10.10.10.2\VOLUME, then something like:[code]FOR %F IN (\\10.10.10.1\VOLUME \\10.10.10.2\VOLUME \\10.10.10.2\VOLUME) DO START ROBOCOPY T:\SUBDIR %F\SUBDIR /MIR /COPY:ALL /R:3 /W:3[code]I would also recommend testing to make sure your command is correct by using echo:
[code]FOR %F IN (\\10.10.10.1\VOLUME \\10.10.10.2\VOLUME \\10.10.10.2\VOLUME) DO ECHO START ROBOCOPY T:\SUBDIR %F\SUBDIR /MIR /COPY:ALL /R:3 /W:3[code]
I use robocopy to copy bulk data on my eDiscovery servers and it still works great. You did a great job explaining its use cases. Thank you Leos.
Thanks Surender.
Thanks for posting this. I have used the MIR function of Robocopy to maintain a software distribution scheme so that all of the destinations end up with the same files as the source regularly. It “finds” corruption and corrects it nicely against a known good copy and helps force good update practices.
How in the world is robocopy finding corruption and correcting it? afaik robocopy doesn’t have a built-in compare function.
The MIR function at a minimum compares timestamps. Unless there is a way to change a file without altering the timestamp (there maybe, I am unsure) then robocopy /MIR will ensure the source files overwrite the destination files if there is a change recorded in the OS.
It probably wouldn’t repair hardware or driver level corruption, but it is great at fixing the corruption introduced by users.
I think I see what you’re saying. If someone alters the destination file, it would be replaced by the source to restore the ‘corruption’.
Yes, exactly. It makes even more sense when you understand the files we are overwriting are a mixture of executables, reports, dlls, and database edits.
Whenever I have a need for something to be read only like this though, I’ll set set the permissions to read-only.
Yes, we were told by the software vendor not to do that. So this is a bit of a work-around.
Some programs do change few bytes in files without altering the timestamps.
Microsoft Office did so (or still does?) in files of the old DOC, XLS, PPT format. As I remember, it was enough to open and close the files to see the changes.
I got to know Robocopy years ago from a colleague in I.T.
It didn’t take me long to realize I hate creating batch files so I did a google search for a Robocopy GUI
and I found these There are these:
SyncToy
Gs Richcopy 360
Syncback
Some are free but the pay versions are reasonably priced.
SyncToy is free and does not use Robocopy but it is pretty decent and was developed by a man from Microsoft.
Gs Richcopy 360 and Syncback are fantastic and able to replace Robocopy , and are easy to use,full_featured, and can solve the issues that robocopy can’t handle.
A quick Google search will yield results so I won’t post any URLs here.
I am a GUI lover so I love these options.
Be careful with admitting that you are a GUI lover. This is almost like being an anti royalist these days. 😉
🤣🤣
You are right Michael 🤣🤣
If you were in Australia it would be the opposite. Being a royalist is seriously on the nose. In fact I think GUI’s are far more palatable than royalists.
Yeah bad comparison. The only parallel is that both sites are getting somewhat emotional when the topic comes up. I guess with VR / 3D GUIs and AI that writes code we will soon get into the next round.
Hello Sarah
I too recognize that the GUI is very practical and useful … but only in certain cases: Simple and unitary tasks
For all that is complex tasks – even simple but repetitive – nothing beats the command line and the script.
you named 3 tools: SyncToy – Gs Richcopy 360 – Syncback
Try copying millions of files, possibly without keeping the same tree structure at the target as at the source, and logging errors (the most frequent being missing NTFS permissions at the source), I wish you good luck with these tools.
These 3 tools are undeniably practical for personal use only and limited in volume, not for professional use.
As per my experince , You mean SyncToy , the Microsoft tool , but for this comment
“Try copying millions of files, possibly without keeping the same tree structure at the target as at the source, and logging errors (the most frequent being missing NTFS permissions at the source) ” ,
Gs Richcopy 360 and Syncback are created exactly for such cases.
Backup/sync GUI tools like Gs Richcopy 360 and Syncback have a nice and simple GUI, able to copy to local drives, remote servers, LANs, WANs, and clouds, able to copy all the permissions types from source to destination, and also able to copy time stamps.
and as I remember, there are options to throttle the connection speed to prevent bandwidth consumption, a feature to email you after the job is finished, an excellent task scheduler and it will never crash while transferring a large amount of data.
there are also other differences between such a GUI tools and CLI tools, just try to search
I’ll take a command line over a GUI any day since it’s batchable. Plus I like the keyboard so anything that keeps me away from the mouse is my preferred platform.
Hi
Can anyone provide me with a script to move my files from my NAS File Server Storage Location directly to my External USB connected drive without interference.
Regards
Mario
So … how would I copy all copies of, say, fdeploy.dll from one drive to another?
ROBOCOPY C:\ I:\ FDEPLOY.DLL /S /XO /XJ /W:O /R:O /ETA /FP
… starts a complete copy of C:\ to I:\ rather tan looking for any version of FDEPLOY.DLL on C: and when found, then copying each found file to a similar directory on I:
Stuck!
PS – The overall task is to replace files identified as missing by an SFC /SCANNOW on drive I: when it was installed in another matching computer as drive C:, drive I: is missing some stuff it can’t replace on it’s own, so I want to grab all missing files from another working drive, and I have dozens to go.
Thanks.
Robocopy isn’t the most effective tool for copying a single file. It is aimed and designed squarely around copying directories. It would be much more effective just to scan the drive with powershell and copy those files. Maybe something like this..
[code]
Get-ChildItem -Path C:\ -recurse -file -include fdeploy.dll | foreach-object { copy-item -Path $_ -Destination ($_.FullName.Replace(‘C:’, ‘I:’)) }
[/code]
Coralon
I just was playing aroung with ChatGPT and asked how to preserve NTFS hardlinks when copying files from one drive to another. The answer was that the option /B of robocopy “causes the files to be copied in binary mode, so all attributes and properties, including hardlinks, are preserved.” I never used the option /B.
It would be great if you could you say something about this option.
It seems you encountered a typical ChatGPT hallucination because the /b switch stands for “backup mode” and not “binary mode.” Check out the robocopy documentation.