Filename too long when copying to External USB

lordsheepface

I have Windows 10. I'm trying to copy a folder to an external USB. I'd like to preserve the folder / directory structure.

When I copy using XCOPY it seems to run out of memory. When I copy using regular Windows copy and paste it comes up with an error of the filename being too long at some point (I didn't screenshot the error)

Is there a better way to do this? The copy obviously takes 3 or 4 hours so I want to be able to leave it running without having to check for errors.

This folder is 623GB in total and contains 214,867 files and 13,532 subfolders.

Thanks for your help

omegab

Robocopy

Also maybe you've created a longer folder to put them into on the USB drive? So that you then go beyond the full length:
https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

SnowyMay

lordsheepface wrote: »

I have Windows 10. I'm trying to copy a folder to an external USB. I'd like to preserve the folder / directory structure.

When I copy using XCOPY it seems to run out of memory. When I copy using regular Windows copy and paste it comes up with an error of the filename being too long at some point (I didn't screenshot the error)

Is there a better way to do this? The copy obviously takes 3 or 4 hours so I want to be able to leave it running without having to check for errors.

This folder is 623GB in total and contains 214,867 files and 13,532 subfolders.

Thanks for your help

Ok - stupid question, but you do have enough space on the destination drive, yeah?

Have you tried Robocopy? It can just be run as a cmd, and will maintain folder structure, metadata, and isn't particularly intense.

Can give you some basic info and switches if you aren't familiar with it.

lordsheepface

Thanks for the replies. I can't say I fully understand it all but I have been using this very successfully for some time (saved in a .BAT file).

xcopy "C:\Users\john doe\Dropbox\*" "d:\john doe backup\" /E /C /w

With the result being that all files in my "c:\Users\john doe\dropbox" folder are copied into my "d:\john doe backup\" folder and if "d:\john doe backup\" does not exist it creates it.

My external USB is brand new. It's formatted NTFS and has 931GB free space.

So now I want to try and duplicate my xcopy but using robocopy. I've gone with this:

robocopy "C:\Users\john doe\Dropbox\*" "D:\john doe backup\" /mt /z

It seems my source destination is an invalid paramter?

Ideally I just want to run a .BAT file to copy this folder and for it to skip any errors and tell me at the end what the errors were. I don't want to have to have any interaction during the long copy.

My normal process would be that I would change the \ folder and it would be "john doe backup 1 " and then "john doe backup 2" etc. Normally I would pick and chose specific source directory but this time I want the hole full dropbox one. This is the first time I've tried to copy a folder this large or this many files / subfolders.

Appreciate any help.

lordsheepface

omegab wrote: »

Robocopy

Also maybe you've created a longer folder to put them into on the USB drive? So that you then go beyond the full length:
https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

I just see your edit now.

Surely copying from c-users-USERNAME\Dropbox to D\USERNAME\ actually creates a shorter file path than existed beforehand? Or am I misunderstanding you

(I've had to remove some colons from the directorys above as the forum software doesn't seem to like them!)

SnowyMay

lordsheepface wrote: »

Thanks for the replies. I can't say I fully understand it all but I have been using this very successfully for some time (saved in a .BAT file).

xcopy "C:\Users\john doe\Dropbox\*" "d:\john doe backup\" /E /C /w

With the result being that all files in my "c:\Users\john doe\dropbox" folder are copied into my "d:\john doe backup\" folder and if "d:\john doe backup\" does not exist it creates it.

My external USB is brand new. It's formatted NTFS and has 931GB free space.

So now I want to try and duplicate my xcopy but using robocopy. I've gone with this:

robocopy "C:\Users\john doe\Dropbox\*" "D:\john doe backup\" /mt /z

It seems my source destination is an invalid paramter?

Ideally I just want to run a .BAT file to copy this folder and for it to skip any errors and tell me at the end what the errors were. I don't want to have to have any interaction during the long copy.

My normal process would be that I would change the \ folder and it would be "john doe backup 1 " and then "john doe backup 2" etc. Normally I would pick and chose specific source directory but this time I want the hole full dropbox one. This is the first time I've tried to copy a folder this large or this many files / subfolders.

Appreciate any help.

Ok - wiser people than me will maybe hop in, but the asterisks in your source path should be removed. Also, don’t leave either paths with a trailing backslash. Robocopy is awesome, but sticks to his rules, and can get confused.

/Mt will give you multithreading to the power of 8 threads at a time (you can set higher if you wish, but I personally wouldn’t be bothered for this size of data) and /z is useful for restarting if something fails for some reason, but consider adding /e (for subfolders including empty ones), /v for having a clue of what’s going on with your copy (this is “verbose” and will show you why files fail if they do fail. Give yourself a log - /log+:”[logpath+name.txt]” (the + will mean that logs are appended to rather than overwritten if things go belly up).

Oh - and edit - for fun - if you want to put your log into a folder, this folder has to exist - so Robocopy will replicate existing folders, but won’t create a new one for logs. But logs are great.

There are other things you can do, but this should cover you for this.

You can also write this into a batch file and set it for regular Windows scheduling if you want, but this should do for what you want.

Lots of people use extra switches in Robocopy like /dat, but these are already built in as default, and pointless unless you’re trying to show integrity for data copying many years later from a forensic standpoint.

Give it a go.

28064212

Try:

robocopy "C:\Users\john doe\Dropbox" "D:\john doe backup" /mt /z

You also might want to add the /mir switch, but make sure you know what it does and test it before using live data. https://ss64.com/nt/robocopy.html

SnowyMay

28064212 wrote: »

Try:

robocopy "C:\Users\john doe\Dropbox" "D:\john doe backup" /mt /z

You also might want to add the /mir switch, but make sure you know what it does and test it before using live data. https://ss64.com/nt/robocopy.html

Noooo. No mirror!! That switch can just feck you over if you don’t need and know it. Also, mirror has a strange thing in Robocopy where, if there are hidden files, it can copy those hidden attributes to the entire copy job. Ok, can be resolved, but it’s an unnecessary pain in this situation.

OP - mirror not!!

Edit to add - put in something like /W:5 and /R:5 so that the copy doesn’t run forever if it gets stuck (so, wait five seconds, retry five times). The default with RC is one million retries with a 30 second interval. That takes a while if it gets stuck.

lordsheepface

Will give these a bash and take on board those /mir comments.
Will report back.

Thanks for the suggestions.

lordsheepface

Looks like this went well for me

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :     13533     13533         0         0         0         0
   Files :    214867    214867         0         0         0         0
   Bytes : 622.999 g 622.999 g         0         0         0         0
   Times :  19:51:42   1:58:27                       0:00:00   0:14:55


   Speed :            94122108 Bytes/sec.
   Speed :            5385.710 MegaBytes/min.
   Ended : Monday 7 December 2020 23:44:17

Thanks for the help.

65535

There's a ROBOCOPY GUI called Sync Toy

https://www.microsoft.com/en-us/download/details.aspx?id=15155