Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Hi all,
Vanilla are planning an update to the site on April 24th (next Wednesday). It is a major PHP8 update which is expected to boost performance across the site. The site will be down from 7pm and it is expected to take about an hour to complete. We appreciate your patience during the update.
Thanks all.

Filename too long when copying to External USB

  • 07-12-2020 6:45pm
    #1
    Registered Users Posts: 231 ✭✭


    I have Windows 10. I'm trying to copy a folder to an external USB. I'd like to preserve the folder / directory structure.

    When I copy using XCOPY it seems to run out of memory. When I copy using regular Windows copy and paste it comes up with an error of the filename being too long at some point (I didn't screenshot the error)

    Is there a better way to do this? The copy obviously takes 3 or 4 hours so I want to be able to leave it running without having to check for errors.

    This folder is 623GB in total and contains 214,867 files and 13,532 subfolders.

    Thanks for your help


Comments

  • Registered Users Posts: 39 omegab


    Robocopy

    Also maybe you've created a longer folder to put them into on the USB drive? So that you then go beyond the full length:
    https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation


  • Closed Accounts Posts: 544 ✭✭✭SnowyMay


    I have Windows 10. I'm trying to copy a folder to an external USB. I'd like to preserve the folder / directory structure.

    When I copy using XCOPY it seems to run out of memory. When I copy using regular Windows copy and paste it comes up with an error of the filename being too long at some point (I didn't screenshot the error)

    Is there a better way to do this? The copy obviously takes 3 or 4 hours so I want to be able to leave it running without having to check for errors.

    This folder is 623GB in total and contains 214,867 files and 13,532 subfolders.

    Thanks for your help

    Ok - stupid question, but you do have enough space on the destination drive, yeah?

    Have you tried Robocopy? It can just be run as a cmd, and will maintain folder structure, metadata, and isn't particularly intense.

    Can give you some basic info and switches if you aren't familiar with it.


  • Registered Users Posts: 231 ✭✭lordsheepface


    Thanks for the replies. I can't say I fully understand it all but I have been using this very successfully for some time (saved in a .BAT file).

    xcopy "C:\Users\john doe\Dropbox\*" "d:\john doe backup\" /E /C /w

    With the result being that all files in my "c:\Users\john doe\dropbox" folder are copied into my "d:\john doe backup\" folder and if "d:\john doe backup\" does not exist it creates it.

    My external USB is brand new. It's formatted NTFS and has 931GB free space.

    So now I want to try and duplicate my xcopy but using robocopy. I've gone with this:

    robocopy "C:\Users\john doe\Dropbox\*" "D:\john doe backup\" /mt /z

    It seems my source destination is an invalid paramter?

    Ideally I just want to run a .BAT file to copy this folder and for it to skip any errors and tell me at the end what the errors were. I don't want to have to have any interaction during the long copy.

    My normal process would be that I would change the D:\ folder and it would be "john doe backup 1 " and then "john doe backup 2" etc. Normally I would pick and chose specific source directory but this time I want the hole full dropbox one. This is the first time I've tried to copy a folder this large or this many files / subfolders.

    Appreciate any help.


  • Registered Users Posts: 231 ✭✭lordsheepface


    omegab wrote: »
    Robocopy

    Also maybe you've created a longer folder to put them into on the USB drive? So that you then go beyond the full length:
    https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

    I just see your edit now.

    Surely copying from c-users-USERNAME\Dropbox to D\USERNAME\ actually creates a shorter file path than existed beforehand? Or am I misunderstanding you

    (I've had to remove some colons from the directorys above as the forum software doesn't seem to like them!)


  • Closed Accounts Posts: 544 ✭✭✭SnowyMay


    Thanks for the replies. I can't say I fully understand it all but I have been using this very successfully for some time (saved in a .BAT file).

    xcopy "C:\Users\john doe\Dropbox\*" "d:\john doe backup\" /E /C /w

    With the result being that all files in my "c:\Users\john doe\dropbox" folder are copied into my "d:\john doe backup\" folder and if "d:\john doe backup\" does not exist it creates it.

    My external USB is brand new. It's formatted NTFS and has 931GB free space.

    So now I want to try and duplicate my xcopy but using robocopy. I've gone with this:

    robocopy "C:\Users\john doe\Dropbox\*" "D:\john doe backup\" /mt /z

    It seems my source destination is an invalid paramter?

    Ideally I just want to run a .BAT file to copy this folder and for it to skip any errors and tell me at the end what the errors were. I don't want to have to have any interaction during the long copy.

    My normal process would be that I would change the D:\ folder and it would be "john doe backup 1 " and then "john doe backup 2" etc. Normally I would pick and chose specific source directory but this time I want the hole full dropbox one. This is the first time I've tried to copy a folder this large or this many files / subfolders.

    Appreciate any help.

    Ok - wiser people than me will maybe hop in, but the asterisks in your source path should be removed. Also, don’t leave either paths with a trailing backslash. Robocopy is awesome, but sticks to his rules, and can get confused.

    /Mt will give you multithreading to the power of 8 threads at a time (you can set higher if you wish, but I personally wouldn’t be bothered for this size of data) and /z is useful for restarting if something fails for some reason, but consider adding /e (for subfolders including empty ones), /v for having a clue of what’s going on with your copy (this is “verbose” and will show you why files fail if they do fail. Give yourself a log - /log+:”[logpath+name.txt]” (the + will mean that logs are appended to rather than overwritten if things go belly up).

    Oh - and edit - for fun - if you want to put your log into a folder, this folder has to exist - so Robocopy will replicate existing folders, but won’t create a new one for logs. But logs are great. :)

    There are other things you can do, but this should cover you for this.

    You can also write this into a batch file and set it for regular Windows scheduling if you want, but this should do for what you want.

    Lots of people use extra switches in Robocopy like /dat, but these are already built in as default, and pointless unless you’re trying to show integrity for data copying many years later from a forensic standpoint.

    Give it a go.


  • Advertisement
  • Registered Users Posts: 10,457 ✭✭✭✭28064212


    Try:
    robocopy "C:\Users\john doe\Dropbox" "D:\john doe backup" /mt /z
    
    You also might want to add the /mir switch, but make sure you know what it does and test it before using live data. https://ss64.com/nt/robocopy.html

    Boardsie Enhancement Suite - a browser extension to make using Boards on desktop a better experience (includes full-width display, keyboard shortcuts, and dark mode). Now available through the extension stores

    Firefox: https://addons.mozilla.org/addon/boardsie-enhancement-suite/

    Chrome/Edge/Opera: https://chromewebstore.google.com/detail/boardsie-enhancement-suit/bbgnmnfagihoohjkofdnofcfmkpdmmce



  • Closed Accounts Posts: 544 ✭✭✭SnowyMay


    28064212 wrote: »
    Try:
    robocopy "C:\Users\john doe\Dropbox" "D:\john doe backup" /mt /z
    
    You also might want to add the /mir switch, but make sure you know what it does and test it before using live data. https://ss64.com/nt/robocopy.html

    Noooo. No mirror!! That switch can just feck you over if you don’t need and know it. Also, mirror has a strange thing in Robocopy where, if there are hidden files, it can copy those hidden attributes to the entire copy job. Ok, can be resolved, but it’s an unnecessary pain in this situation.

    OP - mirror not!!

    Edit to add - put in something like /W:5 and /R:5 so that the copy doesn’t run forever if it gets stuck (so, wait five seconds, retry five times). The default with RC is one million retries with a 30 second interval. That takes a while if it gets stuck.


  • Registered Users Posts: 231 ✭✭lordsheepface


    Will give these a bash and take on board those /mir comments.
    Will report back.

    Thanks for the suggestions.


  • Registered Users Posts: 231 ✭✭lordsheepface


    Looks like this went well for me
                   Total    Copied   Skipped  Mismatch    FAILED    Extras
        Dirs :     13533     13533         0         0         0         0
       Files :    214867    214867         0         0         0         0
       Bytes : 622.999 g 622.999 g         0         0         0         0
       Times :  19:51:42   1:58:27                       0:00:00   0:14:55
    
    
       Speed :            94122108 Bytes/sec.
       Speed :            5385.710 MegaBytes/min.
       Ended : Monday 7 December 2020 23:44:17
    

    Thanks for the help.


  • Registered Users Posts: 782 ✭✭✭65535




  • Advertisement
Advertisement