Directory recursion to delete files of a certain age

dcGT

Hi All,
I'm looking to write a script to delete files directory over 20 days old. I was just wondering what would be the best way to implement this. Would it be best to use cut and sed on the date of the file or perhaps compare the date of the file with the current date?
Maybe using find to find all files of a certain date would be an option?

Any help would be great,

Cheers,

DC

Hagar

I'm not a Unix head but beware your script doesn't delete directories older than 20 days as well. AFAIK a directory is a file and would show up as a file on a grep piped command.

ishnid

You should be able to do it easily enough with a quick Perl script - something like this maybe (UNTESTED):

#!/usr/bin/perl -w
use strict;
use File::Find;
                                                                                
my $dir = '/path/to/start/dir';
                                                                                
find ( sub {

   # check that it's a regular file and it's more than 20 days old
   if ( -f $File::Find::name && -M $File::Find::name > 20 ) {

      # delete it if so
      unlink $File::Find::name;
   }
}, $dir );

davej

This is a fairly routine thing to do using find:

[PHP]find / -type f -mtime +20 -exec rm -f {} \;[/PHP]

This will delete all files (not directories) older than 20 days

If you want to clean up empty directories, execute this after the first command:

[PHP]find / -type d -mtime +20 -empty -exec rmdir {} \;[/PHP]

davej

dcGT

Thanks people,
davej, I have been looking at mtime but wasn't sure of its usage.

Cheers!

DC.

dcGT

@davej,

With regard to the 'mtime' switch, what 'time' does this refer to, I see it is for 'modification' time, but with regard to an FTP server, would be the time the file was placed on the FTP server or when the actual file was last modified?
Just checking,

Thanks again,

DC.

krinDar

dcGT wrote:

With regard to the 'mtime' switch, what 'time' does this refer to, I see it is for 'modification' time, but with regard to an FTP server, would be the time the file was placed on the FTP server or when the actual file was last modified?

Modification time is when the file was changed by write (and some other
commands). When you ftp the file, the ftp server will create a file, and
then write the file to disk. Thus the modification time would be when you
ftp the file to the server. If after that you edited the file and wrote those
changes, then that too would update the modification time.

ctime, or change time refers to the same as modification time, along with
when the permissions were changed, it was renamed etc ...

The manpage for stat(2) defines these.

davej

what krinDar says is correct...

Just a note on the commands i listed above, the php code tag seems to have removed a slash at the end of the commands (need to escape the ';')

find / -type f -mtime +20 -exec rm -f {} \;
find / -type d -mtime +20 -empty -exec rmdir {} \;

davej

ishnid

VBulletin's php tags do that alright. The plain old [code] tag is better.

davej

VBulletin's php tags do that alright. The plain old [code] tag is better.

Ahh right. Thanks I'll remember that for the next time.

davej

dcGT

davej wrote:

This is a fairly routine thing to do using find:

[PHP]find / -type f -mtime +20 -exec rm -f {} \;[/PHP]

This will delete all files (not directories) older than 20 days

If you want to clean up empty directories, execute this after the first command:

[PHP]find / -type d -mtime +20 -empty -exec rmdir {} \;[/PHP]

davej

I was just looking into this, and I've realised that there are hidden files that will be deleted using this command:

find / -type f -mtime +20 -exec rm -f {} \;

Like '.bashrc' for example. Is there a way to find all files that ARE NOT hidden and delete just those files?

Thanks again,

DC.

davej

ok. To ignore "hidden" files (ones with a leading '.' ):

find / \( -name '.*' -prune \) -o \( -type f -mtime +20 \) -exec rm -f {} \;

Note spacing after opening / before closing brackets..

Regs,

davej

dcGT

davej wrote:

ok. To ignore "hidden" files (ones with a leading '.' ):

find / \( -name '.*' -prune \) -o \( -type f -mtime +20 \) -exec rm -f {} \;

Note spacing after opening / before closing brackets..

Regs,

davej

That's perfect davej. Thanks a lot! I just have to try to figure out how that works now!

DC.

dcGT

davej wrote:

If you want to clean up empty directories, execute this after the first command:

[PHP]find / -type d -mtime +20 -empty -exec rmdir {} \;[/PHP]

davej

@davej: Is there a way to only remove empty directories if they are, say, 4 levels down or more. For example, an empty directory in /var/tmp/1/2/ would be removed, but an empty directory in /var/tmp/1/ would not be. I've had a look at 'find' and the only thing that seems to be relevant is '-mindepth', but I don't think this refers to directory structures.

Thanks again,

DC.

davej

dcGT,

the mindepth argument is the right one to use!

find / -mindepth 4 -type d -mtime +20 -empty -exec rmdir {} \;

Good Luck....

davej

dcGT

Thanks davej, the first time I tried this switch, it didn't yield the results I expected. I must have executed the command incorrectly. I tried it again and it seems to work as expected now.

Cheers!

DC.

thesteve

Howdy lads, quick question... I'm a complete Perl newbie, had a three day introductory course a few weeks ago and am now basically trying to implement the same task as this thread (except to delete directories and their contained files, and not just the files themselves) but in a Windows enviroment. I know this is completely the wrong board but I though seeing as its the same topic/language....

Anyway the perl course I was on was on Knoppix, but isn't Perl a platform independant language? Can someone tell me why the code listed here won't work on a windows system (apart from the slashes, etc)... I've tried a number of scripts and have trawled the net but still can't seem to pull it off

hostyle

thesteve wrote:

Howdy lads, quick question... I'm a complete Perl newbie, had a three day introductory course a few weeks ago and am now basically trying to implement the same task as this thread (except to delete directories and their contained files, and not just the files themselves) but in a Windows enviroment. I know this is completely the wrong board but I though seeing as its the same topic/language....

Anyway the perl course I was on was on Knoppix, but isn't Perl a platform independant language? Can someone tell me why the code listed here won't work on a windows system (apart from the slashes, etc)... I've tried a number of scripts and have trawled the net but still can't seem to pull it off

What errors are you getting? isnid's script should do what you want with some modification.

thesteve

Apologies, I wrote that post while on the night shift and my brain wasn't working... I got it working now...

Thing is I got it working to delete files, and I also got it working to delete empty directories (as two seperate scripts). Is there a command I can use instead of rmdir to get the program to remove a directory whether its empty or not?

Cheers for the help.

hostyle

thesteve wrote:

Apologies, I wrote that post while on the night shift and my brain wasn't working... I got it working now...

Thing is I got it working to delete files, and I also got it working to delete empty directories (as two seperate scripts). Is there a command I can use instead of rmdir to get the program to remove a directory whether its empty or not?

Cheers for the help.

rm -rf but be very careful.

thesteve

Thanks

ishnid

If you're looking for pure-perl solution, this will remove any files over 20 days old and empty directories over 20 days old. The File::Find::Rule module returns the directory first, then the contained files. By reversing the list, we remove all the files first, so that the directory is empty when we try to delete it. Adapt it to your purposes (oh, and handle with care!):

#!/usr/bin/perl -w
use strict;
use File::Find::Rule;

my @files_and_dirs = File::Find::Rule->in( '/home/daithi/test/rmtest' );

for ( reverse @files_and_dirs ) {
   if ( -f ) {
      unlink if -M > 20;
   }
   else {
      rmdir if -M > 20;
   }
}

thesteve

Thanks again, much appreciated, I'm going to use this on production servers to clean up log files over a month old (a new folder is created every day that contains various log files, these are ftp'd everyday), I'll make sure to test it out first before scheduling it as a task...

ishnid

thesteve wrote:

I'll make sure to test it out first before scheduling it as a task...

Good idea