Command to recursively count files of a certain extension

steveland?

Hey, I need to know is there a comman to count the amount of files within a directory with a certain extension...

I've used ls -R /folder > ~/files.txt to list all the files in the folder and any subfolders

What I want is a command that'll count all the files *.mp3 or *.avi in a folder (and all it's subfolders) for me...

Anyone have any ideas?

Crumbs

Something like

ls -alR | grep "\.mp3" | wc -l

should do the job.

thos

use find
find /path -name '*.mp3'

Khannie

Crumbs wrote:

Something like

ls -alR | grep "\.mp3" | wc -l

should do the job.

problem.pwnt();

That's one of those elegant "unix > windows" solutions.

I have a question on this (since the original question has been answered, I'm hijacking)....


Why does

ls -alR *.mp3

not work the same way as

ls -alR | grep "\.mp3"

?

krinDar

Khannie wrote:

Why does

ls -alR *.mp3

not work the same way as

ls -alR | grep "\.mp3"

?

In the first example, the shell uses *.mp3 as a pattern, and expands to match all the files it can find and those filenames are then passed to 'ls -alR' as arguments.

Eg - Assume you have a directory as follows:

./one.mp3
./two.mp3
./a/a1.mp3
./a/a2.mp3
./b/b1.mp3
./something
./file1
./file3
./x/file4

When you issues the command 'ls -alR *.mp3' what happens is the shell expands *.mp3 to the filenames (in the current directory as the pattern does not match directory slashes) and passes that to the ls, so in effect the command you are using is:

ls -alR one.mp3 two.mp3

When you do ls -alR however it lists everything in the current directory and in all the directories in the current directory. All of these are piped into grep which looks for anything with '.mp3' in it.

Khannie

Ah yes....forgot for a second there how the shell interprets the wildcards first then hands over to the program. Nice answer.

steveland?

Cheers folks

Khannie

More hijacking. Curiosity's getting the better of me.

What if you did ls -alR *.asdf

Output is "no such file or directory". If the shell can't match, does it just hand over the original argument untouched?

niallb

No. It should still output the correct expansion which is an empty string.
The "no such file or directory" is on standard error, not standard output.

Try ls -alR *.asdf | wc -l and see what you get. 0 lines of output.

Send std error to /dev/null for a clearer picture ls -alR *.asdf 2>/dev/null

NiallB

krinDar

Khannie wrote:

What if you did ls -alR *.asdf

Output is "no such file or directory". If the shell can't match, does it just hand over the original argument untouched?

Typically that is what it does. In bash you can also configure the shell to remove the pattern if it does not match anything. The option is nullgob and it should be set to on:

shopt -s nullglob
(shopt -u nullglob to disable)

ls does not know about patterns, so it looks for a file called '*.asdf'.

Khannie

Thanks for the replies lads.

niallb wrote:

It should still output the correct expansion which is an empty string.

Not so methinks......

That would then be the equivalent of:

ls -alR

Catch_22

thos wrote:

use find
find /path -name '*.mp3'

find does NOT support regluar expressions to -name.
you would have to find |grep but as said above the ls is a much nicer solution.

krinDar

Catch_22 wrote:

find does NOT support regluar expressions to -name.
you would have to find |grep but as said above the ls is a much nicer solution.

You are correct, it does not support regular expressions, but it does support shell patterns which is what you have quoted. In other words, what you quoted will work properly.

niallb

Khannie wrote:

Thanks for the replies lads.
Originally Posted by niallb
It should still output the correct expansion which is an empty string.

Not so methinks......
That would then be the equivalent of:
ls -alR

Oh, methinks so!
The shell expansion is performed first and than handed off to
ls.
This is why it is not the equivalent of a plain ls -alR
Without argument, ls is implicitly listing the current directory, . .

The wildcard is expanded by the shell and handed as a list of
filenames to the ls command, which then performs its listing on them.

When such an expansion is empty, you will get an empty listing.

You'll see evidence of this if you create a directory
and overpopulate it with files. The maximum depends on
your platform, and your filesystem.

Such a directory is a great tool for demonstrating where the shell breaks.
An ls will work. An ls * will not. Older systems show this
problem up much more conveniently ;-) (see also: xargs

This is one of the great things about un*x.
All the solutions posted here work, but there are
philosophical differences in how and why each command is constructed,
and each one will suit different people in different circumstances.
They have personality.

cat is smarter than mouse.

NiallB

krinDar

niallb wrote:

The shell expansion is performed first and than handed off to
ls.
This is why it is not the equivalent of a plain ls -alR
Without argument, ls is implicitly listing the current directory, . .

Correct.

The wildcard is expanded by the shell and handed as a list of
filenames to the ls command, which then performs its listing on them.

Correct.

When such an expansion is empty, you will get an empty listing.

Yes ... and no....ish. To quote the bash man page:

http://www.gnu.org/software/bash/manual/bashref.html#SEC34

Bash scans each word for the characters `*', `?', and `['. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of file names matching the pattern. If no matching file names are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.

So, if the glob provided does not match a filename, which I would consider to be an "empty" expansion, then the word is left unchanged. This is then passed as an argument to what ever the shell is exec'ing, in this case, the ls.

To test this, consider this. If you run the command:

echo /*

you get the list of files in the root directory. The shell expands the glob and this is passed as an argument to the echo command.

If you are correct, and you execute the command:

echo /*.fooo

then you will get no output. I say that you will get the output

/*.fooo

This is of course assuming that you do not have a file in the root ending in .fooo and that the shell option nullglob is not set.

If, however, the shell option nullglob is set, then the word is removed, so the argument is removed.

This would mean that

ls -laR *.asdf

would be the same as a

ls -laR

i.e without an argument, or at least it would produce the same output.

niallb

I think the difference is this however:
When you give no argument at all to ls, there is an implicit argument of '.'
With an empty shell expansion, that implicit argument is omitted,
and the contents of the directory are not listed.

That said, I just turned nullglob on, and it does indeed list the current directory
if no match is returned. That's useful to know!

NiallB

thos

Catch_22 wrote:

find does NOT support regluar expressions to -name.
you would have to find |grep but as said above the ls is a much nicer solution.

You've already been correct on the find statement, but I would really argue 'ls' being a better solution for anything. As pointed out already, it's the 'windows' way of thinking.

Depending on the size of the system, frequency of scan etc. One of the better and less intrusive ways would be through the use of updatedb/locate. The updatedb indexes the system, run it from cron whenever system isnt busy (middle of night), and then just use 'locate' during the day - so you're checking against a database index as opposed to traversing the whole system.

krinDar

niallb wrote:

I think the difference is this however:
When you give no argument at all to ls, there is an implicit argument of '.'

Not really. If there is no filename/path argument at all to ls, it lists the current directory. You might call it implicit, but that implies that the shell supplies it to ls as an argument, which is not the case.

With an empty shell expansion, that implicit argument is omitted,
and the contents of the directory are not listed.

No. As I pointed out in my previous post, when the shell pattern/glob does not match anything, i.e an "empty shell expansion" the glob/pattern is left as it is and passed to ls as a filename/path argument. This is why ls does not list the current directory.