[PaulBrewer]

I've just been speaking to one of the magazines about the quality of equipment reviews.
We both feel they're all too vague, perhaps like the equipment they're reviewing !

We've come up with the idea of marking units performance out of 100 against an Industry standard (or standards) which stands at 50.

So each item can be plus or minus in relation to the standard's 50 points.

For example, an Sm57 is clearly a standard, but I've often heard the Audix i5 being mentioned around here as being superior, so it might score 65 or whatever.

Similarly an cheap vocal condenser might score 20 against a U87.

To be Topical from another thread, a Behringer DI might score 10 against a BSS.

So instead of comparing stuff to 'Good for the price' you compare to a quality.

'If I spend X I can get to half an agreed quality but if I spend Y I can get to 80% of the standard'

Would that enable guys to make better and easier decisions regarding gear?

This is literally a first draft idea so I'd be interested in feedback on it.

[ICN]

Thats 1/2 of it alright..

Confidence for me is knowing that the source of the review is Kosher.

Its funny how you hardly ever see a really bad review of anything.. unless its "Budget" or made by a fledgling company. Easy targets.


Independant reviews are the best.. I'd always do a few Google reviews & find out the word on the Forums about all the OS conflicts / bugs etc.. Basically all the stuff that doesn't get printed.

Can't totally trust a review from some Guy who is on the payroll.

If you could get trusted independent reviews with your system.. then you'd be on to a winner. If it was on the Net - Reviews / Reviewers could themselves be rated / have comments left in their profile.

Once you start going down the route of Trade Shows Shmoozatons, Press Kits & not wanting to offend Whoever.. then you are undoubtebly already in that "Bad Place".

And thats where most Reviewers / Editors probably are.

[bedbugs]

hey paul.

I always get the fear when I hear the term "industry standard". It really depends on what division of the industry yr talking about; film, theatre, rock bands, tv, classcal etc..

[PaulBrewer]

Originally Posted by ICN

Thats 1/2 of it alright..

Confidence for me is knowing that the source of the review is Kosher.

Its funny how you hardly ever see a really bad review of anything.. unless its "Budget" or made by a fledgling company. Easy targets.


Independant reviews are the best.. I'd always do a few Google reviews & find out the word on the Forums about all the OS conflicts / bugs etc.. Basically all the stuff that doesn't get printed.

Can't totally trust a review from some Guy who is on the payroll.

If you could get trusted independent reviews with your system.. then you'd be on to a winner. If it was on the Net - Reviews / Reviewers could themselves be rated / have comments left in their profile.

Once you start going down the route of Trade Shows Shmoozatons, Press Kits & not wanting to offend Whoever.. then you are undoubtebly already in that "Bad Place".

And thats where most Reviewers / Editors probably are.

Yes - but someone like George Shilling you know you can trust (if not agree) with his opinion because -
1. He's walked the walk, worked as an equipment user and made hit records
2. He has a long history in doing fair reviews.

But let's separate the reviewer from the review process and talk about the process alone for the moment.

[Seziertisch]

I think that when it comes to gear there is no magic bullet. Different gear performs differently in different situations. What makes it great in one, might make it not so great in another. Variety is the spice of life.

For example, on vocals certain mics will perform differently with different vocalists, sounding great in one case and not so great in another. If you happen to have one of those vocalists that mic x works well for, you might see "works great on male vocals" or something in the review. Whether this means works great on this particular male vocalist or male vocalists in general is a little difficult to say sometimes.

This where the Internet really comes to the fore, sure there is a lot of bull**** and bull****ters, gear rising and falling in popularity, a certain degree of skepticism is required, many grains of salt, but there is still solid info to be found from real people in real situations using (and sometimes not properly using) the gear.

Another problem with a standards comparison is that people have different aims and aspirations with recordings. If you are trying to make the next Darkside of the Moon a 58 and an M-box won't bring you too far, if you just want to do some recordings they are perfect.

[PaulBrewer]

Originally Posted by bedbugs

hey paul.

I always get the fear when I hear the term "industry standard". It really depends on what division of the industry yr talking about; film, theatre, rock bands, tv, classcal etc..

Why should you 'get the fear' about anything ?

There are industry standards, full stop !

You know there are, what's to be afraid of ?

Obviously if you're talking about a hand held mic boom it does not refer to rock'n'roll. And a 57 isn't going to play a part in a staged period drama.

In most elements that make up a recording studio there are standards - wouldn't comparisons to those benchmarks give you a better idea of what a review meant ?

[PaulBrewer]

Originally Posted by Seziertisch

I but there is still solid info to be found from real people in real situations using (and sometimes not properly using) the gear.

But isn't my suggestion better than the current style ? You compare something to something else people are probably familiar with ?

[trackmixstudio]

Ha!
Good luck with this.
"Hello, S0und on S0und"
"Hi. John from @lesis here. We are releasing a new firewire mixer and are thinking of running a full page inside front cover ad for 4 issues. We are sending over one for review but will need a copy of the review prior to committing to the ad."
"Yeah, sure John. Fire it over. We will give it a good review"

All industry magazines are driven by advertising and, as such, cannot be trusted when it comes to reviews. That has been discussed here before and it's not going to change.

[Welease]

Originally Posted by PaulBrewer

I've just been speaking to one of the magazines about the quality of equipment reviews.
We both feel they're all too vague, perhaps like the equipment they're reviewing !

We've come up with the idea of marking units performance out of 100 against an Industry standard (or standards) which stands at 50.

So each item can be plus or minus in relation to the standard's 50 points.

For example, an Sm57 is clearly a standard, but I've often heard the Audix i5 being mentioned around here as being superior, so it might score 65 or whatever.

Similarly an cheap vocal condenser might score 20 against a U87.

To be Topical from another thread, a Behringer DI might score 10 against a BSS.

So instead of comparing stuff to 'Good for the price' you compare to a quality.

'If I spend X I can get to half an agreed quality but if I spend Y I can get to 80% of the standard'

Would that enable guys to make better and easier decisions regarding gear?

This is literally a first draft idea so I'd be interested in feedback on it.

From a purely process point of view (ignoring that the score will always subjective and based on application)... I'm not sure your scale (if i am reading it right) would make any sense..

SM57 is assigned 50 (it would be a nightmare in itself to decide (and agree) what *is* the standard). It costs 100 quid..

What would a slightly better sounding 1000 quid mic score? 60 cos it's sounds better?... 20 cos its 10 times the cost? 40 cos its better but costs a load more? Would someone reading that score be able to decipher why it got that score, without reading the full article and explanation in which case why even bother scoring it like that..

From my experience, people are used to a 1-10 (or similar) scale and understand what it respresents, any new scale would require too much explanation to the casual user to be of any real value. Any complication of the scoring mechanism just distracts the reader from the detail they are actually interested in.

Sorry

[Seziertisch]

Originally Posted by PaulBrewer

But isn't my suggestion better than the current style ? You compare something to something else people are probably familiar with ?

The U47 is an industry standard, but I have never even seen one let alone used one.

The SM57. Mic x sounds better than a SM57 on vocals. With the vocalists we tested this mic sounded better than a 57. Again, it doesn't tell me much.

I was recording last weekend. For the guitar sound we were after a V72 beat the competition (Neve, Daking, API, Tubetech) out the gate. The others were all undesirable for different reasons in that particular situation. I have been in other situations where a V72 was similarly uncomplimenatary to other sounds, lacking the crispness and fast transients of the API or whatever. I think to be able to offer an accurate overview of a piece of gear, it needs to be used everyday in real life scenarios (which some reviews do involve) but ultimately opinions are still going to vary.

[PaulBrewer]

Originally Posted by trackmixstudio

Ha!
Good luck with this.
"Hello, S0und on S0und"
"Hi. John from @lesis here. We are releasing a new firewire mixer and are thinking of running a full page inside front cover ad for 4 issues. We are sending over one for review but will need a copy of the review prior to committing to the ad."
"Yeah, sure John. Fire it over. We will give it a good review"

All industry magazines are driven by advertising and, as such, cannot be trusted when it comes to reviews. That has been discussed here before and it's not going to change.

Ah ole SOS !

I agree with what you say (in fact it's what's driven this idea) but I think you'll also agree that Resolution's Speaker reviews for example are primarily technical, and stuff gets hung out to dry there regularly.

All industry mags are of course driven by ads , but that doesn't mean it is, or should be lies.

Any regular reader will also know who to trust , Mr. Shilling is my example.





Why did you put the 0 instead of the O Michael ?

[PaulBrewer]

Originally Posted by Welease

From a purely process point of view (ignoring that the score will always subjective and based on application)... I'm not sure your scale (if i am reading it right) would make any sense..

SM57 is assigned 50 (it would be a nightmare in itself to decide (and agree) what *is* the standard). It costs 100 quid..

What would a slightly better sounding 1000 quid mic score? 60 cos it's sounds better?... 20 cos its 10 times the cost? 40 cos its better but costs a load more? Would someone reading that score be able to decipher why it got that score, without reading the full article and explanation in which case why even bother scoring it like that..

From my experience, people are used to a 1-10 (or similar) scale and understand what it respresents, any new scale would require too much explanation to the casual user to be of any real value. Any complication of the scoring mechanism just distracts the reader from the detail they are actually interested in.

Sorry

Ah Head - if the scaling system is too difficult to understand there's no hope for any of us !

[PaulBrewer]

Originally Posted by Seziertisch

The U47 is an industry standard, but I have never even seen one let alone used one.

The SM57. Mic x sounds better than a SM57 on vocals. With the vocalists we tested this mic sounded better than a 57. Again, it doesn't tell me much.

I was recording last weekend. For the guitar sound we were after a V72 beat the competition (Neve, Daking, API, Tubetech) out the gate. The others were all undesirable for different reasons in that particular situation. I have been in other situations where a V72 was similarly uncomplimenatary to other sounds, lacking the crispness and fast transients of the API or whatever. I think to be able to offer an accurate overview of a piece of gear, it needs to be used everyday in real life scenarios (which some reviews do involve) but ultimately opinions are still going to vary.

I think it would be primarily of use in the lower end of the market where differences can be more marked.

I agree with your points regarding the upper end - as I've said here before virtually ALL the posh stuff sounds 'good' but different 'good'.

There would be no value in comparing an SSL to a Neve.
But there could be in comparing a Mackie to a Neve.

Especially for those who don't have access to one.

[PaulBrewer]

Originally Posted by Welease

What would a slightly better sounding 1000 quid mic score? 60 cos it's sounds better?...

Exactly !

[bedbugs]

Originally Posted by PaulBrewer

In most elements that make up a recording studio there are standards - wouldn't comparisons to those benchmarks give you a better idea of what a review meant ?

No. Because it'd have to delve into what application the mic was being used for. If your review idea is for a specific aspect of audio recording, fair enough, but I like the reviews in mags like SOS because they give me a general idea of different applications for each piece of kit that's reviewed.