Can people tell the difference between stouts?

cavedave

A pub in town serves 4 types of stouts (Guinness, Murphys, Beamish and O'Haras). I have a hypothesis that people cannot identify a stout in a blind tasting.

I have figured out how to run the test but I need help working out the statistics. Based on peoples classifications how do I tell whether the hypothesis is confirmed or not?

So I will have a table of whether each person classified right

Person Guiness Beamish Murphys O'Haras
A 1 0 0 1
b 1 1 1 1
c 0 0 1 1
d 0 0 0 0

I expect to have about a dozen people trying to classify. Is this even enough to make a proper test?

Culchie

cavedave wrote: »

A pub in town serves 4 types of stouts (Guinness, Murphys, Beamish and O'Haras). I have a hypothesis that people cannot identify a stout in a blind tasting.

I have figured out how to run the test but I need help working out the statistics. Based on peoples classifications how do I tell whether the hypothesis is confirmed or not?

So I will have a table of whether each person classified right

Person Guiness Beamish Murphys O'Haras
A 1 0 0 1
b 1 1 1 1
c 0 0 1 1
d 0 0 0 0

I expect to have about a dozen people trying to classify. Is this even enough to make a proper test?

The mathematical part of your question is over my head I'm afraid, but 12 people is too small a sample to get an accurate survey.

If you tossed a coin 12 times, a 8/4 or even 9/3 result wouldn't be uncommon. I would suggest that you would need minimum 200 people to get a representative survey.

You then will have variables to factor. e.g Will a Corkman recognise Murphys, while a Leitrim man never saw or tasted a pint of Murphy's before, may say 'guinness' if he tastes it first.


A good survey would be to offer your blind tasting survey based on Ale, Stout, Lager and water to people who have consumed 3 pints or more.
I think you will get some very surprising results there, if my ancedotal evidence is anything to go by.;)

Procasinator

I'm not particularly good at maths/stats, but do remember about hearing about confidence intervals. which may help you. I personally don't think a sample size of a dozen is enough.

Culchie makes some great points to - such as a Cork man will be more likely to distinguish between Guiness and Murphy's. Other things to consider, such as age, gender, etc that you can simply not cover in such a small sample size.

Fremen

It might be an idea to narrow the choices down to two beers at a time. Say you give everyone two beers out of the four you listed, instead of all four.

Knowing which two out of the four they got, your test subjects then have to decide which one is which. They get one point for a right answer and lose one for a wrong answer. Over the course of a year, drinking two pints a night each, if the group's score is significantly higher than zero, you'll know you can taste the difference. Plus, you get to spend lots of time in the pub in the interests of science.

This might not be a bad idea anyway, since they all taste the same once you've had a few.

cavedave

Thanks for the advice everyone. Ill write up the "experiment" and post what happened here

MathsManiac

When I contributed to your question in the other forum, where a chi-squared test had been proposed, I didn't realise you were talking about such a small sample.

Given such a small data set, you could examine the probabilities using exact combinatorial methods. For example, you could calculate the total of the scores achieved by all of your subjects. You could then calculate, by exact methods, the probability that this or a greater score would be achieved by such a set of people guessing at random. If this probability is less than 0.05, then you could regard the result as significant at the 95% level.

Apart from such a quantitative analysis, you might just notice some interesting things that might prompt a further experiment. For example, you might see that one of these seems to be more distinguishable from the other three than they are from eachother. (i.e., that one of them is the one most likely to be got right by people.) You could do a follow-up experiment to test that hypothesis, preferably with a different bunch of people. Or say, for example, you notice that people never mix up Guinness and O'Hara's, but they sometimes mix either of these up with the other two, you could do a follow-up experiment using only these two to confirm that.

As Fremen points out, this has the potential for many years of fruitful research (or maybe not fruitful - but who cares!).

cavedave

As Fremen points out, this has the potential for many years of fruitful research (or maybe not fruitful - but who cares!).

Thanks for the reply.
I have decided to run the test with a small number of people on the grounds that I might mess up some of the blinding or something and do not want to waste a properly organised experiment. I will put up the results to see if anything leaps out and that can be tested for at some later stage anyway.

cavedave

Did the test last night

Person|Beamish|Guinness|Murphy’s|O’Hara’s|Score
A|Beamish|Guinness|Murphy’s|O’Hara’s|4
B|Guinness|Beamish|Murphy’s|O’Hara’s|2
C|O’Hara’s|Guinness|Beamish|Murphy’s|1
D|Guinness|O’Hara’s|Murphy’s|Beamish|1
E|Beamish|Murphy’s|O’Hara’s|Guinness|1
F|Guinness|Murphy’s|O’Hara’s|Beamish|0
G|Beamish|Guinness|O’Hara’s|Murphy’s|2
H|Guinness|Murphy’s|Beamish|O’Hara’s|1
I|Guinness|Beamish|O’Hara’s|Murphy’s|0

Did three rounds (as people turned up) of Person goes to bar brings A beer back, then B beer then C beer then D with the order randomised. People right down their answers and hand them in then discussed what they thought.

Now to analysis, the average right was 1.333. Which is not much above the 1 you would expect to get at random. Any other thoughts/analysis?

Grudaire

Whoever A was actually bought the pints and blinfolded themselves?

Other then that you don't need any statistical test to say that people don't know their stouts...

hivizman

Instead of analysing the results by reference to the tasters, you could look at the results for the four different stouts. There is a bit of a pattern here: the stouts fall into two groups - Beamish and Guinness (B&G) on the one hand and Murphy's and O'Hara's (M&O) on the other. Out of the 18 tastings of B&G, 13 tastings came up with either B or G (not necessarily the correct one), while only 5 came up with either M or O. The results are reversed for M&O: out of 18 tastings, 13 came up with either M or O, and only 5 with either B or G.

If you formalise this as the null hypothesis "drinkers are unable, given a specific stout, to tell whether it's either (a) Beamish or Guinness, or (b) Murphy's or O'Hara's", you can use the chi-squared test to see whether this result is statistically significant and thus allows you to reject the null hypothesis. If you can, then this would suggest that drinkers are on average able to narrow down their stouts to one of the two pairs, even if they aren't very good at distinguishing between the stouts within each pair.

I'm not a stout drinker, but I understand that both Murphy's and O'Hara's are brewed to be somewhat lighter and less bitter than Beamish and Guinness, so the tasting results could be consistent with this.

cavedave

ok so for two hypothesis I think i need
N -> number of possible outcomes
E-> expected number of outcomes
O -> observed number fo outcomes

So for Hypothesis 1. "People cannot tell 4 stouts apart" These would be
N=2 (expected outcomes are correct or wrong guess)
E=9
O=12

so by Pearson's chi-squared test
(((12-9)*(12-9))/9)+(((24-27)*(24-27))/27)=1.333

This has one degree of freedom.
The two-tailed P value equals 0.2482

For Hypothesis 2 "people can tell G&B from M&O'H"
N=2
E=9
O=13

so by Pearson's chi-squared test
(((13-9)*(13-9))/9)+(((5-9)*(5-9))/9)=3.555
This has one degree of freedom.
The two-tailed P value equals 0.0593

Is this right? and if so what does it mean?

*edited to include p values.

hivizman

I did it as a 2*2 table:


Correct|B&G|M&O
Choice||
B&G|13|5
M&O|5|13

If the choices are random, we would expect 9 in each of the four cells.

Calculating chi-squared, with one degree of freedom, and ignoring Yates' correction, I get a test statistic of 7.11, which has a probability of less than 0.01. This means that there is less than one chance in 100 that your results could have come from a population of people who simply named the stouts at random.

Culchie

goddam ... this thread keeps bringing back to the mathematics forum (20 years too late my mother would say!!!) and I know nothing about it!!

Are all the contestants male? Any breakdown on this?

cavedave

Culchie
goddam ... this thread keeps bringing back to the mathematics forum

Sorry I should have said this earlier. There is a write up on the test here.

If theres anything I have not explained right or if you have any ideas for other tests please reply.

Teg Veece

Interesting thread.

I'd like to see how giving people the option of using the same Stout name to identify more than one of the stouts would influence the results. Would this remove correlation between the pints? At the moment, unless the test subject knows all stouts then the highest they can score is 2/4 which would seem to skew the results.

So someone could say Guinness, Guiness, O'Hara, Murphys if they are 50:50 on the first two drinks.

Similarly, someone could say Guinness, Guiness,Guinness, Guiness if they have absolutely no idea and get 1/4 correct which is what you'd expect on the law of averages.

Not too sure if this would improve the accuracy of the test.
Any opinions?

Grudaire

Teg Veece wrote: »

Interesting thread.

I'd like to see how giving people the option of using the same Stout name to identify more than one of the stouts would influence the results. Would this remove correlation between the pints? At the moment, unless the test subject knows all stouts then the highest they can score is 2/4 which would seem to skew the results.

So someone could say Guinness, Guiness, O'Hara, Murphys if they are 50:50 on the first two drinks.

Similarly, someone could say Guinness, Guiness,Guinness, Guiness if they have absolutely no idea and get 1/4 correct which is what you'd expect on the law of averages.

Not too sure if this would improve the accuracy of the test.
Any opinions?

You make a valid point, as you are treating it as one test a persons opinion on one drink affects his/her opinion on the next..

cavedave

Teg Veece

At the moment, unless the test subject knows all stouts then the highest they can score is 2/4 which would seem to skew the results.

? They could still get 4/4 it is just not likely.