Number of Participants vs. Number of Answers

GTE

Hello,

I am doing some research into sound where a listening test is used to get the participants preferred way of creating the sound.

- There are three ways used per musical piece.
- There are two musical pieces used of differing style used
- Each musical piece has two extracts from it to ensure answers are consistent.

These considerations result each preference question being asked 4 times per participant.

Musical Piece 1, section 1
A v B
A v C
B v C

Musical Piece 1, section 2
A v B
A v C
B v C

Musical Piece 2, section 1
A v B
A v C
B v C

Musical Piece 2, section 2
A v B
A v C
B v C


I have had 9 participants so for and I was advised to get 20 so I can have statistically valid results. My supervisor is away for a few days so I wanted to see if I could get an insight while I had this thought in my head.

Getting to 20 participants is proving to be difficult in the time frame. I have noticed that the total answers of a question is actually 36 since the A v B etc. is being repeated in the way described above. Could this be taken as being statistically valid?

I would imagine that it could be seen that way if every A v B question came out with the same answer for each participant, which is probably very implausible. I will check that when I can.

Anyway, just a thought.

Thanks for reading!

SmallTeapot

Tl;dr (Sorry! )...but I think I have the gist.

Will you be running inferential analysis with your data? (I.e. SPSS, R etc.)

I was always advised as a rule of thumb to have 25 participants per condition. So if I had 3 conditions, I was advised to recruit ~75 people. Reasons why: hopes of reaching significance; was not as restricted in terms of what stats I could conduct; can afford to disregard outliers. (Just as an fyi, I use SPSS and sometimes AMOS)

Do you have several conditions? Or do you have no grouping and are going to conduct a 'split' of some sort (something which is doable in quant research, but generally frowned upon tbh, unless really justified).


Sorry for all the qs, I just need a little bit more info before advising as I'd wouldn't want to steer you wrong.

Black Swan

bbk wrote: »

I have had 9 participants so for and I was advised to get 20 so I can have statistically valid results. My supervisor is away for a few days so I wanted to see if I could get an insight while I had this thought in my head.


Getting to 20 participants is proving to be difficult in the time frame.

Given that I do not have all the necessary details regarding your research questions and design (caution):

An n=20 is a very small sample size, and as a rule-of-thumb, minimal for running nonparametric analysis. I don't have the citations here in this javahouse, but we typically use a minimum of n=30 for parametric analysis (dependent upon the design, which may or may not require a larger sample size). Of course, if you were pilot testing an instrument, Isaac and Michaels suggests that you may use as small as n=10, but not for the main study where a larger sample would be needed.

It should be noted that small sample sizes tend to increase the likelihood of accepting the null hypothesis; i.e., insignificance.

GTE

SmallTeapot wrote: »

Tl;dr (Sorry! )...but I think I have the gist.

Will you be running inferential analysis with your data? (I.e. SPSS, R etc.)

I was always advised as a rule of thumb to have 25 participants per condition. So if I had 3 conditions, I was advised to recruit ~75 people. Reasons why: hopes of reaching significance; was not as restricted in terms of what stats I could conduct; can afford to disregard outliers. (Just as an fyi, I use SPSS and sometimes AMOS)

Do you have several conditions? Or do you have no grouping and are going to conduct a 'split' of some sort (something which is doable in quant research, but generally frowned upon tbh, unless really justified).


Sorry for all the qs, I just need a little bit more info before advising as I'd wouldn't want to steer you wrong.

The questions are no bother, it is a new area to me and I am sure I am leaving out important stuff! :P

Inferential Analysis
After looking it up, yes I will be.

If I get 20 people to take part, by looking at their answers the results will show whether method A, B or C is the most preferred. There are four additional questions in each section where attributes of what participants are hearing will be tested to see what they think about that, which can help shed light on the justification of why a method is better then another, if that happens of course.

Grouping

All twelve sections are a paired comparison with five questions. The first being preference, the rest being specific attributes. All questions are the same for each section.

There are twelve sections because three methods are getting judged together twice for each song and there are two songs. ((3+3)+(3+3))

So, A1 is the section 1 of the first piece. 2A1 is section 2 of the first piece. Perhaps this is a confusing way to name it, but it has worked so far in Excel.

A1 v B1, A1 v C1, B1 v C1
2A1 v 2B1, 2A1 v 2C1, 2B1 v 2C1

A2 v B2, A2 v C2, B2 v C2
2A2 v 2B2, 2A2 v 2C2, 2B2 v 2C2

Thanks!

GTE

Black Swan wrote: »

Given that I do not have all the necessary details regarding your research questions and design (caution):

An n=20 is a very small sample size, and as a rule-of-thumb, minimal for running nonparametric analysis. I don't have the citations here in this javahouse, but we typically use a minimum of n=30 for parametric analysis (dependent upon the design, which may or may not require a larger sample size). Of course, if you were pilot testing an instrument, Isaac and Michaels suggests that you may use as small as n=10, but not for the main study where a larger sample would be needed.

It should be noted that small sample sizes tend to increase the likelihood of accepting the null hypothesis; i.e., insignificance.

For the purposes of the MSc and its time frame and judging from other tests from the Audio Engineering Society journal, it looks like a sample of 20 is what people aim for.

I have been advised to go for 20. For each question, I have 18 answers from 9 participants as each question is asked more than once so I am wondering if there is a disaster and I can only ever get 9 participants, can the fact that I have a 18 answers actually mean that I can take some significant from it?

Really, it is a small "what if" question which I am sure I can get an answer to eventually when my lecturer pops up again.

From what you said about the increase in null hypothesis likelihood, maybe that is why that each question was advised to be asked twice. A way of knowing if a participants results would be to see how they answered the same question the second time.

Thanks for the help, I do appreciate that I could be leaving things out however I am wary of writing an essay of unrelated information.

GTE

I had a meeting with a stats guy at the Uni and said that the Thurstone Law of Comparative Judgment is what I should use.

Doing some research now to see how I can implement it in a less mind melting way then a lot of the papers are showing it.