How to present survey data

Some_randomer

Hi just found this forum so hope my question is suitable. We're a tech startup and we're having trouble agreeing on a .com domain name for our main platform, so we decided to heed the wisdom of the crowd and put all the options in a survey to send to colleagues, friends, and family. Now we're not sure as to the best way to present all the options as none of us have any experience in research or gathering data.

I'm guessing people here have created questionnaires or collected data as part of their work or studies so would love to get some thoughts on this. We have about 50 different website names so if anyone has opinions on the questions below or generally the best way to present data such as this we're all ears:

- is it best to have a list where people can only select only one name, or should we have a score like 1 to 10 for each one?
- how do people respond to surveys i.e. if we have a score for each name will they get bored and just start clicking scores at random after the first few?
- would people be more biased towards the first name they see in a list of names, or the last? What's the best way to order / present the data?
- we're thinking of using either Google forms or surveymokey.com. Anyone have a preference, or know of a better platform for things like this?

All thoughts and feedback much appreciated, thanks in advance.

Black Swan

Some_randomer wrote: »

Hi just found this forum so hope my question is suitable.

You are welcome to ask questions here in our Researcher forum about survey platforms, questionnaire design, analysis, etc. I'm a bit tied up at the moment, and will visit this thread a day or two from now. Hopefully you will get some replies before I return.

ResearchPsych

Hi Some_randomer, 

I'll try to help with your questions here.

- is it best to have a list where people can only select only one name, or should we have a score like 1 to 10 for each one?

This really depends on what you are measuring, and on how you decide to measure it. If you want to have a graded response, where you want to measure how much they like each name, you could use a Likert scale with a range of scores, but this is obviously not necessary for yes/no questions (e.g. do you like name A Y/N), or for elimination processes where you ask them to select the name they like the least of each group of, say, 5 names.

- how do people respond to surveys i.e. if we have a score for each name will they get bored and just start clicking scores at random after the first few?

This is always a possibility, and so is central tendency. This is when people avoid selecting answers at the extreme ends of the range, and opt for the middle range, even if this doesn’t necessarily reflect what they really think. Another problem you may find is social desirability bias, which is when people answer what they think is socially acceptable, or what  they think you want to hear. There are steps you can take to try to control for these problems, but with your particular survey, I'm not sure what these would look like.

For social desirability bias, you can add a statement in the beginning to let the participant know that the answers will be anonymous (and make sure that they are), and to request them to answer as accurately as possible. Explaining why it is important to answer honestly could also help. For random responding and central tendency, you can build your questionnaire introducing the same question at different stages, with different phrasing, or asking the opposite question at different points, to see if responses match. However, building good questionnaires and reducing sources of measurement error takes training and skill, so perhaps you could consider consulting with a psychometrician. They can help you out with this. Even if this is not a psychometric survey, you may want to get ideas on how to structure the questionnaire, and how to ask the questions (e.g., you may not want to tell the participants the names only, but try to tease out different associations with each name).

- would people be more biased towards the first name they see in a list of names, or the last? What's the best way to order / present the data?

Both primacy and recency effect can bias responses, so there is no best order unfortunately. Rather, this is something you need to consider when you interpret your results. Again, building a strong questionnaire is important here, and I would suggest to consult with someone with experience in this area, to make sure that your survey introduces the least error of measurement possible. 

- we're thinking of using either Google forms or surveymokey.com. Anyone have a preference, or know of a better platform for things like this?

I used Google forms and tested Survey Monkey, and I liked them both. You just need to consider the costs, as Survey Monkey had a subscription fee, as far as I remember, and Google forms was free. I don't have a personal preference and I don't know if either platform added new features, so maybe other members of the forum will be able to help with this  

Hope this helps, Some_randomer. Best of luck with your research!