Qualtrics Survey Platform

Anakena · 27-06-2018 7:50pm #1

Hi all,

Just opening up a thread to talk about Qualtrics for anyone interested in using a survey platform. I have personally used it for my thesis research and found it to be very resourceful in the types of features available, ranging from survey creation through to distribution and data analysis.

I was able to set up my account for free as I am registered through my university but I know that a free trial account is also otherwise available.

If anyone has any questions about the user experience on Qualtrics, post them here, and if any other members have any advice please share!

Cheers!

Black Swan · 28-06-2018 11:40pm

diana_micu wrote: »

If anyone has any questions about the user experience on Qualtrics, post them here, and if any other members have any advice please share!

I have never used Qualtrics for online survey construction and administration. Just wondering if you piloted your survey questionnaire before conducting your main survey? Were there any bugs to work out, and if so, what were they?

Did you test to see if your Qualtrics survey ran seamlessly on different operating systems and browsers (e.g. OS-X or MS 7-10 or Linux; or browsers IE, Firefox, Safari, Chrome, Opera)? I know of a doctorate student doing his pilot and ran into problems using Qualtrics. He sent his survey from an Apple laptop using a FireFox browser and when sent it to a pilot subject using an HP desktop using Internet Explorer his email provider automatically sent his Qualtrix survey into the Spam folder.

Anakena · 30-06-2018 2:26am

Black Swan wrote: »

I have never used Qualtrics for online survey construction and administration. Just wondering if you piloted your survey questionnaire before conducting your main survey? Were there any bugs to work out, and if so, what were they?

Did you test to see if your Qualtrics survey ran seamlessly on different operating systems and browsers (e.g. OS-X or MS 7-10 or Linux; or browsers IE, Firefox, Safari, Chrome, Opera)?

Yes, I pilot tested my survey with a few friends before publishing the final version. Feedback was positive, nobody encountered any difficulties in taking the survey and I know for a fact that different platforms were used i.e. mobile (ios and android) and laptop (chrome, opera, safari). I also didn't encounter any bugs with the survey distribution - I used an anonymous link and QR code and both worked fine.

As for distributing the survey through email, I did not have any problems there either. I sent the link to my supervisor through Gmail on the Opera browser and the email didn't go to spam for her, though I don't know whether that's because there was already a conversation history between our email addresses.

Did he send the email directly from Qualtrics' Distribution section? As perhaps there might be some kind of trigger there that might classify such an email as spam, otherwise I would think that sending the email from his personal account would end up in the regular Inbox folder regardless of what browser was used. If the email was sent directly from Qualtrics' website, it would send from an automated "noreply" type email address, so that could have been caught as spam. Perhaps Qualtrics could provide a better explanation for this though.

Black Swan · 01-07-2018 12:34am

diana_micu wrote: »

As for distributing the survey through email, I did not have any problems there either. I sent the link to my supervisor through Gmail on the Opera browser and the email didn't go to spam for her, though I don't know whether that's because there was already a conversation history between our email addresses.

If the email was sent directly from Qualtrics' website, it would send from an automated "noreply" type email address, so that could have been caught as spam.

The spam problem had been corrected by embedding the Qualtrics link in a Microsoft Word document.

But then new problems emerged. There is now a MS security warning popping up when you click on the Qualtrics link, which could affect the survey return rate.

diana_micu wrote: »

Perhaps Qualtrics could provide a better explanation for this though.

There is a Qualtrics Community forum for such discussions, and I have asked him to consult with them.

Black Swan · 01-07-2018 1:21am

What statistical software programme are you using to analyze your survey data? SPSS? Does the data collected in Qualtrics download easily into SPSS (or a different statistical programme)? Do you just check a box in Qualtrics, or what?

Anakena · 03-07-2018 12:34am

Black Swan wrote: »

What statistical software programme are you using to analyze your survey data? SPSS? Does the data collected in Qualtrics download easily into SPSS (or a different statistical programme)? Do you just check a box in Qualtrics, or what?

I've downloaded the data both to SPSS and Excel (CSV format) to compare; both ways were quick and easy. There is an export option in the Data & Analysis section so you just click that and choose your format, it was pretty straightforward. Also, a difference I noticed between the two versions is that SPSS presents some answers numerically (i.e. 1-7 for a 7-point Likert scale) whereas Excel presents them through words (strongly disagree - strongly agree). However, SPSS provides data interpretation in a separate tab for this.

Black Swan · 03-07-2018 2:27pm

diana_micu wrote: »

I've downloaded the data both to SPSS and Excel (CSV format) to compare; both ways were quick and easy. There is an export option in the Data & Analysis section so you just click that and choose your format, it was pretty straightforward. Also, a difference I noticed between the two versions is that SPSS presents some answers numerically (i.e. 1-7 for a 7-point Likert scale) whereas Excel presents them through words (strongly disagree - strongly agree). However, SPSS provides data interpretation in a separate tab for this.

Grand information. I will pass it on to my student later today.

Black Swan · 07-07-2018 4:43pm

If you, or someone else reading this post, has used both Qualtrics and a different online survey platform (e.g., SurveyMonkey, etc.), how does Qualtrics compare? User-friendliness? Ease of survey construction? Transmitting online survey to different operating systems and different browsers? Ease of online survey administration? Collecting and downloading data for statistical analysis?

Anakena · 07-07-2018 11:07pm

Black Swan wrote: »

If you, or someone else reading this post, has used both Qualtrics and a different online survey platform (e.g., SurveyMonkey, etc.), how does Qualtrics compare? User-friendliness? Ease of survey construction? Transmitting online survey to different operating systems and different browsers? Ease of online survey administration? Collecting and downloading data for statistical analysis?

I'm afraid I haven't, sorry!

Black Swan · 09-07-2018 4:54am

Has anyone else reading this thread used both Qualtrics and SurveyMonkey, or a different online survey programme?

Black Swan · 12-07-2018 11:46pm

Just had another question from a student who plans to use Qualtrics for their online survey. How do you ensure that the same subject does not repeat taking the same survey more than once? Precisely, is there a box [x] you check in Qualtrics that only allows one submission? If so, where is it found in Qualtrics?

Anakena · 12-07-2018 11:59pm

Black Swan wrote: »

Just had another question from a student who plans to use Qualtrics for their online survey. How do you ensure that the same subject does not repeat taking the same survey more than once? Precisely, is their a box [x] you check in Qualtrics that only allows one submission? If so, where is it found in Qualtrics?

The only answer to this that I can think of is if the survey is distributed via a 'personal link'. In this method you input the intended participant's details so that only they can access the link and take the survey. However, if you use any other method, i.e. anonymous link or QR code, I'm not aware of a feature that would ensure the same person doesn't take the survey twice. I do know that you are able to check the IP address of the respondent but I don't know how much help that is to you.

Maybe they can get a better answer from Qualtrics though.

Black Swan · 13-07-2018 12:04am

Thanks diana for your quick and detailed reply.

Black Swan · 14-07-2018 2:17pm

diana_micu wrote: »

Maybe they can get a better answer from Qualtrics though.

I am referring them to Qualtrics Community to discuss this problem further, and hopefully they will address it accordingly.

Black Swan · 14-07-2018 7:17pm

Black Swan wrote: »

I am referring them to Qualtrics Community to discuss this problem further, and hopefully they will address it accordingly.

Preventing respondents from taking your survey more than once:

Open Survey Options in Survey tab
Select the Prevent Ballot Box Stuffing checkbox
Save

Anakena · 15-07-2018 12:06pm

Black Swan wrote: »

Preventing respondents from taking your survey more than once:
Open Survey Options in Survey tab

Select the Prevent Ballot Box Stuffing checkbox

Save

Great, thanks for sharing!

Some_randomer · 06-08-2018 9:46pm

Hi OP how well would Qualtrics be suited for a survey where we require respondents to select one option (i.e. their favourite) out of around 50?

I posted a question here where we're looking for guidance, would love some feedback if you have any thoughts on it:

https://www.boards.ie/b/thread/2057899027

Anakena · 06-08-2018 10:40pm

Some_randomer wrote: »

Hi OP how well would Qualtrics be suited for a survey where we require respondents to select one option (i.e. their favourite) out of around 50?

I posted a question here where we're looking for guidance, would love some feedback if you have any thoughts on it:

https://www.boards.ie/b/thread/2057899027

Hi! Just read your original question there. Qualtrics would work just fine for such a question. In fact, you have a bit of flexibility with how you can present the options/names i.e. vertically, horizontally, single choice, multiple choice etc. Qualtrics lets you add as many options as you want (over 100 from what I've tested).

In response to the 2nd bullet point in your original post, I would recommend that you don't use a scale question (strongly like -> strongly dislike) as I think people would lose interest after going through a few names. Maybe the best thing to do would be to set the question up as a multiple choice question, and get respondents to indicate their top 3 choices.

With regards to point 3, I personally wouldn't give more attention to the first or last name in the list. I would just choose what I liked best, but haven't studied the psychology behind that so I wouldn't know if the position of the name influences choice (I think it more so influences recall).

Also, I'm not sure if you're aware but Qualtrics offers a free trial version of the platform so you could test it out.

Hope that helps and if you have any more questions, feel free to ask!

Some_randomer · 07-08-2018 1:49pm

Many thanks will check out Qualtrics and play around with it.

Black Swan · 14-08-2018 1:38am

When surveying a population with different first languages, one Qualtrics user told me that you could give a subject their choice of answering your questionnaire in either English or Spanish. Just wondering what other languages are already built into the Qualtrics platform? Anyone know?

Anakena · 15-08-2018 12:26pm

Black Swan wrote: »

When surveying a population with different first languages, one Qualtrics user told me that you could give a subject their choice of answering your questionnaire in either English or Spanish. Just wondering what other languages are already built into the Qualtrics platform? Anyone know?

You are able to select any language as the language that the survey is written in. Or if not, there is an option to translate the survey to any language (out of roughly 75) and I think you can add as many languages as you want the survey to be translated in.

There is also a choice to "create other language" in the translating section.

Black Swan · 16-08-2018 4:44pm

Useful Anakena. A related question. Can you run a survey with a choice at its beginning where the participant may [x] check either English or Arabic and that version pops up for them to answer?

Secondly, with 2 different language choices for each participant in the survey, is there a checkbox for the Qualtrics researcher to combine both language versions into one for SPSS analysis?

Anakena · 20-08-2018 3:42pm

Black Swan wrote: »

Useful Anakena. A related question. Can you run a survey with a choice at its beginning where the participant may [x] check either English or Arabic and that version pops up for them to answer?

Secondly, with 2 different language choices for each participant in the survey, is there a checkbox for the Qualtrics researcher to combine both language versions into one for SPSS analysis?

Sorry, I don't know about this. I didn't see these kind of options when playing around with the survey but I'm sure someone at Qualtrics could give you an answer for this!

Black Swan · 02-09-2018 1:20pm

Black Swan wrote: »

A related question. Can you run a survey with a choice at its beginning where the participant may [x] check either English or Arabic and that version pops up for them to answer?

Secondly, with 2 different language choices for each participant in the survey, is there a checkbox for the Qualtrics researcher to combine both language versions into one for SPSS analysis?

Anakena wrote: »

Sorry, I don't know about this. I didn't see these kind of options when playing around with the survey but I'm sure someone at Qualtrics could give you an answer for this!

Found our answer to these 2 questions from Qualtrics:

With our Translate Survey feature, you can add multiple languages to a single survey. Each respondent can see the survey in a language they are comfortable with, and because there is just one survey containing all translations, all results will come back into the same dataset.

There is a large number of languages too in the Qualtrics survey platform. Scroll down to Available Language Codes.

Black Swan · 22-09-2018 5:09pm

The grad students working as RAs at our university have been helpful with questions regarding Qualtrics survey platform use, but their knowledge does not evidence great depth. Alternatively, the techs at Qualtrics have been very receptive to recent questions by one of my grad students, and because our university has a Qualtrics contract, did not charge him anything for help. Something to check out if you are using Qualtrics at your university.

You can also Google for Qualtrics vids on the web, and they are mostly free.

Qualtrics Survey Platform

Comments