Likert Scales Ordinal or Interval?

Black Swan

It makes a substantial difference in terms of selecting the appropriate statistical formulas to analyze data if such data is treated at the ordinal or interval levels, the first allowing for nonparametric analysis, and the second parametric.

The below scale item appears to be at the ordinal level, in that we cannot assume with absolute certainty that there are equal intervals between answers.

It's the duty of doctors to keep patients alive as long as possible:
[ ] Strongly disagree
[ ] Disagree
[ ] Somewhat disagree
[ ] Somewhat agree
[ ] Agree
[ ] Strongly agree

But if we assign numbers on a scale with distances between answers that appear to have equal intervals (e.g., 1, 2, 3, 4, 5, 6), many researchers will treat such a scale as if it were at the interval level, using parametric statistical formulas and assumptions to analyze the data (e.g., t-test, ANOVA, Pearson r, Regression). I have seen such treatments published in scholarly peer-reviewed journals in psychology, education, medicine, etc., wondering if such treatments were a violation of statistical assumptions.

Your thoughts?

Reference:
Jamieson, S. (2004). Likert scales: how to (ab)use them. MEDICAL EDUCATION 38: 1212–1218.

Ostrom

Black Swan wrote: »

It makes a substantial difference in terms of selecting the appropriate statistical formulas to analyze data if such data is treated at the ordinal or interval levels, the first allowing for nonparametric analysis, and the second parametric.

The below scale item appears to be at the ordinal level, in that we cannot assume with absolute certainty that there are equal intervals between answers.

It's the duty of doctors to keep patients alive as long as possible:
[ ] Strongly disagree
[ ] Disagree
[ ] Somewhat disagree
[ ] Somewhat agree
[ ] Agree
[ ] Strongly agree

But if we assign numbers on a scale with distances between answers that appear to have equal intervals (e.g., 1, 2, 3, 4, 5, 6), many researchers will treat such a scale as if it were at the interval level, using parametric statistical formulas and assumptions to analyze the data (e.g., t-test, ANOVA, Pearson r, Regression). I have seen such treatments published in scholarly peer-reviewed journals in psychology, education, medicine, etc., wondering if such treatments were a violation of statistical assumptions.

Your thoughts?

Reference:
Jamieson, S. (2004). Likert scales: how to (ab)use them. MEDICAL EDUCATION 38: 1212–1218.

It's so ubiquitous now, it passes unquestioned. Constrained 'interval' variables are by definition problematic (i.e. those with upper limits such as percentages or rating scales), but we happily throw them into models of all sorts, diagnose the model and proceed regardless.

My own preference is to collapse them into smaller categories and run a probit or multinomial model if you really have to model a single likert item. There are other ways around the problem of limited variance in the outcome - in psychology you often see composite scales built from averaging multiple likert style questions which give a bit more latitude on the outcome, or else factor analyses/PCA where the individual is assigned a loading value.

I'm a bit more hesitant about these - what you are actually modelling becomes a little less intuitive. Ultimately, I think these approaches are fine - the real issue is less the specifics of meeting assumptions (I'm inclined to think the above are more often than not violated), but about full disclose of the modelling process. Dropping variables, making selections, and imposing dodgy weightings are an intrinsic part of generating 'publishable' research output, and the recent Reinhart and Rogoff controversy has really brought this home url]http://nymag.com/daily/intelligencer/2013/04/grad-student-who-shook-global-austerity-movement.html[/url.

deanmartin

Technically you are correct Black swan. A likert-scale is an ordinal ranking. The problem is that we treat interval as such, we assume equal distance between scores, when distances may actually vary.

So, technically yes, this is incorrect. Does this produce a conservative or a liberal bias when testing effects? I would maintain it creates a conservative bias, since an ordinal is a less powerful discriminator than an interval or ratio scale, and thus suppresses variance.

I also take an empirical lens with respect to such issues. Unless the use of scale type actually changes results, then does it really matter?

Black Swan

Sincere thanks for your well informed replies efla and deanmartin. They were highly valued.

Although I am aware that "It's so ubiquitous now, it passes unquestioned," especially in certain disciplines and their peer-reviewed journals, if ordinal level data was to be treated as interval for analysis purposes, then I believe that a very short note should be placed in the methodology or limitations or conclusions of the research report or published article that states these treatments.

Given that we cannot prove anything through the use of the scientific method, only suggest, I believe that such cautions have value, especially for practical decision-makers who may be from outside those disciplines that consider such treatments as ubiquitous. Further, several research grant sources that derive their monies from public taxes have strongly endorsed that we report our results to a larger audience outside of those familiar with these statistical issues.