Diary Day 90: Pace Trial 5. Using Likert Scales Postively Skew Research Results. Anyone Got Copies Of PACE Case Report Forms?

(Screen capture from Measurement In Health Behaviour: Methods For Research and Evaluation by C K Dilorio) 

I’m keeping today’s post short as the last 2 have been long and at times technical and my brain hurts, I didn’t rest enough yesterday and felt awful because of it. I don’t know about you!!

Yesterday I re-read the research protocol for the PACE trial. 

I re-read it because I wondered what tools the researchers used to measure the outcomes for the participants in the PACE trial. In the Abstract, they call them the Case Report Forms.

What I mean is, what did the form / online form / piece of paper look like – the ones that were handed to the participants that asked them to say how they were before, during and after treatment.

I was interested because the type of form you hand out to research participants can make a difference to how a participant responds to it.

I’m not being clear about this …I’ll try again.

Here’s a Likert scale – you’ve seen these before, right..

Likert Scale

Well, researchers have noted that there are a lot of problems using Likert scales in research.

download

Mainly because people tend to answer towards the positive end of the research scale, rather than the middle or negative end.

Humans are attracted by positive statements in these scales. It’s a human thing about our cognitive biases – how we think.

If the forms aren’t carefully written, it produces a bias that gives the potential for a positive skew which is reflected in the research results, as this diagram shows: 

Skewness

There are ways around designing a positive skew into your research if you are using a Likert Scale type response

What’s a positive skew? 

It produces unnaturally positive results. The statistics will be wrong and so the results will be wrong too.

You don’t want to build this into your research design because it essentially unbalances and invalidates the research results.

What can you do about it? 

As you aren’t allowed to stand over participants in research studies and tell them what to do when they are filling out forms, you have to go back to the drawing board and make sure your research response form is right in the first place.

Thankfully in using Likert scales, there’s a lot of good work that has already been done by research statisticians, to make the right changes in response form design.

Pilot study 

Good research practice also dictates that researchers test out the response forms. They given out with a sample of people and tested out before they use them in a research study – this is called a Pilot Study.

The Pilot Study is designed to eradicate or indicate where tweaks need to be made before you go ahead into the research study proper.

Question 1 : Was a Likert scale used as the response form or forms in the PACE trial?

Question 2: Was a pilot study carried out using the PACE trial response forms?

Question 3: Were there any issues with the response forms that the authors acknowledged could have skewed the participant response? 

Question 4: What changes were made as a result? 

Question 5: Does anyone have access to the response forms used with participants in the PACE trial and could circulate a copy?

PACE trial measurement scales

(extract of details of patient self-report scales utilised in the PACE trial)

I will join the dots with this another day, but now you’ve seen this post, I’m sure you are smart enough to see the implications….:-)

© Lindy 2015

Advertisements

One thought on “Diary Day 90: Pace Trial 5. Using Likert Scales Postively Skew Research Results. Anyone Got Copies Of PACE Case Report Forms?

  1. How odd. I always thought – and I am not relying on any professional qualifications of relevance here – that ‘human nature’ would tend to see people opt for the middle option on a 1-5 scale.
    Of course if you have swallowed CBT then you might be inclined to overscore positively I suppose 😉

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s