# Question Regarding Analysis

8 messages
Open this post in threaded view
|

## Question Regarding Analysis

 I have been asked to analyze data from a large (n=402) study of an intervention for adolescents. One of the measures, that is quite highly used, is the CAFAS, which assesses functioning in children and youth, age 6 to 18. There are eight items, scored from 0 to 30 in increments of 10 (i.e., 0, 10, 20, 30). The eight items produce a total score ranging from 0 to 240. A review of the literature does not reveal really good reliability, either alpha, or ICC (the measure is clinician-completed). Item-total correlations range from .20 to .57. The distribution of scores on all items is skewed, either positively or negatively. I'm proposing to analyze each of the scales as if they were ordinal and the total score as interval, using a repeated measures approach since there are multiple measures per youth. I neither designed the study nor chose the measures. I've simply been commissioned to analyze the data. Any thoughts on treating the items as ordinal? I know it's conservative, but I have difficulty with a four-point discontinuous scale as really interval in nature. Brian ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Open this post in threaded view
|

## Re: Question Regarding Analysis

 Google shows me that this is a scale with a long history of use, sampling in various populations; and that the 4-point scoring stands for Severe/ Moderate/ Mild/ Little or no impairment. On reliability: It is the wrong idea, to use "internal consistency" as /the/ criterion for reliability for a composite that covers a variety of areas. You want rater-rater comparisons for assessing consistency, and some external comparisons for validity. On "equal intervals": If I wanted evidence of the non-interval nature of the scaling, I would look at the cross-time tabulations, even better than the cross-rater tabulations.  What I would look for is the frequency of changes between categories. If, for instance, no one "ever" moves from 0 to 10 (compared to other changes), then that suggests that the interval is largest between 0 and 10. But is that perhaps a function of the high-risk population you are sampling? Do you have a subjective feeling that the distances are unequal? It is very, very common for 4-point scales to be analyzed by ANOVA as if they equal-interval to a suitable degree - For a much-used scale, apparently others have been satisfied. The question is not "Are these unequal intervals" but, rather, "Are these intervals improved enough by some transformation to justify the complication of computation, and the confusion in presenting results?" On transformation to rank: The simple rule of thumb is that, if the means do not provide a good comparison between groups, then you probably should transform.  The usual "non-parametric" test gives an analysis, essentially by ANOVA, of the rank-transformed scores.  So, for a few skewed variables, compute the rank-transformed scores. (If 400 scores are /equally/ divided into four groups, your result effectively matches the "intervals" you started out with, since the 100 ties in each group give you average ranks of  50.5, 150.5, 250.5, and 350.5 -- Transforming "10" to 50.5, etc., gives you the same ANOVA F, since you have simply performed the same linear transformation on each of the scores.) Does this version of "interval", when applied to the skewed variables (where it makes a difference) make more sense? In my experience, I usually haven't preferred the simple, rank scoring. Advanced scale development goes a step further, and uses a logistic transformation on the average-rank.  This DOES give a spacing for which there is some theoretical justification, and better "normality".  I won't say any more about that, except that I want a "norming" sample if I'm going to set norms for that version of scaling -- and , personally, I never did play with scale development that intensively. -- Rich Ulrich From: SPSSX(r) Discussion <[hidden email]> on behalf of Dates, Brian <[hidden email]> Sent: Tuesday, May 7, 2019 10:33 AM To: [hidden email] Subject: Question Regarding Analysis   I have been asked to analyze data from a large (n=402) study of an intervention for adolescents. One of the measures, that is quite highly used, is the CAFAS, which assesses functioning in children and youth, age 6 to 18. There are eight items, scored from 0 to 30 in increments of 10 (i.e., 0, 10, 20, 30). The eight items produce a total score ranging from 0 to 240. A review of the literature does not reveal really good reliability, either alpha, or ICC (the measure is clinician-completed). Item-total correlations range from .20 to .57. The distribution of scores on all items is skewed, either positively or negatively. I'm proposing to analyze each of the scales as if they were ordinal and the total score as interval, using a repeated measures approach since there are multiple measures per youth. I neither designed the study nor chose the measures. I've simply been commissioned to analyze the data. Any thoughts on treating the items as ordinal? I know it's conservative, but I have difficulty with a four-point discontinuous scale as really interval in nature. Brian ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Open this post in threaded view
|

## Re: Question Regarding Analysis

Open this post in threaded view
|

## Re: Question Regarding Analysis

Open this post in threaded view
|

## Re: Question Regarding Analysis

Open this post in threaded view
|