Interrater Agreement Statistics for Nominal Data

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Interrater Agreement Statistics for Nominal Data


Please forgive this rather long post. As some of you know, I've been interested in interrater agreement statistics for nominal data for some time, principally because of the amount of work I've done which required multiple raters performing coding. When anyone is interested in using one of the statistics, there is often a lot of frustration involved in finding information about the statistic and/or a computer-based solution. So several years ago I embarked on a journey which I thought would be helpful. I'm including a link to my Google Drive (while I start development of a website). This should hopefully bring you to a zip file from which you can download six files. The 'Narrative' file explains approximately 25 statistics for interrater agreement of nominal data. These occur in four sections, "Raw Agreement" (including some interesting recent work by Alexander Von Eye at Michigan State University), Statistics Based on Uniform Distribution of Categories (Bennett's S is the standard bearer here, although Cooil and Rust's PRL is interesting), Statistics Based on a Single Distribution of Categories Reflecting Observed Data (including Fleiss' kappa, Krippendorff's alpha, and Gwet's AC1), and finally Statistics Based on Distributions of Categories Reflecting Individual Raters (Cohen, Light, Hubert/Conger). A Reference Section is included so the user can go farther is desired. The second file provides directions for the use of syntax for each. There are four files of syntax, each labeled with the type of statistics involved. Both the general directions for syntax and each separate syntax contain fairly detailed information about setting up the data and completing the macros necessary to run everything, all of which are based in matrix language. I've included a way to get in touch with me should you have questions or encounter any difficulties. Enjoy!


===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD