Shapiro-Wilks Statistic

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Shapiro-Wilks Statistic

Baker, Harley
Colleagues,

I am trying to get the Shapiro-Wilks Statistics and having a difficult time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina. According to the manual, the following syntax (generated by point-and-click methods) should work. It produces the K-S test, but not W-S test:

EXAMINE VARIABLES=F1
  /PLOT BOXPLOT HISTOGRAM NPPLOT
  /COMPARE GROUPS
  /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
  /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
  /STATISTICS DESCRIPTIVES EXTREME
  /CINTERVAL 95
  /MISSING LISTWISE
  /NOTOTAL.

What am I doing/not doing such that the Shapiro-Wilks test statistic is not produced? What do I do to fix it?

Thanks!

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Bruce Weaver
Administrator
Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests.  

Why do you want to test for normality?



Baker, Harley wrote

> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but not
> W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic is
> not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands
> One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Anthony Babinec
Do not know if this is the case in Harley's situation, but there
are some requirements in order to produce the statistic. From syntax
guide(v27):

NPPLOT. Normal and detrended Q-Q plots with the Tests of Normality table
presenting Shapiro-Wilk's
statistic and a Kolmogorov-Smirnov statistic with a Lilliefors significance
level for testing normality. If noninteger
weights are specified, the Shapiro-Wilk's statistic is calculated when the
weighted sample size lies
between 3 and 50. For no weights or integer weights, the statistic is
calculated when the weighted
sample size lies between 3 and 5,000.

Tony Babinec

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce
Weaver
Sent: Wednesday, September 30, 2020 12:24 PM
To: [hidden email]
Subject: Re: Shapiro-Wilks Statistic

Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests.  

Why do you want to test for normality?



Baker, Harley wrote

> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but
> not W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic
> is not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

3J LEMA
Any tips on when to use the Shapiro-Wilk statistic over the Kolmogorov-Smirnov Statistic in testing normality?

Thank you.

On Thu, 1 Oct 2020 at 01:38, Anthony Babinec <[hidden email]> wrote:
Do not know if this is the case in Harley's situation, but there
are some requirements in order to produce the statistic. From syntax
guide(v27):

NPPLOT. Normal and detrended Q-Q plots with the Tests of Normality table
presenting Shapiro-Wilk's
statistic and a Kolmogorov-Smirnov statistic with a Lilliefors significance
level for testing normality. If noninteger
weights are specified, the Shapiro-Wilk's statistic is calculated when the
weighted sample size lies
between 3 and 50. For no weights or integer weights, the statistic is
calculated when the weighted
sample size lies between 3 and 5,000.

Tony Babinec

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce
Weaver
Sent: Wednesday, September 30, 2020 12:24 PM
To: [hidden email]
Subject: Re: Shapiro-Wilks Statistic

Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests. 

Why do you want to test for normality?                         



Baker, Harley wrote
> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but
> not W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic
> is not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Jon Peck
You might consider as well the Anderson-Darling test.  If you use the simulation procedure on your active file,  you can choose the relevant variables and let the dialog fit the selected variables and choose the best one from a large number of possibilities,

On Wed, Sep 30, 2020 at 7:02 PM 3J LEMA <[hidden email]> wrote:
Any tips on when to use the Shapiro-Wilk statistic over the Kolmogorov-Smirnov Statistic in testing normality?

Thank you.

On Thu, 1 Oct 2020 at 01:38, Anthony Babinec <[hidden email]> wrote:
Do not know if this is the case in Harley's situation, but there
are some requirements in order to produce the statistic. From syntax
guide(v27):

NPPLOT. Normal and detrended Q-Q plots with the Tests of Normality table
presenting Shapiro-Wilk's
statistic and a Kolmogorov-Smirnov statistic with a Lilliefors significance
level for testing normality. If noninteger
weights are specified, the Shapiro-Wilk's statistic is calculated when the
weighted sample size lies
between 3 and 50. For no weights or integer weights, the statistic is
calculated when the weighted
sample size lies between 3 and 5,000.

Tony Babinec

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce
Weaver
Sent: Wednesday, September 30, 2020 12:24 PM
To: [hidden email]
Subject: Re: Shapiro-Wilks Statistic

Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests. 

Why do you want to test for normality?                         



Baker, Harley wrote
> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but
> not W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic
> is not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

3J LEMA
Is the Anderson-Darling test available in SPSS? I cannot see  Anderson-Darling test in the point click.

On Thu, 1 Oct 2020 at 09:58, Jon Peck <[hidden email]> wrote:
You might consider as well the Anderson-Darling test.  If you use the simulation procedure on your active file,  you can choose the relevant variables and let the dialog fit the selected variables and choose the best one from a large number of possibilities,

On Wed, Sep 30, 2020 at 7:02 PM 3J LEMA <[hidden email]> wrote:
Any tips on when to use the Shapiro-Wilk statistic over the Kolmogorov-Smirnov Statistic in testing normality?

Thank you.

On Thu, 1 Oct 2020 at 01:38, Anthony Babinec <[hidden email]> wrote:
Do not know if this is the case in Harley's situation, but there
are some requirements in order to produce the statistic. From syntax
guide(v27):

NPPLOT. Normal and detrended Q-Q plots with the Tests of Normality table
presenting Shapiro-Wilk's
statistic and a Kolmogorov-Smirnov statistic with a Lilliefors significance
level for testing normality. If noninteger
weights are specified, the Shapiro-Wilk's statistic is calculated when the
weighted sample size lies
between 3 and 50. For no weights or integer weights, the statistic is
calculated when the weighted
sample size lies between 3 and 5,000.

Tony Babinec

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce
Weaver
Sent: Wednesday, September 30, 2020 12:24 PM
To: [hidden email]
Subject: Re: Shapiro-Wilks Statistic

Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests. 

Why do you want to test for normality?                         



Baker, Harley wrote
> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but
> not W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic
> is not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Jon Peck
That’s why I said to use the SIMULATION procedure.  When you get to the simulation tab, click Fit All and after that click on Fit Details.  You will see all the fits tried with their parameter estimates and fit test.

If you are still stuck, I’ll send you more details tomorrow

On Wed, Sep 30, 2020 at 8:59 PM 3J LEMA <[hidden email]> wrote:
Is the Anderson-Darling test available in SPSS? I cannot see  Anderson-Darling test in the point click.

On Thu, 1 Oct 2020 at 09:58, Jon Peck <[hidden email]> wrote:
You might consider as well the Anderson-Darling test.  If you use the simulation procedure on your active file,  you can choose the relevant variables and let the dialog fit the selected variables and choose the best one from a large number of possibilities,

On Wed, Sep 30, 2020 at 7:02 PM 3J LEMA <[hidden email]> wrote:
Any tips on when to use the Shapiro-Wilk statistic over the Kolmogorov-Smirnov Statistic in testing normality?

Thank you.

On Thu, 1 Oct 2020 at 01:38, Anthony Babinec <[hidden email]> wrote:
Do not know if this is the case in Harley's situation, but there
are some requirements in order to produce the statistic. From syntax
guide(v27):

NPPLOT. Normal and detrended Q-Q plots with the Tests of Normality table
presenting Shapiro-Wilk's
statistic and a Kolmogorov-Smirnov statistic with a Lilliefors significance
level for testing normality. If noninteger
weights are specified, the Shapiro-Wilk's statistic is calculated when the
weighted sample size lies
between 3 and 50. For no weights or integer weights, the statistic is
calculated when the weighted
sample size lies between 3 and 5,000.

Tony Babinec

-----Original Message-----
From: SPSSX(r) Discussion <[hidden email]> On Behalf Of Bruce
Weaver
Sent: Wednesday, September 30, 2020 12:24 PM
To: [hidden email]
Subject: Re: Shapiro-Wilks Statistic

Hello Harley.  I copied your syntax, changed F1 to the name of a variable in
the data file I had open, and ran it.  The Tests of Normality table it
generated showed both K-S and S-W tests. 

Why do you want to test for normality?                         



Baker, Harley wrote
> Colleagues,
>
> I am trying to get the Shapiro-Wilks Statistics and having a difficult
> time in SPSS v27 (Mac; release 27.0.0.0; 64-bit) running on Catalina.
> According to the manual, the following syntax (generated by
> point-and-click methods) should work. It produces the K-S test, but
> not W-S test:
>
> EXAMINE VARIABLES=F1
>   /PLOT BOXPLOT HISTOGRAM NPPLOT
>   /COMPARE GROUPS
>   /MESTIMATORS HUBER(1.339) ANDREW(1.34) HAMPEL(1.7,3.4,8.5) TUKEY(4.685)
>   /PERCENTILES(5,10,25,50,75,90,95) HAVERAGE
>   /STATISTICS DESCRIPTIVES EXTREME
>   /CINTERVAL 95
>   /MISSING LISTWISE
>   /NOTOTAL.
>
> What am I doing/not doing such that the Shapiro-Wilks test statistic
> is not produced? What do I do to fix it?
>
> Thanks!
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Bruce Weaver
Administrator
In reply to this post by 3J LEMA
I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)  



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Baker, Harley
Hi Bruce,

An editorial reviewer is requiring me to do so. Otherwise, I would not. The data are ordinal in nature, so the issue of normality doesn't even apply to my situation. Regardless, I am complying . . . 

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 5:26 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=Ix%2BBFHa0yn6LxO34w0WvuZtCNdoa5yokfx6gEGNiXWQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=LQAFSVr%2FSx2BNd18u642DIIbTYXsq4PG2plFjzpkWKg%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Bruce Weaver
Administrator
Ah, I see!  In that situation, I would first try explaining to the reviewer
why the test of normality serves no useful purpose, and hope that the editor
was strong enough to support me.  And if the only way to get the manuscript
accepted was to report the test of normality, I would insist on including a
footnote explaining why it is not a useful test!  

Good luck!  ;-)  



Baker, Harley wrote

> Hi Bruce,
>
> An editorial reviewer is requiring me to do so. Otherwise, I would not.
> The data are ordinal in nature, so the issue of normality doesn't even
> apply to my situation. Regardless, I am complying . . .
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands
> One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Baker, Harley
Thanks, Bruce

Alas, I have done that. The editor agrees it serves no useful purpose. (In fact, I argued including the S-W is harmful because it helps perpetuate the practice of assessing normality in situations where it is meaningless.) This is one of the few remaining sticking points before full acceptance. I have negotiated it so that I do not have to put the test results in the ms, but do need to put them before the reviewer. As an editorial reviewer myself, I have no problem with that, though it is annoying!

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 8:00 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


Ah, I see!  In that situation, I would first try explaining to the reviewer
why the test of normality serves no useful purpose, and hope that the editor
was strong enough to support me.  And if the only way to get the manuscript
accepted was to report the test of normality, I would insist on including a
footnote explaining why it is not a useful test!

Good luck!  ;-)



Baker, Harley wrote
> Hi Bruce,
>
> An editorial reviewer is requiring me to do so. Otherwise, I would not.
> The data are ordinal in nature, so the issue of normality doesn't even
> apply to my situation. Regardless, I am complying . . .
>
> Harley
>
> Dr. Harley Baker
> Professor Emeritus of Psychology
> California State University Channel Islands
> One University Drive
> Camarillo, CA 93012





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C8bb68ad6939a46df445408d8661ad679%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C0%7C637371612745010465&amp;sdata=3F6IsJ70Gz5HyFqZVdx7Y82jcjploEMEqhBNEpUpn2A%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C8bb68ad6939a46df445408d8661ad679%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C0%7C637371612745010465&amp;sdata=RVjk0QrnnBEJOc6YJ4Qt0hUccrmirHuYUqjWHjNtxDQ%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Rich Ulrich
In reply to this post by Baker, Harley
I'm a little confused.  You say,
"The data are ordinal in nature, so the issue of normality doesn't even
apply to my situation."

"... the issue doesn't even apply ... "
That implies to me that you are doing analyses after rank-transform
("nonparametric") without considering ordinary ANOVA -- and the
reviewer asks you to justify abandoning the raw scores.

If I were reviewer, I might ask the same.  I hope I would put the
request more clearly than what you seem to have received.

Bruce writes, unclear about the situation -
If it is to justify use of a parametric test, my advice would be, "Don't
bother!"  Like me, Bruce is biased against non-parametric testing for
decently-behaved ordinal data.
--
Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 10:38 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
Hi Bruce,

An editorial reviewer is requiring me to do so. Otherwise, I would not. The data are ordinal in nature, so the issue of normality doesn't even apply to my situation. Regardless, I am complying . . . 

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 5:26 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=Ix%2BBFHa0yn6LxO34w0WvuZtCNdoa5yokfx6gEGNiXWQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=LQAFSVr%2FSx2BNd18u642DIIbTYXsq4PG2plFjzpkWKg%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Baker, Harley
I have a number of religious identity labels that participants rate in terms of how well they fit. There are five response categories from ‘not at all’ to ‘very well.’ As is the case in most research like this, the distribution of responses is basically U-shaped. (Basically, a label either fits or it does not, very few in-between responses.) Mardia’s statistic shows a huge violation of multivariate normality, as one would expect. As such, univariate normality is not an issue, which is what I should have said originally. Pearson correlations underestimate the degree of linear relationship in these situations, and we often turn to polychoric correlations instead. (I am loathe to use nonparametric correlations when perfectly good parametric methods exist. When categorical data are well-behaved, I also prefer to use parametric models. I agree with one of my mentors, Fred Lord: the data don’t know where they came from. So, if they act parametric, treat them that way. If not, don’t.) Polychoric correlations provide a more accurate estimate of the true degree of linear association under these conditions. The matrix was then submitted to an EFA to see what patterns of self-labeling emerge, if any.

Hope this clarifies the situation a bit better.

Harley


Sent from my iPad

On Oct 1, 2020, at 10:52 AM, Rich Ulrich <[hidden email]> wrote:


CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm

I'm a little confused.  You say,
"The data are ordinal in nature, so the issue of normality doesn't even
apply to my situation."

"... the issue doesn't even apply ... "
That implies to me that you are doing analyses after rank-transform
("nonparametric") without considering ordinary ANOVA -- and the
reviewer asks you to justify abandoning the raw scores.

If I were reviewer, I might ask the same.  I hope I would put the
request more clearly than what you seem to have received.

Bruce writes, unclear about the situation -
If it is to justify use of a parametric test, my advice would be, "Don't
bother!"  Like me, Bruce is biased against non-parametric testing for
decently-behaved ordinal data.
--
Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 10:38 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
Hi Bruce,

An editorial reviewer is requiring me to do so. Otherwise, I would not. The data are ordinal in nature, so the issue of normality doesn't even apply to my situation. Regardless, I am complying . . . 

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 5:26 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=Ix%2BBFHa0yn6LxO34w0WvuZtCNdoa5yokfx6gEGNiXWQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=LQAFSVr%2FSx2BNd18u642DIIbTYXsq4PG2plFjzpkWKg%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Art Kendall
You said you had ordinal data.  When I have reviewed manuscripts where other
reviewers have raised the issue of level of measurement this is what I have
suggested.
Try CATREG in SPSS.  CATREG does a test between models with different
assumptions about measurement level. See whether there is a meaningful
difference between MODEL fits with  ORDINAL specification and INTERVAL
specifications.  (You might also try a NOMINAL specification.)
If there is no meaningful difference in fits, use the interval level results
and add a footnote that CATREG showed no meaningful difference in fits when
ORDINAL was specified.



YMMV but I have tried this CATREG approach a dozen or so times and never
found a meaningful difference between the fits.

also see CATPCA in help



-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Baker, Harley
Thanks, Art

Polychoric correlations provide exactly what I need, as the EFA is the first step in a longer process (produce factor scores for use in the next phases of the analyses, etc.) The reviewer just wanted - I hope - some confirmation that the data were not suitable for Pearson r at the univariate level beyond the skew and kurtosis values that were all highly statistically signifiant with associated strong effect sizes. So the data don't fit Pearson assumptions likely due to the categorization imposed by the response format, but they nicely fit the assumptions/requirements for polychoric correlations to estimate the real correlations among them (e.g.Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, I., & Vila-Abad, E. (2010). Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Quality & Quantity: International Journal of Methodology, 44(1), 153–166.)

 
Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Art Kendall <[hidden email]>
Sent: Thursday, October 1, 2020 12:03 PM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


You said you had ordinal data.  When I have reviewed manuscripts where other
reviewers have raised the issue of level of measurement this is what I have
suggested.
Try CATREG in SPSS.  CATREG does a test between models with different
assumptions about measurement level. See whether there is a meaningful
difference between MODEL fits with  ORDINAL specification and INTERVAL
specifications.  (You might also try a NOMINAL specification.)
If there is no meaningful difference in fits, use the interval level results
and add a footnote that CATREG showed no meaningful difference in fits when
ORDINAL was specified.



YMMV but I have tried this CATREG approach a dozen or so times and never
found a meaningful difference between the fits.

also see CATPCA in help



-----
Art Kendall
Social Research Consultants
--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C02fe731981fb47641b4c08d8663cafd4%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C0%7C637371758116794955&amp;sdata=ceCGM61o5GOIZ6YrTlR2eQO65T2BwHIOxfq8aIVRLR4%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Bruce Weaver
Administrator
In reply to this post by Baker, Harley
Thanks for providing that detail, Harley.  For the benefit of readers who may
not be terribly familiar with polychoric correlation, here is the definition
from the relevant Wikipedia page:

"In statistics, polychoric correlation[1] is a technique for estimating the
correlation between two theorised normally distributed continuous latent
variables, from two observed ordinal variables."

And this paragraph from Rigdon & Ferguson (1991) goes even further in saying
that the underlying variables are assumed to have a bivariate normal
distribution:

"The polychoric correlation coefficient, introduced by Pearson, is an
alternative to the Pearson r specifically for situations in which the
variables of interest are continuous but the measurement instruments yield
data that may only be ordinal (Pearson and Pearson 1922; Ritchie-Scott
1918). Olsson (1979b) developed two maximum likelihood procedures for
estimating the polychoric, both based on the assumption that the unseen
underlying variables are continuous and have a bivariate normal
distribution."

Latent variables are a bit outside of my statistical wheelhouse, so maybe
this is a naive question, but I'll ask it anyway!  If the underlying
variables are (approximately) normal or bivariate normal, how can the
distributions of the observed ordinal variables by U-shaped?  I was about to
ask if there is any literature on how non-normality of the latent variables
affects things, but then I found an article by Jin & Yang-Wallentin (2017)
which tells me there has been some work on that question.  I don't have time
to read it right now, but a quick skim suggests that it does not discuss
U-shaped distributions.  

Bruce

PS- I am well aware of what George Box (1976) said about normality and
linearity in the real world, and understand that when we work with real
world data, approximate normality is the best we can hope for.  


References

Box, G. E. (1976). Science and statistics. Journal of the American
Statistical Association, 71(356), 791-799.
http://mkweb.bcgsc.ca/pointsofsignificance/img/Boxonmaths.pdf

Jin, S., & Yang-Wallentin, F. (2017). Asymptotic robustness study of the
polychoric correlation estimation. psychometrika, 82(1), 67-85.

Rigdon, E. E., & Ferguson Jr, C. E. (1991). The performance of the
polychoric correlation coefficient and selected fitting functions in
confirmatory factor analysis with ordinal data. Journal of marketing
research, 28(4), 491-497.



Baker, Harley wrote

> I have a number of religious identity labels that participants rate in
> terms of how well they fit. There are five response categories from ‘not
> at all’ to ‘very well.’ As is the case in most research like this, the
> distribution of responses is basically U-shaped. (Basically, a label
> either fits or it does not, very few in-between responses.) Mardia’s
> statistic shows a huge violation of multivariate normality, as one would
> expect. As such, univariate normality is not an issue, which is what I
> should have said originally. Pearson correlations underestimate the degree
> of linear relationship in these situations, and we often turn to
> polychoric correlations instead. (I am loathe to use nonparametric
> correlations when perfectly good parametric methods exist. When
> categorical data are well-behaved, I also prefer to use parametric models.
> I agree with one of my mentors, Fred Lord: the data don’t know where they
> came from. So, if they act parametric, treat them that way. If not,
> don’t.) Polychoric correlations provide a more accurate estimate of the
> true degree of linear association under these conditions. The matrix was
> then submitted to an EFA to see what patterns of self-labeling emerge, if
> any.
>
> Hope this clarifies the situation a bit better.
>
> Harley





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Rich Ulrich
In reply to this post by Baker, Harley
Thanks -
I agree that univariate normality should not be at issue, and I
wonder anew at what your reviewer had in mind.  Showing the
extreme kurtosis should be enough (if there were any question).

From Bruce's citation - I have doubts that the assumptions of
underlying bivariate normality are met, when it comes to polychoric.
If the polychoric r's are larger, then they also have larger standard
errors than the Pearson r's.  I think I had one set of data that I ever
played with, testing polychoric for factoring, and I ended up using
Pearson's.

What also came to mind was Item (or Latent) Response Theory, which
uses logistic transformations. I know IRT from reading, too, rather than
hands-on experience.  I don't know if they have examples with U-shapes.

(a) Underlying Logistic is shaped like Normal with slightly fatter tails, so
it might not meet assumptions any better for U shaped data.  (b) If both
tails are always fat, the correlations won't change by much.  (c) The
people who perform IRT have some interesting ancillary statistics. 

--
Rich Ulrich

From: Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 2:36 PM
To: Rich Ulrich <[hidden email]>
Cc: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
I have a number of religious identity labels that participants rate in terms of how well they fit. There are five response categories from ‘not at all’ to ‘very well.’ As is the case in most research like this, the distribution of responses is basically U-shaped. (Basically, a label either fits or it does not, very few in-between responses.) Mardia’s statistic shows a huge violation of multivariate normality, as one would expect. As such, univariate normality is not an issue, which is what I should have said originally. Pearson correlations underestimate the degree of linear relationship in these situations, and we often turn to polychoric correlations instead. (I am loathe to use nonparametric correlations when perfectly good parametric methods exist. When categorical data are well-behaved, I also prefer to use parametric models. I agree with one of my mentors, Fred Lord: the data don’t know where they came from. So, if they act parametric, treat them that way. If not, don’t.) Polychoric correlations provide a more accurate estimate of the true degree of linear association under these conditions. The matrix was then submitted to an EFA to see what patterns of self-labeling emerge, if any.

Hope this clarifies the situation a bit better.

Harley


Sent from my iPad

On Oct 1, 2020, at 10:52 AM, Rich Ulrich <[hidden email]> wrote:


CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm

I'm a little confused.  You say,
"The data are ordinal in nature, so the issue of normality doesn't even
apply to my situation."

"... the issue doesn't even apply ... "
That implies to me that you are doing analyses after rank-transform
("nonparametric") without considering ordinary ANOVA -- and the
reviewer asks you to justify abandoning the raw scores.

If I were reviewer, I might ask the same.  I hope I would put the
request more clearly than what you seem to have received.

Bruce writes, unclear about the situation -
If it is to justify use of a parametric test, my advice would be, "Don't
bother!"  Like me, Bruce is biased against non-parametric testing for
decently-behaved ordinal data.
--
Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 10:38 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
Hi Bruce,

An editorial reviewer is requiring me to do so. Otherwise, I would not. The data are ordinal in nature, so the issue of normality doesn't even apply to my situation. Regardless, I am complying . . . 

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 5:26 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=Ix%2BBFHa0yn6LxO34w0WvuZtCNdoa5yokfx6gEGNiXWQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=LQAFSVr%2FSx2BNd18u642DIIbTYXsq4PG2plFjzpkWKg%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Jon Peck
In reply to this post by Bruce Weaver
And the SPSSINC HETCOR extension command calculates Pearson, polychoric,and polyserial correlations based on he measurement levels.

On Thu, Oct 1, 2020 at 4:27 PM Bruce Weaver <[hidden email]> wrote:
Thanks for providing that detail, Harley.  For the benefit of readers who may
not be terribly familiar with polychoric correlation, here is the definition
from the relevant Wikipedia page:

"In statistics, polychoric correlation[1] is a technique for estimating the
correlation between two theorised normally distributed continuous latent
variables, from two observed ordinal variables."

And this paragraph from Rigdon & Ferguson (1991) goes even further in saying
that the underlying variables are assumed to have a bivariate normal
distribution:

"The polychoric correlation coefficient, introduced by Pearson, is an
alternative to the Pearson r specifically for situations in which the
variables of interest are continuous but the measurement instruments yield
data that may only be ordinal (Pearson and Pearson 1922; Ritchie-Scott
1918). Olsson (1979b) developed two maximum likelihood procedures for
estimating the polychoric, both based on the assumption that the unseen
underlying variables are continuous and have a bivariate normal
distribution."

Latent variables are a bit outside of my statistical wheelhouse, so maybe
this is a naive question, but I'll ask it anyway!  If the underlying
variables are (approximately) normal or bivariate normal, how can the
distributions of the observed ordinal variables by U-shaped?  I was about to
ask if there is any literature on how non-normality of the latent variables
affects things, but then I found an article by Jin & Yang-Wallentin (2017)
which tells me there has been some work on that question.  I don't have time
to read it right now, but a quick skim suggests that it does not discuss
U-shaped distributions. 

Bruce

PS- I am well aware of what George Box (1976) said about normality and
linearity in the real world, and understand that when we work with real
world data, approximate normality is the best we can hope for. 


References

Box, G. E. (1976). Science and statistics. Journal of the American
Statistical Association, 71(356), 791-799.
http://mkweb.bcgsc.ca/pointsofsignificance/img/Boxonmaths.pdf

Jin, S., & Yang-Wallentin, F. (2017). Asymptotic robustness study of the
polychoric correlation estimation. psychometrika, 82(1), 67-85.

Rigdon, E. E., & Ferguson Jr, C. E. (1991). The performance of the
polychoric correlation coefficient and selected fitting functions in
confirmatory factor analysis with ordinal data. Journal of marketing
research, 28(4), 491-497.



Baker, Harley wrote
> I have a number of religious identity labels that participants rate in
> terms of how well they fit. There are five response categories from ‘not
> at all’ to ‘very well.’ As is the case in most research like this, the
> distribution of responses is basically U-shaped. (Basically, a label
> either fits or it does not, very few in-between responses.) Mardia’s
> statistic shows a huge violation of multivariate normality, as one would
> expect. As such, univariate normality is not an issue, which is what I
> should have said originally. Pearson correlations underestimate the degree
> of linear relationship in these situations, and we often turn to
> polychoric correlations instead. (I am loathe to use nonparametric
> correlations when perfectly good parametric methods exist. When
> categorical data are well-behaved, I also prefer to use parametric models.
> I agree with one of my mentors, Fred Lord: the data don’t know where they
> came from. So, if they act parametric, treat them that way. If not,
> don’t.) Polychoric correlations provide a more accurate estimate of the
> true degree of linear association under these conditions. The matrix was
> then submitted to an EFA to see what patterns of self-labeling emerge, if
> any.
>
> Hope this clarifies the situation a bit better.
>
> Harley





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

Mike
In reply to this post by Rich Ulrich
Peripherally related to the main topic of this thread but related to Rich's
mention of using IRT is the following article:

Harwell, M. R., & Gatti, G. G. (2001). Rescaling ordinal data to interval data
in educational research. Review of Educational Research, 71(1), 105-131.

Abstract
Many statistical procedures used in educational research are described as
requiring that dependent variables follow a normal distribution, implying an
interval scale of measurement. Despite the desirability of interval scales,
many dependent variables possess an ordinal scale of measurement in
which the differences among values composing the scale are unequal in
terms of what is being measured, permitting only a rank ordering of scores.
This means that data possessing an ordinal scale will not satisfy the assumption
of normality needed in many statistical procedures and may produce biased
statistical results that threaten the validity of inferences. This article shows
how the measurement technique known as item response theory can be used
to rescale ordinal data to an interval scale. The authors provide examples of
rescaling using student performance data and argue that educational
researchers should routinely consider rescaling ordinal data using item response theory.

Relative to the use of polychoric correlation, I suggest looking at the section
"Manifest and Latent Variables" on page 110 (6th page of the PDF).

The article is available at the following link:

-Mike Palij
New York University


On Thu, Oct 1, 2020 at 10:38 PM Rich Ulrich <[hidden email]> wrote:
Thanks -
I agree that univariate normality should not be at issue, and I
wonder anew at what your reviewer had in mind.  Showing the
extreme kurtosis should be enough (if there were any question).

From Bruce's citation - I have doubts that the assumptions of
underlying bivariate normality are met, when it comes to polychoric.
If the polychoric r's are larger, then they also have larger standard
errors than the Pearson r's.  I think I had one set of data that I ever
played with, testing polychoric for factoring, and I ended up using
Pearson's.

What also came to mind was Item (or Latent) Response Theory, which
uses logistic transformations. I know IRT from reading, too, rather than
hands-on experience.  I don't know if they have examples with U-shapes.

(a) Underlying Logistic is shaped like Normal with slightly fatter tails, so
it might not meet assumptions any better for U shaped data.  (b) If both
tails are always fat, the correlations won't change by much.  (c) The
people who perform IRT have some interesting ancillary statistics. 

--
Rich Ulrich

From: Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 2:36 PM
To: Rich Ulrich <[hidden email]>
Cc: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
I have a number of religious identity labels that participants rate in terms of how well they fit. There are five response categories from ‘not at all’ to ‘very well.’ As is the case in most research like this, the distribution of responses is basically U-shaped. (Basically, a label either fits or it does not, very few in-between responses.) Mardia’s statistic shows a huge violation of multivariate normality, as one would expect. As such, univariate normality is not an issue, which is what I should have said originally. Pearson correlations underestimate the degree of linear relationship in these situations, and we often turn to polychoric correlations instead. (I am loathe to use nonparametric correlations when perfectly good parametric methods exist. When categorical data are well-behaved, I also prefer to use parametric models. I agree with one of my mentors, Fred Lord: the data don’t know where they came from. So, if they act parametric, treat them that way. If not, don’t.) Polychoric correlations provide a more accurate estimate of the true degree of linear association under these conditions. The matrix was then submitted to an EFA to see what patterns of self-labeling emerge, if any.

Hope this clarifies the situation a bit better.

Harley


Sent from my iPad

On Oct 1, 2020, at 10:52 AM, Rich Ulrich <[hidden email]> wrote:


CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm

I'm a little confused.  You say,
"The data are ordinal in nature, so the issue of normality doesn't even
apply to my situation."

"... the issue doesn't even apply ... "
That implies to me that you are doing analyses after rank-transform
("nonparametric") without considering ordinary ANOVA -- and the
reviewer asks you to justify abandoning the raw scores.

If I were reviewer, I might ask the same.  I hope I would put the
request more clearly than what you seem to have received.

Bruce writes, unclear about the situation -
If it is to justify use of a parametric test, my advice would be, "Don't
bother!"  Like me, Bruce is biased against non-parametric testing for
decently-behaved ordinal data.
--
Rich Ulrich

From: SPSSX(r) Discussion <[hidden email]> on behalf of Baker, Harley <[hidden email]>
Sent: Thursday, October 1, 2020 10:38 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
Hi Bruce,

An editorial reviewer is requiring me to do so. Otherwise, I would not. The data are ordinal in nature, so the issue of normality doesn't even apply to my situation. Regardless, I am complying . . . 

Harley

Dr. Harley Baker
Professor Emeritus of Psychology
California State University Channel Islands
One University Drive
Camarillo, CA 93012
 


From: SPSSX(r) Discussion <[hidden email]> on behalf of Bruce Weaver <[hidden email]>
Sent: Thursday, October 1, 2020 5:26 AM
To: [hidden email] <[hidden email]>
Subject: Re: Shapiro-Wilks Statistic
 
CAUTION: This email originated from outside of CSUCI. Do not click links or open attachments unless you validate the sender and know the content is safe. Please forward this email to [hidden email] if you believe this email is suspicious. For more information on how to detect Phishing scams, please visit https://www.csuci.edu/its/security/phishing.htm


I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-)



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsites.google.com%2Fa%2Flakeheadu.ca%2Fbweaver%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=Ix%2BBFHa0yn6LxO34w0WvuZtCNdoa5yokfx6gEGNiXWQ%3D&amp;reserved=0

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspssx-discussion.1045642.n5.nabble.com%2F&amp;data=02%7C01%7CHarley.Baker%40CSUCI.EDU%7C009a2a7e1919432bf31108d866053d26%7Ce30f5bdb7f18435b84369d84aa7b96dd%7C1%7C1%7C637371519978360100&amp;sdata=LQAFSVr%2FSx2BNd18u642DIIbTYXsq4PG2plFjzpkWKg%3D&amp;reserved=0

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Shapiro-Wilks Statistic

3J LEMA
In reply to this post by Bruce Weaver
Hi, Bruce.

Mine was just an exercise. 

If we don't bother to test normality, why did book authors bothered to discuss normality and parametric  tests. -:) Moreover, why SPSS bothered to include procedures for testing normality? -:)

-:)

3J

On Thu, 1 Oct 2020 at 20:26, Bruce Weaver <[hidden email]> wrote:
I'll repeat the question I asked earlier:

> Why do you [Harley & 3J LEMA] want to test for normality?

If it is to justify use of a parametric test, my advice would be, "Don't
bother!"   ;-) 



3J LEMA wrote
> Any tips on when to use the Shapiro-Wilk statistic over the
> Kolmogorov-Smirnov Statistic in testing normality?
>
> Thank you.





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
12