Historical query regarding SPSS functionality

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Historical query regarding SPSS functionality

margmacd

Season’s greetings!

 

I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:

 

  • the McNemar-Bowker test as an extension to the McNemar test

 

and

 

  • the Fisher-Freeman-Halton test as an extension to Fisher’s Exact test?

 

Thanks in advance

Best wishes
Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education

(Senior Lecturer)
Centre for Population Health Sciences
Usher Institute

University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG
Email: [hidden email]

https://www.ed.ac.uk/profile/margaret-macdougall

 

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Mike
I think that Jon Peck will have to join this thread in order to correct errors or clarify
points in my presentation.  That being said, consider the following:

(1)  If I am not mistaken, prior to the 1990s, SPSS was not able to perform "exact tests"
that are represented by the McNemar-Bowker test and the Fisher-Freeman-Halton.
This was a general limitation of the major statistical software packages (e.g., SAS)
because of hardware and programming limitations.

(2)  The statisticians Cyrus Mehta and Nitin Patel had worked on computer programming
of exact tests and developed the software StatXact (misc statistical tests) and LogXact
(exact logistic regression).  Their algorithms were able to overcome the limitations of
previous implementations and was used in the development of commercial statistical
software.  Version 1 of the PC version of StatXact was released around 1990 and here
are a couple reviews of the early software:

Mehta, C. (1991). StatXact: A Statistical Package for Exact Nonparametric Inference.
The American Statistician, 45(1), 74-75. doi:10.2307/2685246

Sprent, P., & Mehta, C. (1990). Journal of the Royal Statistical Society. Series C
(Applied Statistics), 39(3), 391-395. doi:10.2307/2347397

StatXact was developed on mainframe and similar computers but StatXact was
developed specifically for MS-DOS systems.  Subsequent versions continued
to be available across different computer platforms.

(3)  What is significant about Mehta and Patel's work is that it was easier for
the major statistical packages to either provide the capability to call StatXact
(if it were on the computer system) or the algorithms could be licensed into
the software.  I believe that SAS was the first system to incorporate M&P's
exact test procedure.  By 1995 SPSS provide an exact tests module and
incorporated some of the tests into existing procedures (e.g., crosstabs;
not sure which version of SPSS this was).  Hilbe (2004) wrote a review of
SPSS ver 12 in which he focuses on the logistic regression procedure
(apparently the LogXact software was not used by SPSS) and the Exact
tests module -- he covers some of the history between SPSS and StatXact
which might be useful.  See:

Hilbe, J. (2004). A Review of SPSS 12.01, Part 2. The American Statistician,
58(2), 168-171. Retrieved December 31, 2020, from http://www.jstor.org/stable/27643530

(4)  Jon Peck might know if StatXact code was used in SPSS in version before
version 12 and may be able to give a better/more accurate/exact? date (I bet
it is between 1993 to 1995).

I am curious though as to why you are interested in this question.  I would also
suggest searching Jstor (which has ASA journal and other relevant statistical
journals) for reviews of software and related issues.

-Mike Palij
New York University

On Thu, Dec 31, 2020 at 1:26 PM MACDOUGALL Margaret <[hidden email]> wrote:

Season’s greetings!

 

I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:

 

  • the McNemar-Bowker test as an extension to the McNemar test

 

and

 

  • the Fisher-Freeman-Halton test as an extension to Fisher’s Exact test?

 

Thanks in advance

Best wishes
Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education

(Senior Lecturer)
Centre for Population Health Sciences
Usher Institute

University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG
Email: [hidden email]

https://www.ed.ac.uk/profile/margaret-macdougall

 

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Jon Peck
It is true that the Fisher-Freeman-Halton and McNeman-Bowker tests depend on the StatXact code via the METHOD subcommand of CROSSTABS and require the Exact Tests option.  The Command Syntax Reference manual started listing new and updated syntax as of SPSS version 13.  This subcommand is not mentioned as a change, so it would have been added in version 12 or earlier, but I don't have an exact date.  I no longer have access to the SPSS code base, which would have an exact record of the changes.  My recollection is that integrating the StatXact code, which was fortunately not my task, was a nightmare due to the way that code was written.

On Thu, Dec 31, 2020 at 1:49 PM Michael Palij <[hidden email]> wrote:
I think that Jon Peck will have to join this thread in order to correct errors or clarify
points in my presentation.  That being said, consider the following:

(1)  If I am not mistaken, prior to the 1990s, SPSS was not able to perform "exact tests"
that are represented by the McNemar-Bowker test and the Fisher-Freeman-Halton.
This was a general limitation of the major statistical software packages (e.g., SAS)
because of hardware and programming limitations.

(2)  The statisticians Cyrus Mehta and Nitin Patel had worked on computer programming
of exact tests and developed the software StatXact (misc statistical tests) and LogXact
(exact logistic regression).  Their algorithms were able to overcome the limitations of
previous implementations and was used in the development of commercial statistical
software.  Version 1 of the PC version of StatXact was released around 1990 and here
are a couple reviews of the early software:

Mehta, C. (1991). StatXact: A Statistical Package for Exact Nonparametric Inference.
The American Statistician, 45(1), 74-75. doi:10.2307/2685246

Sprent, P., & Mehta, C. (1990). Journal of the Royal Statistical Society. Series C
(Applied Statistics), 39(3), 391-395. doi:10.2307/2347397

StatXact was developed on mainframe and similar computers but StatXact was
developed specifically for MS-DOS systems.  Subsequent versions continued
to be available across different computer platforms.

(3)  What is significant about Mehta and Patel's work is that it was easier for
the major statistical packages to either provide the capability to call StatXact
(if it were on the computer system) or the algorithms could be licensed into
the software.  I believe that SAS was the first system to incorporate M&P's
exact test procedure.  By 1995 SPSS provide an exact tests module and
incorporated some of the tests into existing procedures (e.g., crosstabs;
not sure which version of SPSS this was).  Hilbe (2004) wrote a review of
SPSS ver 12 in which he focuses on the logistic regression procedure
(apparently the LogXact software was not used by SPSS) and the Exact
tests module -- he covers some of the history between SPSS and StatXact
which might be useful.  See:

Hilbe, J. (2004). A Review of SPSS 12.01, Part 2. The American Statistician,
58(2), 168-171. Retrieved December 31, 2020, from http://www.jstor.org/stable/27643530

(4)  Jon Peck might know if StatXact code was used in SPSS in version before
version 12 and may be able to give a better/more accurate/exact? date (I bet
it is between 1993 to 1995).

I am curious though as to why you are interested in this question.  I would also
suggest searching Jstor (which has ASA journal and other relevant statistical
journals) for reviews of software and related issues.

-Mike Palij
New York University

On Thu, Dec 31, 2020 at 1:26 PM MACDOUGALL Margaret <[hidden email]> wrote:

Season’s greetings!

 

I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:

 

  • the McNemar-Bowker test as an extension to the McNemar test

 

and

 

  • the Fisher-Freeman-Halton test as an extension to Fisher’s Exact test?

 

Thanks in advance

Best wishes
Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education

(Senior Lecturer)
Centre for Population Health Sciences
Usher Institute

University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG
Email: [hidden email]

https://www.ed.ac.uk/profile/margaret-macdougall

 

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Mike
Hilbe (2004, see full reference below) goes into a fair amount of detail about SPSS ver 12.01
and goes over StatXact's contribution.  I have the feeling that it is unlikely that StatXact
code was included in earlier versions.

On a related note, I dug up my copy of the SPSS Exact Tests ver 21 manual which
has the curious copyright dates of 1989 and 2012 -- 1989 is before the release of
StatXact version 1 (both copyrights of IBM).  I have no idea why 1989 is used especially
since the reference list has no publication for 1989 (does it refer to an unpublished
document?).  Anyway, in the front matter of the manual is the following Acknowledgment:

Acknowledgments
Exact Tests is the result of a collaboration between Cytel Software Corporation and SPSS Inc. The
exact algorithms were developed by Cytel. Integrating the exact engines into the user interface and
documenting the statistical methods in a comprehensive user manual were tasks shared by both
organizations. We would like to thank our fellow developers, Yogesh Gajjar, Hemant Govil, Pralay
Senchaudhuri, and Shailesh Vasundhara of Cytel.

We owe a special debt to Professor Marvin Zelen for creating an exciting intellectual
environment in the Department of Biostatistics at Harvard. He encouraged us to work on a
number of challenging research problems in computational statistics, and this research has
culminated in the development of Exact Tests.

Cyrus R. Mehta and Nitin R. Patel
Cytel Software Corporation and Harvard School of Public Health
Cambridge, Massachusetts
****************************************

Final point:  one of the references is for a 1986 paper that describes a Fortran program
titled FEXACT for the Fisher test of contingency tables.  This leads me to think that
most of StatXact was also probably written in Fortran.  But by the 1990s I would assume
that SPSS was no longer using Fortran code -- wasn't there a conversion to some
version of C by that time (I think SAS had overhauled its code around this time),
If this is true, then I can imagine converting the StatXact Fortran into SPSS C code
would have been a difficult task.  Wasn't this the problem with BMDP?  Just curious.

Take care and have a Better New Year,
-Mike Palij
New York University


On Thu, Dec 31, 2020 at 4:25 PM Jon Peck <[hidden email]> wrote:
It is true that the Fisher-Freeman-Halton and McNeman-Bowker tests depend on the StatXact code via the METHOD subcommand of CROSSTABS and require the Exact Tests option.  The Command Syntax Reference manual started listing new and updated syntax as of SPSS version 13.  This subcommand is not mentioned as a change, so it would have been added in version 12 or earlier, but I don't have an exact date.  I no longer have access to the SPSS code base, which would have an exact record of the changes.  My recollection is that integrating the StatXact code, which was fortunately not my task, was a nightmare due to the way that code was written.

On Thu, Dec 31, 2020 at 1:49 PM Michael Palij <[hidden email]> wrote:
I think that Jon Peck will have to join this thread in order to correct errors or clarify
points in my presentation.  That being said, consider the following:

(1)  If I am not mistaken, prior to the 1990s, SPSS was not able to perform "exact tests"
that are represented by the McNemar-Bowker test and the Fisher-Freeman-Halton.
This was a general limitation of the major statistical software packages (e.g., SAS)
because of hardware and programming limitations.

(2)  The statisticians Cyrus Mehta and Nitin Patel had worked on computer programming
of exact tests and developed the software StatXact (misc statistical tests) and LogXact
(exact logistic regression).  Their algorithms were able to overcome the limitations of
previous implementations and was used in the development of commercial statistical
software.  Version 1 of the PC version of StatXact was released around 1990 and here
are a couple reviews of the early software:

Mehta, C. (1991). StatXact: A Statistical Package for Exact Nonparametric Inference.
The American Statistician, 45(1), 74-75. doi:10.2307/2685246

Sprent, P., & Mehta, C. (1990). Journal of the Royal Statistical Society. Series C
(Applied Statistics), 39(3), 391-395. doi:10.2307/2347397

StatXact was developed on mainframe and similar computers but StatXact was
developed specifically for MS-DOS systems.  Subsequent versions continued
to be available across different computer platforms.

(3)  What is significant about Mehta and Patel's work is that it was easier for
the major statistical packages to either provide the capability to call StatXact
(if it were on the computer system) or the algorithms could be licensed into
the software.  I believe that SAS was the first system to incorporate M&P's
exact test procedure.  By 1995 SPSS provide an exact tests module and
incorporated some of the tests into existing procedures (e.g., crosstabs;
not sure which version of SPSS this was).  Hilbe (2004) wrote a review of
SPSS ver 12 in which he focuses on the logistic regression procedure
(apparently the LogXact software was not used by SPSS) and the Exact
tests module -- he covers some of the history between SPSS and StatXact
which might be useful.  See:

Hilbe, J. (2004). A Review of SPSS 12.01, Part 2. The American Statistician,
58(2), 168-171. Retrieved December 31, 2020, from http://www.jstor.org/stable/27643530

(4)  Jon Peck might know if StatXact code was used in SPSS in version before
version 12 and may be able to give a better/more accurate/exact? date (I bet
it is between 1993 to 1995).

I am curious though as to why you are interested in this question.  I would also
suggest searching Jstor (which has ASA journal and other relevant statistical
journals) for reviews of software and related issues.

-Mike Palij
New York University

On Thu, Dec 31, 2020 at 1:26 PM MACDOUGALL Margaret <[hidden email]> wrote:

Season’s greetings!

 

I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:

 

  • the McNemar-Bowker test as an extension to the McNemar test

 

and

 

  • the Fisher-Freeman-Halton test as an extension to Fisher’s Exact test?

 

Thanks in advance

Best wishes
Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education

(Senior Lecturer)
Centre for Population Health Sciences
Usher Institute

University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG
Email: [hidden email]

https://www.ed.ac.uk/profile/margaret-macdougall

 

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Art Kendall
It would be interesting to know list members' experiences wrt how often and
under what conditions exact tests led to different substantive statements.



-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Jon Peck
In reply to this post by Mike
Mr Pack Rat!

The copyright dates are an artifact of the IBM acquisition of SPSS.  IBM dictated that everything be marked with a 1989 copyright notice - I have no idea why, but there was a lot of SPSS code and other materials that didn't have a copyright date in it at the time, so that may have been an approximation.   IBM required a copyright notice in every single thing that SPSS owned.

The Cytel code was written in C, but it used a number of not so good practices that made it hard to port or adapt.  SPSS was using both Fortran and C code at the time - maybe C++ by then, but had stopped doing new development in Fortran (it used a Ratprep preprocessor for it).  There is still plenty of good Fortran code in the product, but Fortran wasn't the problem for the StatXact code.

On Thu, Dec 31, 2020 at 2:55 PM Michael Palij <[hidden email]> wrote:
Hilbe (2004, see full reference below) goes into a fair amount of detail about SPSS ver 12.01
and goes over StatXact's contribution.  I have the feeling that it is unlikely that StatXact
code was included in earlier versions.

On a related note, I dug up my copy of the SPSS Exact Tests ver 21 manual which
has the curious copyright dates of 1989 and 2012 -- 1989 is before the release of
StatXact version 1 (both copyrights of IBM).  I have no idea why 1989 is used especially
since the reference list has no publication for 1989 (does it refer to an unpublished
document?).  Anyway, in the front matter of the manual is the following Acknowledgment:

Acknowledgments
Exact Tests is the result of a collaboration between Cytel Software Corporation and SPSS Inc. The
exact algorithms were developed by Cytel. Integrating the exact engines into the user interface and
documenting the statistical methods in a comprehensive user manual were tasks shared by both
organizations. We would like to thank our fellow developers, Yogesh Gajjar, Hemant Govil, Pralay
Senchaudhuri, and Shailesh Vasundhara of Cytel.

We owe a special debt to Professor Marvin Zelen for creating an exciting intellectual
environment in the Department of Biostatistics at Harvard. He encouraged us to work on a
number of challenging research problems in computational statistics, and this research has
culminated in the development of Exact Tests.

Cyrus R. Mehta and Nitin R. Patel
Cytel Software Corporation and Harvard School of Public Health
Cambridge, Massachusetts
****************************************

Final point:  one of the references is for a 1986 paper that describes a Fortran program
titled FEXACT for the Fisher test of contingency tables.  This leads me to think that
most of StatXact was also probably written in Fortran.  But by the 1990s I would assume
that SPSS was no longer using Fortran code -- wasn't there a conversion to some
version of C by that time (I think SAS had overhauled its code around this time),
If this is true, then I can imagine converting the StatXact Fortran into SPSS C code
would have been a difficult task.  Wasn't this the problem with BMDP?  Just curious.

Take care and have a Better New Year,
-Mike Palij
New York University


On Thu, Dec 31, 2020 at 4:25 PM Jon Peck <[hidden email]> wrote:
It is true that the Fisher-Freeman-Halton and McNeman-Bowker tests depend on the StatXact code via the METHOD subcommand of CROSSTABS and require the Exact Tests option.  The Command Syntax Reference manual started listing new and updated syntax as of SPSS version 13.  This subcommand is not mentioned as a change, so it would have been added in version 12 or earlier, but I don't have an exact date.  I no longer have access to the SPSS code base, which would have an exact record of the changes.  My recollection is that integrating the StatXact code, which was fortunately not my task, was a nightmare due to the way that code was written.

On Thu, Dec 31, 2020 at 1:49 PM Michael Palij <[hidden email]> wrote:
I think that Jon Peck will have to join this thread in order to correct errors or clarify
points in my presentation.  That being said, consider the following:

(1)  If I am not mistaken, prior to the 1990s, SPSS was not able to perform "exact tests"
that are represented by the McNemar-Bowker test and the Fisher-Freeman-Halton.
This was a general limitation of the major statistical software packages (e.g., SAS)
because of hardware and programming limitations.

(2)  The statisticians Cyrus Mehta and Nitin Patel had worked on computer programming
of exact tests and developed the software StatXact (misc statistical tests) and LogXact
(exact logistic regression).  Their algorithms were able to overcome the limitations of
previous implementations and was used in the development of commercial statistical
software.  Version 1 of the PC version of StatXact was released around 1990 and here
are a couple reviews of the early software:

Mehta, C. (1991). StatXact: A Statistical Package for Exact Nonparametric Inference.
The American Statistician, 45(1), 74-75. doi:10.2307/2685246

Sprent, P., & Mehta, C. (1990). Journal of the Royal Statistical Society. Series C
(Applied Statistics), 39(3), 391-395. doi:10.2307/2347397

StatXact was developed on mainframe and similar computers but StatXact was
developed specifically for MS-DOS systems.  Subsequent versions continued
to be available across different computer platforms.

(3)  What is significant about Mehta and Patel's work is that it was easier for
the major statistical packages to either provide the capability to call StatXact
(if it were on the computer system) or the algorithms could be licensed into
the software.  I believe that SAS was the first system to incorporate M&P's
exact test procedure.  By 1995 SPSS provide an exact tests module and
incorporated some of the tests into existing procedures (e.g., crosstabs;
not sure which version of SPSS this was).  Hilbe (2004) wrote a review of
SPSS ver 12 in which he focuses on the logistic regression procedure
(apparently the LogXact software was not used by SPSS) and the Exact
tests module -- he covers some of the history between SPSS and StatXact
which might be useful.  See:

Hilbe, J. (2004). A Review of SPSS 12.01, Part 2. The American Statistician,
58(2), 168-171. Retrieved December 31, 2020, from http://www.jstor.org/stable/27643530

(4)  Jon Peck might know if StatXact code was used in SPSS in version before
version 12 and may be able to give a better/more accurate/exact? date (I bet
it is between 1993 to 1995).

I am curious though as to why you are interested in this question.  I would also
suggest searching Jstor (which has ASA journal and other relevant statistical
journals) for reviews of software and related issues.

-Mike Palij
New York University

On Thu, Dec 31, 2020 at 1:26 PM MACDOUGALL Margaret <[hidden email]> wrote:

Season’s greetings!

 

I have a historical question! Is anyone able to advise me on when functionality to run the following tests first became available in SPSS:

 

  • the McNemar-Bowker test as an extension to the McNemar test

 

and

 

  • the Fisher-Freeman-Halton test as an extension to Fisher’s Exact test?

 

Thanks in advance

Best wishes
Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education

(Senior Lecturer)
Centre for Population Health Sciences
Usher Institute

University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG
Email: [hidden email]

https://www.ed.ac.uk/profile/margaret-macdougall

 

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD


--
Jon K Peck
[hidden email]



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Mike
In reply to this post by Art Kendall

On Thu, Dec 31, 2020 at 5:14 PM Art Kendall <[hidden email]> wrote:
It would be interesting to know list members' experiences wrt how often and
under what conditions exact tests led to different substantive statements.

I haven't had the need to use exact tests in recent years (decades?) but
in Chapter 2 of the SPSS Exact Tests ver 21 manual (p31) the authors
identify the conditions under which the "asymptotic test/prob" (i.e., usual
test statistic) can be different from the "Exact test/prob".  I copy the
relevant text below:

When to Use Asymptotic P Values
Although the exact p value can be shown to converge mathematically to the corresponding asymptotic
p value as the sample size becomes infinitely large, this property is not of much practical value
in guaranteeing the accuracy of the asymptotic p value for any specific data set. There are many
different data configurations where the asymptotic methods perform poorly. These include small data
sets, data sets containing ties, large but unbalanced data sets, and sparse data sets. A numerical
example follows for each of these situations.
*********************************************************

There are probably other conditions where the obtained/asymptotic test/probability
will differ from the exact test/prob but I think the major distinction might be between
experimental designs and observational studies, with the latter providing more
opportunities for differences between asymptotic vs exact value.  With good experimental
design one should be able to avoid small datasets, unbalanced groupings, sparse
data set, etc.  But this may also depend upon whether the independent variables
are actually under control of the research, the populations that are sampled have
adequate numbers of available participants, and other oddities that can affect data
collection.

-Mike Palij
New York University
 
-----
Art Kendall
Social Research Consultants
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Art Kendall
There are circumstances where results CAN be different.  

From a theoretical and policy perspective, I am a strong believer in
applying the SO WHAT test.

I was interested in hearing about experiences when results  WERE  or WERE
NOT different in a meaningful way in comparing conventional test results and
exact test results. I.e., for many purposes P LT .055, .050. and .045 are
substantively equivalent.

Similarly, on this list (and in other contexts) people have often worried
whether intervals between values are exactly equal. (To me the question is
whether or not the intervals are severely discrepant.)  What I have
suggested is using CATREG to actually test whether there was a substantive
difference in the results under varied level of measurement assumptions.
*To date* I have not heard of a situation when reporting the conventional
interval level assumption results made a substantive difference.  To prevent
problems with reviewers, it helps to add a note that the level of
measurement assumption was looked at.





-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Jon Peck
I have used CATREG on occasion to model Likert scales.  The transformation plots it can produce can be helpful in evaluating the measurement level.  If the transformation plot is pretty linear, it's a good sign that ordinary regression and the like will not be misleading.

On Fri, Jan 1, 2021 at 7:53 AM Art Kendall <[hidden email]> wrote:
There are circumstances where results CAN be different.   

From a theoretical and policy perspective, I am a strong believer in
applying the SO WHAT test.

I was interested in hearing about experiences when results  WERE  or WERE
NOT different in a meaningful way in comparing conventional test results and
exact test results. I.e., for many purposes P LT .055, .050. and .045 are
substantively equivalent.

Similarly, on this list (and in other contexts) people have often worried
whether intervals between values are exactly equal. (To me the question is
whether or not the intervals are severely discrepant.)  What I have
suggested is using CATREG to actually test whether there was a substantive
difference in the results under varied level of measurement assumptions.
*To date* I have not heard of a situation when reporting the conventional
interval level assumption results made a substantive difference.  To prevent
problems with reviewers, it helps to add a note that the level of
measurement assumption was looked at.





-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Andy W
In reply to this post by Art Kendall
Art,

It is pretty common to get higher power for exact tests. So if you have
smaller sample sizes (and assuming the test itself makes sense, many say
Fisher's only is valid if the marginals are fixed), they can make quite a
bit of sense. Here are a few examples I have written about:

 - Quantifying racial bias is peremptory challenges is an example for
Fisher's
(https://andrewpwheeler.com/2014/09/25/quantifying-racial-bias-in-peremptory-challenges/)

 - Benford's test with small samples
(https://andrewpwheeler.com/2017/03/31/using-the-exact-reference-distribution-for-small-sample-benford-tests/).
So this isn't for bivariate tables, but a single set of categories, e.g. A =
25%, B = 50%, C = 25%, etc. and see if they deviate from those null
proportions.

Andy





-----
Andy W
[hidden email]
http://andrewpwheeler.wordpress.com/
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Andy W
apwheele@gmail.com
http://andrewpwheeler.wordpress.com/
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Art Kendall
Thanks for the feedback.

I really like the careful reasoning in the examples.

I have rarely dealt with samples this small except as exercises or in
discovery sampling. Usually when I have seen numbers of cases smaller than a
few hundred it was in an actual experiment.  In quasi-experimental
situations, small numbers of cases were N=n  or close to N=n.

In my later years when the exact tests were available in SPSS, I ran tests
both conventionally and exactly but rarely had fewer than 200 or so cases.
I was aware there could be differences in results, but did not happen on
them.   I never systematically looked at differences in conclusions.



-----
Art Kendall
Social Research Consultants
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|

Re: Historical query regarding SPSS functionality

Bruce Weaver
Administrator
In reply to this post by Art Kendall
A related issue concerns exact vs approximate confidence intervals for
binomial proportions.  As Agresti & Coull (1998) argued, approximate can be
better than exact by some criteria.  

Agresti, A., & Coull, B. A. (1998). Approximate is better than “exact” for
interval estimation of binomial proportions. The American Statistician,
52(2), 119-126.





Art Kendall wrote

> It would be interesting to know list members' experiences wrt how often
> and
> under what conditions exact tests led to different substantive statements.
>
>
>
> -----
> Art Kendall
> Social Research Consultants
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.