grouping/combining long open ended responses

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

grouping/combining long open ended responses

Art Kendall
It is a long time since I had this kind of problem come up and I presume some progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4 sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips my mind what that was.

If there is not an SPSS module, can list members recommend free or inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major consideration.
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Mike
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message -----
From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


> It is a long time since I had this kind of problem come up and I
> presume some
> progress has been made in how to do it.
>
> There are long string responses to a question like
>
> "What information would be helpful for the ad hoc committee to
> consider when
> working on the topic of X"
>
> Responses go up to maybe 350 characters.  Responses may contain up to
> 4
> sentences.
>
> IIRC SPSS had some software for grouping open ended responses, but i
> slips
> my mind what that was.
>
> If there is not an SPSS module, can list members recommend free or
> inexpensive software to assist this task.
>
> This is a one-time effort for a non-profit group and price is a major
> consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Jon Peck
I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message ----- From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


It is a long time since I had this kind of problem come up and I presume some
progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when
working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4
sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips
my mind what that was.

If there is not an SPSS module, can list members recommend free or
inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major
consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Art Kendall
In reply to this post by Mike
Thank you for your response.
I was not clear.

I do not want to combine variables within cases.

I want to group cases that have similar content.

Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Art Kendall
In reply to this post by Jon Peck
Yes Text Analysis for Surveys is what I was thinking of.

I saw it demo'ed at some meeting.
Art Kendall
Social Research Consultants
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Mike
In reply to this post by Jon Peck

Jon, 
 
Here is a link to a review of STAS that was done a few years
ago.  If you get a chance to read it can you comment on whether
it has become any easier to use?  See:
 
NOTE:  alternative programs/tools are also mentioned.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 12:56 PM
Subject: Re: grouping/combining long open ended responses

I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message ----- From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


It is a long time since I had this kind of problem come up and I presume some
progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when
working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4
sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips
my mind what that was.

If there is not an SPSS module, can list members recommend free or
inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major
consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Jon Peck
I have not had any experience with STAS in several years (and never had a lot before that, either).  Text analysis is a difficult area.  IBM SPSS Modeler has much more extensive capabilities.  

IIRC, the motivation for STAS was to help with open-ended survey questions, not to analyze large documents.  These would typically be part of a survey instrument that would also have traditional questions.  I expect but don't know for sure that it can read sav files, which would make sense given its orientation.

There are free Python and R packages for text analysis, but users are expected to be proficient in the programming language.

On Wed, Jul 5, 2017 at 7:28 PM, Mike Palij <[hidden email]> wrote:
Jon, 
 
Here is a link to a review of STAS that was done a few years
ago.  If you get a chance to read it can you comment on whether
it has become any easier to use?  See:
 
NOTE:  alternative programs/tools are also mentioned.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 12:56 PM
Subject: Re: grouping/combining long open ended responses

I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message ----- From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


It is a long time since I had this kind of problem come up and I presume some
progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when
working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4
sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips
my mind what that was.

If there is not an SPSS module, can list members recommend free or
inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major
consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Mike

Jon,
 
Given how "hot" the Natural Language Processing (NLP) area has gotten
in "data science" it would make sense that an SPSS data file with long
string variables containing responses to specific questions, it would seem
that there would be an increasing need for an program that could read
sav files and provide analyses for the responses to specific questions.
 
The example of analyzing a POTUS state of the union speech seems like
it is oriented toward "narrative analysis" in contrast to "short answer essay
or ad lib information supplementation".  The latter would seem to be
much more useful for questionnaire data as well as data from websites
(e.g., comment sections and so on).
 
I think that latter is what Art is interested in but it does seem to be a
PITA to have to learn a new software package, either STAS or
Modeler, in order to do so.  I have seen references to R based
text analysis (which would be an argument in favor of developing
R skills if one is going to be doing this type of "qualitative" analyses).
 
I have also seen reference to "cleaning up" text sources which
raises its own host of questions and, perhaps, developing skills
with other software packages.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 9:40 PM
Subject: Re: grouping/combining long open ended responses

I have not had any experience with STAS in several years (and never had a lot before that, either).  Text analysis is a difficult area.  IBM SPSS Modeler has much more extensive capabilities.  

IIRC, the motivation for STAS was to help with open-ended survey questions, not to analyze large documents.  These would typically be part of a survey instrument that would also have traditional questions.  I expect but don't know for sure that it can read sav files, which would make sense given its orientation.

There are free Python and R packages for text analysis, but users are expected to be proficient in the programming language.

On Wed, Jul 5, 2017 at 7:28 PM, Mike Palij <[hidden email]> wrote:
Jon, 
 
Here is a link to a review of STAS that was done a few years
ago.  If you get a chance to read it can you comment on whether
it has become any easier to use?  See:
 
NOTE:  alternative programs/tools are also mentioned.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 12:56 PM
Subject: Re: grouping/combining long open ended responses

I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message ----- From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


It is a long time since I had this kind of problem come up and I presume some
progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when
working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4
sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips
my mind what that was.

If there is not an SPSS module, can list members recommend free or
inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major
consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

John F Hall

In her book Designing and Doing Survey Research (Sage 2012)  Lesly Andreas has a section on Atlas.ti  which may do what is needed.  I've never used it, but there's a quick tour  on http://downloads.atlasti.com/quicktour/QuickTour_a8_en.pdf 

 

(Mr) John F Hall

[Retired academic survey researcher]

 

 

 

 

 

 

 

 

> Message du 06/07/17 04:13

> De : "Mike Palij" <[hidden email]>
> A : [hidden email]
> Copie à :
> Objet : Re: grouping/combining long open ended responses
>
>
Jon,
 
Given how "hot" the Natural Language Processing (NLP) area has gotten
in "data science" it would make sense that an SPSS data file with long
string variables containing responses to specific questions, it would seem
that there would be an increasing need for an program that could read
sav files and provide analyses for the responses to specific questions.
 
The example of analyzing a POTUS state of the union speech seems like
it is oriented toward "narrative analysis" in contrast to "short answer essay
or ad lib information supplementation".  The latter would seem to be
much more useful for questionnaire data as well as data from websites
(e.g., comment sections and so on).
 
I think that latter is what Art is interested in but it does seem to be a
PITA to have to learn a new software package, either STAS or
Modeler, in order to do so.  I have seen references to R based
text analysis (which would be an argument in favor of developing
R skills if one is going to be doing this type of "qualitative" analyses).
 
I have also seen reference to "cleaning up" text sources which
raises its own host of questions and, perhaps, developing skills
with other software packages.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 9:40 PM
Subject: Re: grouping/combining long open ended responses

>
I have not had any experience with STAS in several years (and never had a lot before that, either).  Text analysis is a difficult area.  IBM SPSS Modeler has much more extensive capabilities.  

>
IIRC, the motivation for STAS was to help with open-ended survey questions, not to analyze large documents.  These would typically be part of a survey instrument that would also have traditional questions.  I expect but don't know for sure that it can read sav files, which would make sense given its orientation.

>
There are free Python and R packages for text analysis, but users are expected to be proficient in the programming language.

>
On Wed, Jul 5, 2017 at 7:28 PM, Mike Palij <[hidden email]> wrote:
>
Jon, 
 
Here is a link to a review of STAS that was done a few years
ago.  If you get a chance to read it can you comment on whether
it has become any easier to use?  See:
 
NOTE:  alternative programs/tools are also mentioned.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 12:56 PM
Subject: Re: grouping/combining long open ended responses

>
I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

>
On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
>
Art,
>
> I take it that you are referring to something outside of
> the use of concat within a compute statement for a
> "superlong" string (I had worked with similar data
> based on the dissertations of people the NYU psych
> dept and their titles were remarkably long but this was
> back in the 1990s)?  I have just checked the ver 23
> syntax manual for SPSS and could not find anything that
> would simplify the following:
>
> compute Answer01 = concat(part1,part2,part3, ...,partN).
>
> A macro can be written for more complex data arrangements
> in order to concatenate subparts but this all depends upon
> on the structure of the data.
>
> From my searches on the web, there is more sophisticated
> text analysis that can be down within the SPSS modeler
> text analytics program -- for more on this point see:
> https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm
>
> I assume that Jon would be much more knowledgeable about this
> topic and I have a feeling that Python might provide plain SPSS
> greater text analysis capabilities.  Others may also have experience
> with working with this type of data.
>
> However, it seems to me that all you want is to concatenate
> string variables in order to produce a complete "answer".
> Are you concerned that you'll hit length limitations with
> extra long strings?  I think I am missing something.
>
> -Mike Palij
> New York University
> [hidden email]
>
>
> ----- Original Message ----- From: "Art Kendall" <[hidden email]>
> To: <[hidden email]>
> Sent: Wednesday, July 05, 2017 10:23 AM
> Subject: grouping/combining long open ended responses
>
>
>
It is a long time since I had this kind of problem come up and I presume some
> progress has been made in how to do it.
>
> There are long string responses to a question like
>
> "What information would be helpful for the ad hoc committee to consider when
> working on the topic of X"
>
> Responses go up to maybe 350 characters.  Responses may contain up to 4
> sentences.
>
> IIRC SPSS had some software for grouping open ended responses, but i slips
> my mind what that was.
>
> If there is not an SPSS module, can list members recommend free or
> inexpensive software to assist this task.
>
> This is a one-time effort for a non-profit group and price is a major
> consideration.
>

>
=====================
> To manage your subscription to SPSSX-L, send a message to
> [hidden email] (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
>

>

>
--
>
Jon K Peck
> [hidden email]
>
>
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD

>

>
--
>
Jon K Peck
> [hidden email]
>
>
===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD ===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: grouping/combining long open ended responses

Jon Peck
In reply to this post by Mike
For those who want to follow up on SPSS Text Analysis for Surveys, there is a free trial available at


On Wed, Jul 5, 2017 at 8:13 PM, Mike Palij <[hidden email]> wrote:
Jon,
 
Given how "hot" the Natural Language Processing (NLP) area has gotten
in "data science" it would make sense that an SPSS data file with long
string variables containing responses to specific questions, it would seem
that there would be an increasing need for an program that could read
sav files and provide analyses for the responses to specific questions.
 
The example of analyzing a POTUS state of the union speech seems like
it is oriented toward "narrative analysis" in contrast to "short answer essay
or ad lib information supplementation".  The latter would seem to be
much more useful for questionnaire data as well as data from websites
(e.g., comment sections and so on).
 
I think that latter is what Art is interested in but it does seem to be a
PITA to have to learn a new software package, either STAS or
Modeler, in order to do so.  I have seen references to R based
text analysis (which would be an argument in favor of developing
R skills if one is going to be doing this type of "qualitative" analyses).
 
I have also seen reference to "cleaning up" text sources which
raises its own host of questions and, perhaps, developing skills
with other software packages.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 9:40 PM
Subject: Re: grouping/combining long open ended responses

I have not had any experience with STAS in several years (and never had a lot before that, either).  Text analysis is a difficult area.  IBM SPSS Modeler has much more extensive capabilities.  

IIRC, the motivation for STAS was to help with open-ended survey questions, not to analyze large documents.  These would typically be part of a survey instrument that would also have traditional questions.  I expect but don't know for sure that it can read sav files, which would make sense given its orientation.

There are free Python and R packages for text analysis, but users are expected to be proficient in the programming language.

On Wed, Jul 5, 2017 at 7:28 PM, Mike Palij <[hidden email]> wrote:
Jon, 
 
Here is a link to a review of STAS that was done a few years
ago.  If you get a chance to read it can you comment on whether
it has become any easier to use?  See:
 
NOTE:  alternative programs/tools are also mentioned.
 
-Mike Palij
New York University
 
----- Original Message -----
Sent: Wednesday, July 05, 2017 12:56 PM
Subject: Re: grouping/combining long open ended responses

I don't know what the grouping principle is here, but the software you are thinking of might be SPSS Text Analysis for Surveys.

On Wed, Jul 5, 2017 at 9:24 AM, Mike Palij <[hidden email]> wrote:
Art,

I take it that you are referring to something outside of
the use of concat within a compute statement for a
"superlong" string (I had worked with similar data
based on the dissertations of people the NYU psych
dept and their titles were remarkably long but this was
back in the 1990s)?  I have just checked the ver 23
syntax manual for SPSS and could not find anything that
would simplify the following:

compute Answer01 = concat(part1,part2,part3, ...,partN).

A macro can be written for more complex data arrangements
in order to concatenate subparts but this all depends upon
on the structure of the data.

From my searches on the web, there is more sophisticated
text analysis that can be down within the SPSS modeler
text analytics program -- for more on this point see:
https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/com.ibm.spss.ta.help/tas_ta_packages.htm

I assume that Jon would be much more knowledgeable about this
topic and I have a feeling that Python might provide plain SPSS
greater text analysis capabilities.  Others may also have experience
with working with this type of data.

However, it seems to me that all you want is to concatenate
string variables in order to produce a complete "answer".
Are you concerned that you'll hit length limitations with
extra long strings?  I think I am missing something.

-Mike Palij
New York University
[hidden email]


----- Original Message ----- From: "Art Kendall" <[hidden email]>
To: <[hidden email]>
Sent: Wednesday, July 05, 2017 10:23 AM
Subject: grouping/combining long open ended responses


It is a long time since I had this kind of problem come up and I presume some
progress has been made in how to do it.

There are long string responses to a question like

"What information would be helpful for the ad hoc committee to consider when
working on the topic of X"

Responses go up to maybe 350 characters.  Responses may contain up to 4
sentences.

IIRC SPSS had some software for grouping open ended responses, but i slips
my mind what that was.

If there is not an SPSS module, can list members recommend free or
inexpensive software to assist this task.

This is a one-time effort for a non-profit group and price is a major
consideration.

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD



--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD



--
Jon K Peck
[hidden email]




--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Loading...