Groups based on LOESS/LOWESS fit

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Groups based on LOESS/LOWESS fit

Juleke2
Hello,

I was wondering if somebody knows how to make groups in SPSS based on a
LOESS/LOWESS fit?

So, I have this scatterplot with approximately 130 points and I would like
to make two groups based on the LOES/LOWESS fit that goes through these
points. So all the points above the LOESS/LOWESS fit line form group 1 and
all the points below this line form group 2.

Is that possible in SPSS? And if so, how does that work?

Many thanks in advance!

- JM




--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Jon Peck
I answered that question on the IBM Predictive Analytics site where you first posted it.

On Mon, Oct 1, 2018 at 5:52 PM Juleke2 <[hidden email]> wrote:
Hello,

I was wondering if somebody knows how to make groups in SPSS based on a
LOESS/LOWESS fit?

So, I have this scatterplot with approximately 130 points and I would like
to make two groups based on the LOES/LOWESS fit that goes through these
points. So all the points above the LOESS/LOWESS fit line form group 1 and
all the points below this line form group 2.

Is that possible in SPSS? And if so, how does that work?

Many thanks in advance!

- JM




--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Bruce Weaver
Administrator
Can you provide a link, Jon?  


Jon Peck wrote
> I answered that question on the IBM Predictive Analytics site where you
> first posted it.
>
> On Mon, Oct 1, 2018 at 5:52 PM Juleke2 &lt;

> j.majoor@

> &gt; wrote:
>
>> Hello,
>>
>> I was wondering if somebody knows how to make groups in SPSS based on a
>> LOESS/LOWESS fit?
>>
>> So, I have this scatterplot with approximately 130 points and I would
>> like
>> to make two groups based on the LOES/LOWESS fit that goes through these
>> points. So all the points above the LOESS/LOWESS fit line form group 1
>> and
>> all the points below this line form group 2.
>>
>> Is that possible in SPSS? And if so, how does that work?
>>
>> Many thanks in advance!
>>
>> - JM
>>
>>
>>
>>
>> --
>> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>>

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
> --
> Jon K Peck

> jkpeck@

>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

David Marso
Administrator
In reply to this post by Jon Peck
https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html



-----
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Bruce Weaver
Administrator
Thanks David.  

I like that bit about "Extensions > Extension Hug".  It should give some
reassurance to all those naysayers who thought SPSS would become a cold and
heartless company after being taken over by IBM.  

:-|  




David Marso wrote

> https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html
>
>
>
> -----
> Please reply to the list and not to my personal email.
> Those desiring my consulting or training services please feel free to
> email me.
> ---
> "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos
> ne forte conculcent eas pedibus suis."
> Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in
> abyssum?"
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Jon Peck
Actually, it refers to the fatal squeeze of a Python.

On Tue, Oct 2, 2018 at 8:18 AM Bruce Weaver <[hidden email]> wrote:
Thanks David. 

I like that bit about "Extensions > Extension Hug".  It should give some
reassurance to all those naysayers who thought SPSS would become a cold and
heartless company after being taken over by IBM. 

:-| 




David Marso wrote
> https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html
>
>
>
> -----
> Please reply to the list and not to my personal email.
> Those desiring my consulting or training services please feel free to
> email me.
> ---
> "Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos
> ne forte conculcent eas pedibus suis."
> Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in
> abyssum?"
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD


--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Bruce Weaver
Administrator
That makes more sense, actually.  

On a slightly more serious note, it would be interesting to see the
proportions of new procedures or capabilities in SPSS that are in each of
the following categories:

1. Native SPSS code (no Python, R, or anything else required)
2. Extensions requiring Python
3. Extensions requiring R
4. Extensions requiring both Python or R
5. Other

I would suggest the start date be whenever programmability was first added.  




Jon Peck wrote
> Actually, it refers to the fatal squeeze of a Python.
>
> On Tue, Oct 2, 2018 at 8:18 AM Bruce Weaver &lt;

> bruce.weaver@

> &gt;
> wrote:
>
>> Thanks David.
>>
>> I like that bit about "Extensions > Extension Hug".  It should give some
>> reassurance to all those naysayers who thought SPSS would become a cold
>> and
>> heartless company after being taken over by IBM.
>>
>> :-|
>>
>>
>>
>>
>> David Marso wrote
>> >
>> https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html
>> >
>> >
>> >
>> > -----
>> > Please reply to the list and not to my personal email.
>> > Those desiring my consulting or training services please feel free to
>> > email me.
>> > ---
>> > "Nolite dare sanctum canibus neque mittatis margaritas vestras ante
>> porcos
>> > ne forte conculcent eas pedibus suis."
>> > Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff
>> in
>> > abyssum?"
>> > --
>> > Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>> >
>> > =====================
>> > To manage your subscription to SPSSX-L, send a message to
>>
>> > LISTSERV@.UGA
>>
>> >  (not to SPSSX-L), with no body text except the
>> > command. To leave the list, send the command
>> > SIGNOFF SPSSX-L
>> > For a list of commands to manage subscriptions, send the command
>> > INFO REFCARD
>>
>>
>>
>>
>>
>> -----
>> --
>> Bruce Weaver
>>

> bweaver@

>> http://sites.google.com/a/lakeheadu.ca/bweaver/
>>
>> "When all else fails, RTFM."
>>
>> NOTE: My Hotmail account is not monitored regularly.
>> To send me an e-mail, please use the address shown above.
>>
>> --
>> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>>

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>
>
> --
> Jon K Peck

> jkpeck@

>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

David Marso
Administrator
In reply to this post by Bruce Weaver
IBM had nothing to do with that.


Bruce Weaver wrote

> Thanks David.  
>
> I like that bit about "Extensions > Extension Hug".  It should give some
> reassurance to all those naysayers who thought SPSS would become a cold
> and
> heartless company after being taken over by IBM.  
>
> :-|  
>
>
>
>
> David Marso wrote
>> https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html
>>
>>
>>
>> -----
>> Please reply to the list and not to my personal email.
>> Those desiring my consulting or training services please feel free to
>> email me.
>> ---
>> "Nolite dare sanctum canibus neque mittatis margaritas vestras ante
>> porcos
>> ne forte conculcent eas pedibus suis."
>> Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff
>> in
>> abyssum?"
>> --
>> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>
>> LISTSERV@.UGA
>
>>  (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>
>
>
>
>
> -----
> --
> Bruce Weaver

> bweaver@

> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Please reply to the list and not to my personal email.
Those desiring my consulting or training services please feel free to email me.
---
"Nolite dare sanctum canibus neque mittatis margaritas vestras ante porcos ne forte conculcent eas pedibus suis."
Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff in abyssum?"
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Jon Peck
In reply to this post by Bruce Weaver
Although most of the information you would need to construct this is available between the CSR and the history information in all the extensions, it would be a lot of work, and it wouldn't cover systemic changes such as the move to Unicode, the big Viewer rewrite etc.  And huge procedures such as simulation and TCM to name just a few shouldn't really be compared with smaller things.

But programmability began with Python in SPSS 14 with big enhancements in 15, 16, 18, and 24.  R programmability began in 16, and Java in maybe 20 or 21.  This brought open source to SPSS Statistics in a way that was impossible before.  That required a LOT of time with legal folks both before and after the IBM acquisition, which occurred roughly in the v18 timeframe.

There are around 60 Python-based extensions and 45 R-based extensions.  There is only one R+Python extension (HETCOR) left that I can recall.  The hybrid extensions were done before there was an R interface.  Python was used to interface with R at that time.  

I designed and wrote most of the extensions.  It is much easier to implement something as an extension in most cases compared to native code, so it was practical to add features that would have a smaller audience.  But Python extensions, particularly, provided a way to extend the central system and transformation engines when doing this natively would have been horribly expensive.  SPSSINC TRANS opened up transformations to the whole world of Python code albeit at a loss of efficiency due to the interfacing overhead.  R extensions allowed a variety of sometimes esoteric statistical methods to be added.  The Python apis also made extensive customization of Viewer output possible, and some of that was subsequently nativized with commands such as OUTPUT MODIFY.  Those are big time savers for production purposes.

I like to think that opening Statistics up through programmability means that when someone asks, can  SPSS do X, the answer now is almost always yes between what's there now and what users can add themselves.



On Tue, Oct 2, 2018 at 11:43 AM Bruce Weaver <[hidden email]> wrote:
That makes more sense, actually. 

On a slightly more serious note, it would be interesting to see the
proportions of new procedures or capabilities in SPSS that are in each of
the following categories:

1. Native SPSS code (no Python, R, or anything else required)
2. Extensions requiring Python
3. Extensions requiring R
4. Extensions requiring both Python or R
5. Other

I would suggest the start date be whenever programmability was first added. 




Jon Peck wrote
> Actually, it refers to the fatal squeeze of a Python.
>
> On Tue, Oct 2, 2018 at 8:18 AM Bruce Weaver &lt;

> bruce.weaver@

> &gt;
> wrote:
>
>> Thanks David.
>>
>> I like that bit about "Extensions > Extension Hug".  It should give some
>> reassurance to all those naysayers who thought SPSS would become a cold
>> and
>> heartless company after being taken over by IBM.
>>
>> :-|
>>
>>
>>
>>
>> David Marso wrote
>> >
>> https://developer.ibm.com/answers/questions/472849/groups-based-on-loesslowess-fit.html
>> >
>> >
>> >
>> > -----
>> > Please reply to the list and not to my personal email.
>> > Those desiring my consulting or training services please feel free to
>> > email me.
>> > ---
>> > "Nolite dare sanctum canibus neque mittatis margaritas vestras ante
>> porcos
>> > ne forte conculcent eas pedibus suis."
>> > Cum es damnatorum possederunt porcos iens ut salire off sanguinum cliff
>> in
>> > abyssum?"
>> > --
>> > Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>> >
>> > =====================
>> > To manage your subscription to SPSSX-L, send a message to
>>
>> > LISTSERV@.UGA
>>
>> >  (not to SPSSX-L), with no body text except the
>> > command. To leave the list, send the command
>> > SIGNOFF SPSSX-L
>> > For a list of commands to manage subscriptions, send the command
>> > INFO REFCARD
>>
>>
>>
>>
>>
>> -----
>> --
>> Bruce Weaver
>>

> bweaver@

>> http://sites.google.com/a/lakeheadu.ca/bweaver/
>>
>> "When all else fails, RTFM."
>>
>> NOTE: My Hotmail account is not monitored regularly.
>> To send me an e-mail, please use the address shown above.
>>
>> --
>> Sent from: http://spssx-discussion.1045642.n5.nabble.com/
>>
>> =====================
>> To manage your subscription to SPSSX-L, send a message to
>>

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
>> command. To leave the list, send the command
>> SIGNOFF SPSSX-L
>> For a list of commands to manage subscriptions, send the command
>> INFO REFCARD
>>
>
>
> --
> Jon K Peck

> jkpeck@

>
> =====================
> To manage your subscription to SPSSX-L, send a message to

> LISTSERV@.UGA

>  (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD





-----
--
Bruce Weaver
[hidden email]
http://sites.google.com/a/lakeheadu.ca/bweaver/

"When all else fails, RTFM."

NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.

--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
--
Jon K Peck
[hidden email]

===================== To manage your subscription to SPSSX-L, send a message to [hidden email] (not to SPSSX-L), with no body text except the command. To leave the list, send the command SIGNOFF SPSSX-L For a list of commands to manage subscriptions, send the command INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Juleke2
In reply to this post by Jon Peck
Thank you very much Jon Peck! It worked :)



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
Reply | Threaded
Open this post in threaded view
|

Re: Groups based on LOESS/LOWESS fit

Juleke2
In reply to this post by Jon Peck
Hi everyone!

@Jon Peck: thanks again for your help with the firs question! It worked out
perfectly!

Now I'm wondering, is it also possible to define more than two groups based
on a LOESS/LOWESS fit? So not only two groups (see my first question in this
post), but like 6 groups based on the percentage distance from the
LOESS/LOWESS fit (see the figure in the attachment for more clarification).

<http://spssx-discussion.1045642.n5.nabble.com/file/t341365/Groups_based_on_LOESS.jpg>

So in this example there are 6 groups:
- All the points below the -60% line = group 1,
- All the points below the -30% line = group 2,
- All the points below the LOESS fit line = group 3,
- All the points above the LOESS fit line = group 4,
- All the points above the 30% line = group 5
- All the points above the 60% line = group 6

Note that the points in group 1 are also in group 2 and 3, but not all the
points of group 3 are in group 1 and 2.  Just as all the points in group 6
are also in group 4 and 5, but not all the points of group 4 are in group 5
and 6.

Is this possible to do in SPSS? And is it possible to draw these
'percentages' lines in SPSS (the pink lines in the picture are not from
SPSS)

Many thanks in advance!

- JM



--
Sent from: http://spssx-discussion.1045642.n5.nabble.com/

=====================
To manage your subscription to SPSSX-L, send a message to
[hidden email] (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD