Discussion:
Randomization test with SPSS
(too old to reply)
b***@yahoo.com
2007-10-10 16:02:02 UTC
Permalink
Hello, guys.

I have 1,200 samples obtained from a randomized experiment.
The samples are not random samples.
So, in order to analyze the data, I would like to conduct
randomization tests with SPSS.
However, so far I have not seen functions related to randomization
tests in SPSS.
Is there any way to do randomization tests with SPSS and to deal with
1200 samples in doing randomization tests?
Thank you in advance.

Britramusr
z***@aol.com
2007-10-10 22:49:10 UTC
Permalink
In order to do randomization test, one should know
the distribution assumption. We should do
normality test, uniform dis'n test, and etc. if we.....
Any way it depends on distri'n.

Frank
c***@gmail.com
2007-10-11 04:41:14 UTC
Permalink
Post by z***@aol.com
In order to do randomization test, one should know
the distribution assumption. We should do
normality test, uniform dis'n test, and etc. if we.....
Any way it depends on distri'n.
Frank
Well, as long as I know, randomization tests are not related to
parametric statistics assumptions.
So, even though I can examine the distribution of my samples, for
randomization tests, I don't think that I have to examine the
distribution.
Yes, I can figure out what you tried to say to me. SPSS has several
commands regarding Rv.; which require information of distribution of
samples.
F***@gmx.de
2007-10-12 16:04:01 UTC
Permalink
That's a myth. Randomizations tests (I assume you mean permutation
tests) for e.g. comparing two tests require that the two groups have
the same distribution (= same variances). They only do not need to be
normal distributed (or have normal distributed residuals).
Randomization tests are similar sensitive to violations against this
assumptions as parametric tests. Violations against the assumption of
unequal variances is far more serious than violations of normal
distribution of standard parametric tests (esp. with large sample
sizes). Furthermore, you can not generalize to the population because
you underestimate your variance (you need e.g. bootstrap confidence
intervals to be able to do this).

The only advantage of a randomization test is, therefore, that you do
not need random samples but what does a "significant" result tell
you? Only that the observed pattern of these 1200 people cannot be
explained by chance - not really a surprise. You cannot generalize
this result to a population because it is not a random sample. So the
reuslt of the permutation test is not very useful either.
Can't you use sampling probability weights to adjust for your non-
random sampling?
See: e.g. http://www2.chass.ncsu.edu/garson/PA765/sampling.htm
Cheers, Felix
Richard Ulrich
2007-10-11 03:21:53 UTC
Permalink
Post by b***@yahoo.com
Hello, guys.
I have 1,200 samples obtained from a randomized experiment.
The samples are not random samples.
Okay, if it was a randomized experiment,
WHY (or why to do you *say*) that the samples
are not "random samples." Is this a systematic
subsampling?
Post by b***@yahoo.com
So, in order to analyze the data, I would like to conduct
randomization tests with SPSS.
If we ignore the fact that you have previously been
using the word "random", etc., we would think that
you want to use Fisher's randomization test on the
means. Did you look for that? But that doesn't seem
to be your project....
Post by b***@yahoo.com
However, so far I have not seen functions related to randomization
tests in SPSS.
Is there any way to do randomization tests with SPSS and to deal with
1200 samples in doing randomization tests?
Thank you in advance.
As another poster indicates, "randomization" needs some
notion of a reference. I think we don't have much notion
of what you are looking for.

"Randomization" another way -- If your N=1200 was selected
from a larger study, there would be various ways to test whether
it is a random subset that would make use of the larger, full N.
--
Rich Ulrich, ***@pitt.edu
http://www.pitt.edu/~wpilib/index.html
c***@gmail.com
2007-10-11 04:57:53 UTC
Permalink
Post by Richard Ulrich
Post by b***@yahoo.com
Hello, guys.
I have 1,200 samples obtained from a randomized experiment.
The samples are not random samples.
Okay, if it was a randomized experiment,
WHY (or why to do you *say*) that the samples
are not "random samples." Is this a systematic
subsampling?
Post by b***@yahoo.com
So, in order to analyze the data, I would like to conduct
randomization tests with SPSS.
If we ignore the fact that you have previously been
using the word "random", etc., we would think that
you want to use Fisher's randomization test on the
means. Did you look for that? But that doesn't seem
to be your project....
Post by b***@yahoo.com
However, so far I have not seen functions related to randomization
tests in SPSS.
Is there any way to do randomization tests with SPSS and to deal with
1200 samples in doing randomization tests?
Thank you in advance.
As another poster indicates, "randomization" needs some
notion of a reference. I think we don't have much notion
of what you are looking for.
"Randomization" another way -- If your N=1200 was selected
from a larger study, there would be various ways to test whether
it is a random subset that would make use of the larger, full N.
--
Hello, Dr. Ulrich.

First, the reason that I said "not random samples" is that I am not
likely to use parametric statistical methods.

Second, there are two kinds of randomness: random selection (sampling)
and random assignment. Conventional parametric statistics are based
on the notion of random sampling. However, when we conduct a
randomized experiment based on random selection, we do not have a
population for statistical generalization but will get a reference
after the experiment. The reference is made by my samples.

Third, Fisher's permutation test is also based on a notion of random
sampling; so my samples which are not based on random selection can be
analyzed with parametric statistical methods.

In sum, my samples are not selected at random, so I am not likely to
use conventional parametric statistical methods, which are based on
random sampling. Thus, I would like to do a re-sampling method,
epecially a randomization test.
However, I cannot find out an appropriate function for the test in
SPSS.

Britramus
c***@gmail.com
2007-10-11 05:01:27 UTC
Permalink
Post by Richard Ulrich
Post by b***@yahoo.com
Hello, guys.
I have 1,200 samples obtained from a randomized experiment.
The samples are not random samples.
Okay, if it was a randomized experiment,
WHY (or why to do you *say*) that the samples
are not "random samples." Is this a systematic
subsampling?
Post by b***@yahoo.com
So, in order to analyze the data, I would like to conduct
randomization tests with SPSS.
If we ignore the fact that you have previously been
using the word "random", etc., we would think that
you want to use Fisher's randomization test on the
means. Did you look for that? But that doesn't seem
to be your project....
Post by b***@yahoo.com
However, so far I have not seen functions related to randomization
tests in SPSS.
Is there any way to do randomization tests with SPSS and to deal with
1200 samples in doing randomization tests?
Thank you in advance.
As another poster indicates, "randomization" needs some
notion of a reference. I think we don't have much notion
of what you are looking for.
"Randomization" another way -- If your N=1200 was selected
from a larger study, there would be various ways to test whether
it is a random subset that would make use of the larger, full N.
--
Hello, Dr. Ulrich.

First, the reason that I said "not random samples" is that I am not
likely to use parametric statistical methods.

Second, there are two kinds of randomness: random selection (sampling)
and random assignment. Conventional parametric statistics are based
on the notion of random sampling. However, when we conduct a
randomized experiment based on random assignment, we usually do not
have a
population for statistical generalization but will get a reference
after the experiment. The reference is made by my samples.

Third, Fisher's permutation test is also based on a notion of random
sampling; so my samples which are not based on random selection cannot
be
analyzed with parametric statistical methods.

In sum, my samples are not selected at random, so I am not likely to
use conventional parametric statistical methods, which are based on
random sampling. Thus, I would like to do a re-sampling method,
especially a randomization test.
However, I cannot find out an appropriate function for the test in
SPSS.

Britramus
Richard Ulrich
2007-10-12 23:54:35 UTC
Permalink
Post by c***@gmail.com
Post by b***@yahoo.com
Hello, guys.
[snip, previous]
Post by c***@gmail.com
Hello, Dr. Ulrich.
First, the reason that I said "not random samples" is that I am not
likely to use parametric statistical methods.
We seem to have a disjunction or discordance of
vocabulary -- or something -- that is rather extreme.

There are random samples, there are observational samples,
there are selective samples, and there are odd observations.
ANY of those might be approached with "parametric statistical
methods," with assumptions that vary according to the case, if
there are any statistical methods to be used at all.

The usual contrast to "parametric" is "non-parametric" -- which
most often implies rank-transformations, followed by tests
that borrow the "parametric" methods for large Ns.

My preliminary guess is that you are fairly new to
statistics, and your use of terms does not follow anyone's
conventions. But I'm willing to consider otherwise.

What you (below) seem to be interested in,
"re-randomization" methods, are typically parametric,
though not the standard set. Perhaps you were seeking
the jargon-term, "robust"?
Post by c***@gmail.com
Second, there are two kinds of randomness: random selection (sampling)
and random assignment. Conventional parametric statistics are based
on the notion of random sampling.
- Yes, randomness is the philosophical underpinning. Most
people do not have trouble extending the statistics to
observational studies, with proper cautions about limits of inference.
Post by c***@gmail.com
However, when we conduct a
randomized experiment based on random assignment, we usually do not
have a
population for statistical generalization but will get a reference
after the experiment. The reference is made by my samples.
Huh? Nonsense? If we don't have a population for generalization,
what do we have?
Post by c***@gmail.com
Third, Fisher's permutation test is also based on a notion of random
sampling; so my samples which are not based on random selection cannot
be
analyzed with parametric statistical methods.
Fisher's permutation is often discussed with "nonparametric"
tests, but it does not fit well under the heading.
Post by c***@gmail.com
In sum, my samples are not selected at random,
? No? It sounded like they were.
Post by c***@gmail.com
so I am not likely to
use conventional parametric statistical methods, which are based on
random sampling.
If they are not selected at *random*, you will be best
served, I suspect, by using conservative, conventional
methods to try to account for the non-randomness.
If you can't account for the non-randomness through
*parameters*, you are probably stuck with anecdotes.

If your problem is an awkward variance structure, based
on *dependencies* in the design, then you might want
something non-conventional. (SAS proc ph has 'sandwich
estimators' that are bootstrapped. I don't know if SPSS
offers the same.)
Post by c***@gmail.com
Thus, I would like to do a re-sampling method,
especially a randomization test.
However, I cannot find out an appropriate function for the test in
SPSS.
There are examples available for constructing bootstraps,
etc., (Google groups, < group:comp.soft-sys.stat bootstrap >
but these are not necessary or useful for most procedures,
for most data, in SPSS.
--
Rich Ulrich, ***@pitt.edu
http://www.pitt.edu/~wpilib/index.html
Loading...