2

I am trying to use a permutation test to test my hypothesis.

I want to make sure I am understanding concept of permutation correctly.

I have control and experimental group. Then I combine them and resample from combined dataset randomly calculating desired statistics. Do this N time and distribute them on a histogram to see where my observed statistics stand.

In the book "practical statistics for data scientists" it says that from combined data (without replacing) resample same size as groupA in my case control group. Then since I have only two groups rest are simply groupB(experimental group)?

So my question simply put, when using permutation test on two groups first you resample size of groupA then rest automatically becomes resample of groupB?

haneulkim
  • 479
  • 5
  • 17

1 Answers1

1

Yes it does, you resample from the whole population without replacement $N_A$ samples, which are associated to group $A$, and the rest of the samples are associated to group $B$. As I understand, you compute the statistic for every possible permutation of the total population and you measure how the actual statistic compares in the distribution of all the permutations.

If your statistic is not extreme in the distribution among all possible permutations, then it's likely that $H_0$ is true. Otherwise, if your statistic is an outlier of that distribution it's likely that $H_0$ is false.

David Masip
  • 6,136
  • 2
  • 28
  • 62