I have some sample data of customers and I am looking at the impact of consistent advertising (e.g. showing the same message) to see the impact of click through rates per session.
The analysis shows an Uplift in click through where a consistent messaging is shown (0.59% up to 0.69%) and I want to test to see if this Uplift is statistically significant.
The data looks something like this;
| Customer | Session | Messaging | Click |
|---|---|---|---|
| 101 | 1 | Consistent | 1 |
| 101 | 2 | Consistent | 0 |
| 102 | 1 | Inconsistent | 0 |
| 103 | 1 | Inconsistent | 1 |
What test i.e. z-test or t-test or F-test should I pursue to determine if the results are significant following consistent advertising?
