Thus, some of your website visitors are less likely to be chosen to see a variation of your A/B test than the others. In simple words, biased sampling means that you select a sample from the population (all visitors of your website) in such a way that some of the representatives of the population are less likely to be included in this sample. Random sampling means that any visitor of your website has the same probability to be chosen to see a variation of your A/B test.īiased sampling is the opposite to random sampling and it will definitely skew your test results. Random sampling is the essence of any A/B test. What is important is that you are aware of the level of that pollution and does it reach the point where you have to pull the plug on the test. Your results might show you the behavior of, let’s say, 100 visitors, while in fact you might have had only 60 real unique visitors.īefore we get into the different types of sample pollution, let me state that every test will have some sort of sample pollution in it.
Sample pollution might cause you to count the same visitor as two or three different new visitors. “Sample pollution” refers to factors that invalidate your A/B test data by influencing the samples or data used while conducting your test.Īlthough you might have limited or no control over these factors, you must be aware of them and carefully monitor them to understand their impact on your data.
15.2 Related Posts: What Is Sample Pollution and Should You Worry About It?
0 kommentar(er)
