What is A/B testing and what constitutes a valid test design in marketing experiments?

• A. Observational study using existing data
• B. Non-randomized comparison of two variants
• C. Randomized experiment with multiple variables
• D. Randomized experiment with two variants, a control group, sufficient sample size, and isolating a single variable

Answer: (D)

A/B testing is a controlled experimentation approach where you compare two variants to see which performs better. In marketing experiments, a valid test design means you randomly assign participants to one of two versions, include a clear baseline or control, ensure you isolate a single variable so the effect can be attributed to that change, and have enough participants to reliably detect a real difference.

This matches the option describing a randomized experiment with two variants, a control group, sufficient sample size, and isolating a single variable. Two variants allow direct comparison; random assignment reduces bias; the control provides a baseline; a sufficient sample size gives statistical power; isolating a single variable ensures the observed effect is due to that variable rather than other changes.

Observational studies lack randomization and control, making causal attributions unreliable. Non-randomized comparisons can be biased by how participants are chosen. Randomized experiments with multiple variables muddy the results because multiple changes are tested at once, preventing clear attribution of effects to any single factor.