A/B Test Calculator
Enter visitor and conversion counts for variants A and B to evaluate statistical significance with a two-tailed z-test.
Variant A
Variant B
Conversion rate A
12.00%
Conversion rate B
15.00%
Difference (B − A)
3.00%
Standard error: 0.0153
Z-score: 1.9630
Two-tailed p-value: 0.0496
95% CI for difference: [0.00%, 6.00%]
How to Use This Calculator
- Enter visitor and conversion counts for both variants.
- Review conversion rates and their difference.
- Use the z-score, p-value, and confidence interval to judge statistical significance.
- Make product or marketing decisions based on whether variant B significantly outperforms variant A.
Formula
Conversion rate = conversions / visitors
Difference Δ = rateB − rateA
Pooled p = (conversionsA + conversionsB) / (visitorsA + visitorsB)
Standard error = √[p(1 − p)(1/nA + 1/nB)]
Z = Δ / SE, p-value = 2 × (1 − Φ(|Z|))
Φ denotes the standard normal cumulative distribution function. Assumes large-sample z-test conditions.
Full Description
A/B testing compares two variants by tracking conversions. This calculator runs a two-proportion z-test using pooled variance, returning the z-score, p-value, and confidence interval for the difference in conversion rates.
Validity improves with larger sample sizes and balanced groups. For small samples or skewed data, consider exact tests or Bayesian methods.
Frequently Asked Questions
Can I run a one-tailed test?
This tool reports two-tailed p-values. For one-tailed tests, halve the reported p-value if the effect direction matches your hypothesis.
What about unequal sample sizes?
The pooled variance accounts for differing visitor counts. Ensure conversions do not exceed visitors.
Is it valid for very small counts?
The normal approximation may be unreliable with low counts. Consider Fisher's exact or Bayesian alternatives in that case.
How do I interpret the confidence interval?
If the 95% CI excludes zero, the difference is statistically significant at α = 0.05.