Variant B converts at 3.8% and variant A at 3.0%. That looks like a win. But with a few thousand visitors in a test, a gap like that can show up by pure chance.
Enter the visitors and conversions of each variant to see whether the difference is statistically significant. If it isn’t yet, the calculator tells you roughly how many more visitors your test needs.
No account required. Everything runs in your browser and the numbers you enter never leave your device.
- A/B test significance calculator
- How many visitors does your test need?
- How to read the result
- Measure your test without cookies
- Where to find these numbers in Plausible
- How the calculator works
A/B test significance calculator
Enter the numbers from your test:
How many visitors does your test need?
Planning a test? Enter your current conversion rate and the smallest improvement you care about to see how many visitors you need before you can call a winner:
Small improvements need a lot of traffic. Detecting a 20% lift on a 3% conversion rate takes around 14,000 visitors per variant. Detecting a 5% lift takes over 200,000. If your site doesn’t get that kind of traffic, test bigger changes. A new headline or a different offer moves the numbers more than a new button color.
How to read the result
Statistically significant at 95% confidence means that if both variants performed the same, a gap this large would show up by chance less than 5% of the time. The p-value is that chance. The lower it is, the more you can trust the result.
Significant doesn’t mean big. With enough visitors, a tiny lift becomes significant and still isn’t worth shipping. Look at the size of the change too.
Decide how many visitors you need before you start and check the result once you get there. If you check every day and stop the moment the result turns significant, you will ship winners that aren’t real. Results swing a lot in the first days of a test.
Run your test for at least a full week, ideally two, so both weekday and weekend visitors are in it.
Testing more than two variants? Compare each one against your original separately and use 99% confidence. The more comparisons you make, the more likely one of them looks like a winner by luck.
Measure your test without cookies
You don’t need a heavy testing suite to measure an A/B test. Your testing tool or a few lines of your own code show each visitor a variant. Plausible counts how each variant converts without cookies and without collecting personal data, so measuring the test adds nothing to your cookie banner.
See how to measure A/B tests with Plausible for the full setup.
Where to find these numbers in Plausible
Use the number of visitors who converted, not the total number of conversions. A visitor who signs up twice counts once. Counting them twice makes a result look more certain than it is.
Variants on different URLs: Filter your dashboard by your goal, open the Entry Pages tab in the Top Pages report and open its details view. It shows each variant’s total visitors and the visitors who converted (Conversions).
Variants on the same URL: If you send the variant as a custom property (Business plan), open the Properties tab in the Goals section and choose variant. Without a goal filter, the Visitors column shows how many visitors saw each variant. Filter by your goal and the same column shows how many of them converted.
How the calculator works
The significance calculator runs a two-sided two-proportion z-test on the numbers you enter. The sample size calculator assumes your traffic is split evenly between two variants and uses 80% statistical power. That means if the improvement you’re looking for really exists, a test of that size has an 80% chance of detecting it.
Once you have a real winner, use the conversion rate calculator to see what the improvement is worth in extra conversions per month.
