Understand your banner A/B test results
Read first choices, full acceptance and the confidence interval, and know what a result does and does not show.
1. Read the comparison table
The results compare the two versions in four columns:
- Version: A - Current version or B - Test version.
- First choices: the participating browsers that made an eligible first choice.
- Fully accepted: the first choices that accepted every applicable optional category, also when this was done in the settings.
- Full acceptance rate: Fully accepted ÷ First choices.
Only the first eligible explicit opt-in choice of each browser’s participation counts. Choices with only necessary cookies and partial acceptances are not full acceptances. Previews, restored consent, repeated deliveries, choices without a valid test assignment and unrelated choices are left out, and a coverage notice says when choices were left out.
2. Know what is counted
The denominator is the participating browsers that made an eligible first choice, not everyone who saw the banner. Visitors who make no choice are not counted. A participation is a browser, not a person: clearing storage or using another browser can create a new participation. A version without first choices shows No data, not 0%.
3. Compare with the Statistics page
If 100 first choices include 50 full acceptances, 20 partial acceptances and 30 rejections, the test’s Full acceptance rate is 50%, while Acceptance rate on the Statistics page would be 70% for the same choices.
The Statistics page also counts later explicit choices; the test does not. A visitor who first rejects and later accepts all stays one observation that is not a full acceptance, and the later choice neither replaces nor adds to it. See Understand your consent statistics.
4. Read the difference
The difference is B minus A, in percentage points. For example:
| Version | First choices | Fully accepted | Full acceptance rate |
|---|---|---|---|
| A - Current version | First choices200 | Fully accepted110 | Full acceptance rate55% |
| B - Test version | First choices180 | Fully accepted108 | Full acceptance rate60% |
B minus A is +5 percentage points. That alone does not show a clear difference: whether the difference is clear depends on the confidence interval.
5. Read the 95% confidence interval
The 95% confidence interval shows the range of differences that are consistent with the data. It is calculated with a method that allows for checking the results repeatedly while the test runs. It is not a 95% chance that B is better. These illustrative intervals are not calculated from the example above:
- −3 to +9 percentage points includes zero, so there is no clear difference.
- +1 to +7 percentage points supports higher full acceptance for B.
- −7 to −1 percentage points supports higher full acceptance for A.
6. Understand the status
- Collecting data: at least one version has no first choices yet.
- No clear difference yet: the test is running and the interval includes zero.
- No clear difference after ending: the test has ended and the interval includes zero.
- B has higher full acceptance, or A has higher full acceptance: shown only when the interval supports it.
- Confidence interval unavailable: the calculation cannot be used, so do not pick a winner from the percentages.
No clear difference does not prove that the two versions perform the same.
7. Know the limits of a result
The result covers the whole test so far and all devices; it has no date or device filters, and a breakdown by device would not change the conclusion. Unequal numbers of first choices are normal with a 50/50 split. You can end a test at any time, even after two days; there is no minimum duration or sample size, but less data usually leaves wider uncertainty. Nothing stops or is published automatically.
The result concerns full acceptance among visitors who made a choice, not all visitors, legal quality or revenue.
To end the test, see End or troubleshoot a banner A/B test. For ideas for your next test, see Cookie banner A/B test ideas and examples.