A/B Testing Popups: 14 Experiments, 1 Clear Winner

Visisto is the complete website growth system with built-in A/B testing for all 24+ widget types. We ran 14 controlled experiments across spin wheels, slide-ins, popups, and sticky bars to find the exact combination that maximises email capture rate. The winning setup increased capture rate by 340% over the control. Here's every test, every result, and the final formula.

How we tested: methodology

Every test followed the same protocol to ensure clean results:

• Duration: 14 days minimum per test
• Traffic: 1,000+ unique impressions per variant
• Split: 50/50 random traffic allocation
• Significance: 95% confidence threshold before declaring a winner
• One variable: Each test changed exactly one element
• Platform: Visisto's built-in A/B testing — no external tools

Tests 1–3: Widget type (the biggest lever)

We started with the variable that has the largest impact: what kind of widget visitors see.

Test 1: Standard popup vs spin wheel

• Control: Standard discount popup — "Get 10% off. Enter your email." → 2.1% capture rate
• Variant: Spin wheel with same 10% prize + other prizes → 7.8% capture rate
• Winner: Spin wheel (+271%). Confirmed at 99.8% confidence.

Test 2: Spin wheel vs scratch card

• Control: Spin wheel (6 prizes) → 8.1% capture rate
• Variant: Scratch card (same prizes hidden) → 5.4% capture rate
• Winner: Spin wheel (+50%). The visible prize options create more anticipation than hidden rewards.

Test 3: Popup vs slide-in

• Control: Centre popup with overlay → 2.8% capture rate
• Variant: Bottom-right slide-in → 3.4% capture rate
• Winner: Slide-in (+21%). Less intrusive = less dismissal. But both lose to spin wheels.

Tests 4–6: Trigger timing

Test 4: Page load vs exit intent

• Control: Show immediately on page load → 1.9% capture rate
• Variant: Show on exit intent → 4.2% capture rate
• Winner: Exit intent (+121%). Page load popups also increased bounce rate by 11%.

Test 5: Time delay — 5s vs 15s vs 30s

• 5 seconds: 2.2% capture rate — too aggressive, visitors haven't engaged yet
• 15 seconds: 3.1% capture rate — sweet spot
• 30 seconds: 2.8% capture rate — many visitors leave before seeing the widget
• Winner: 15-second delay. Long enough to build context, short enough to catch most visitors.

Test 6: Scroll depth — 30% vs 50% vs 70%

• 30% scroll: 2.6% capture rate
• 50% scroll: 3.6% capture rate
• 70% scroll: 3.3% capture rate
• Winner: 50% scroll depth. 70% misses visitors who engage but don't scroll all the way.

Tests 7–10: Offer and copy

Test 7: Percentage discount vs free shipping

• Control: "Get 10% off" → 3.2% capture rate
• Variant: "Get free shipping" → 3.7% capture rate
• Winner: Free shipping (+16%). Free shipping feels like removing a penalty; discounts feel like a deal. Loss aversion wins.

Test 8: Specific vs vague CTA copy

• Control: "Subscribe for exclusive deals" → 1.4% capture rate
• Variant: "Get your 10% code now" → 3.1% capture rate
• Winner: Specific (+121%). Always tell visitors exactly what they get.

Test 9: With countdown timer vs without

• Control: Spin wheel, no timer → 7.6% capture rate
• Variant: Spin wheel + 15-min timer after spin → 9.3% capture rate
• Winner: Timer (+22%). The timer activates loss aversion after the visitor has already "won" a prize.

Test 10: Email field before vs after spin

• Control: Enter email → then spin → 5.2% capture rate
• Variant: Spin → see prize → enter email to claim → 7.3% capture rate
• Winner: Email after spin (+40%). Let visitors play first. The prize creates reciprocity.

Tests 11–14: Design and UX

Test 11: Brand colours vs high-contrast

• Control: On-brand muted palette → 6.8% capture rate
• Variant: High-contrast neon/bright palette → 8.0% capture rate
• Winner: High contrast (+18%). Widgets should feel like games, not branded collateral.

Test 12: Number of spin wheel slots (4 vs 6 vs 8)

• 4 slots: 6.9% capture rate — feels limited
• 6 slots: 8.2% capture rate — enough variety, not overwhelming
• 8 slots: 7.4% capture rate — too much choice, prize text shrinks
• Winner: 6 slots (+12% over 4-slot).

Test 13: Prize reveal — instant vs suspense animation

• Control: Instant stop → 7.8% capture rate
• Variant: 3-second suspense animation → 8.4% capture rate
• Winner: Suspense (+8%). The delay builds emotional investment in the outcome.

Test 14: Mobile — partial overlay vs full-screen

• Control: Partial overlay (desktop-style) → 4.2% mobile capture rate
• Variant: Full-screen takeover → 7.1% mobile capture rate
• Winner: Full-screen (+69%). Mobile needs its own design, not a scaled-down desktop popup.

The winning formula: 340% more captures

Combining every winning variant from all 14 tests produces the optimal widget configuration:

• Widget type: Spin wheel (6 slots, high-contrast colours)
• Trigger: Exit intent OR (50% scroll + 15s delay)
• Email timing: After spin (see prize first, then enter email)
• Post-claim: 15-minute countdown timer for prize redemption
• Prize reveal: 3-second suspense animation
• Mobile: Full-screen takeover
• Frequency: Once per visitor per 7 days

This configuration captures 8.3% of visitors — vs 1.9% for a page-load standard popup (the most common setup on Shopify stores). That's a 340% improvement.

IMPACT ON A STORE WITH 10,000 MONTHLY VISITORS
────────────────────────────────────────────────
Before (standard popup):  190 captures/month
After (optimised wheel):  830 captures/month
Difference:              +640 captures/month
Annual extra contacts:   +7,680

At £15 avg subscriber value = £115,200 annual revenue impact

The bottom line

A/B testing isn't optional — it's the difference between a 1.9% and 8.3% capture rate. But you don't need 14 tests. Start with the biggest lever (widget type), then optimise trigger timing and offer. The full formula above took us 6 months of testing. You can implement it in 15 minutes on Visisto.

Visisto includes built-in A/B testing on every plan — free included. No separate testing tool needed. Set up your first experiment today and let the data decide.