A/B Testing

How A/B testing works (and when it doesn’t)

A/B testing compares two variants (A and B) shown to similar users at the same time. The goal is to isolate the impact of one change on a measurable outcome - for example, enquiries, purchases, bookings, or sign-ups. The key is that the traffic is split randomly and the test runs concurrently; otherwise seasonality, campaigns, or news cycles can distort results.

What to test

• Messaging: headline clarity, value proposition, objections, and trust cues.
• UX friction: form length, steps in checkout/booking, navigation, and mobile layout issues.
• Conversion drivers: CTAs, social proof, guarantees, delivery/returns, and pricing presentation.

Good practice checklist

• Test one primary change (or one hypothesis) at a time so you can attribute the lift.
• Pick a primary metric (e.g. conversion rate) and a guardrail metric (e.g. revenue per visit).
• Run the test long enough to cover normal weekly patterns, and avoid stopping early on “good-looking” days.
• Make sure tracking is correct - a broken event makes a test meaningless.

If you don’t have enough traffic for statistical confidence, you can still improve outcomes by running stronger qualitative research (session recordings, form drop-offs, user testing) and shipping changes in batches with careful measurement.