How to run a simple A/B test on a business website

How to run an A/B test step by step, how much traffic you need for a reliable answer (often more than small sites get), and what to do if you lack it.

By Sohail Hussain21 November 20267 min read

An A/B test shows two versions of a page to randomly split visitors and compares how many of each convert. To run one, pick a single page and a single change, calculate before you start how many visitors each version needs, split the traffic with a testing tool, let it run for whole weeks until it reaches that number, then read the result against the plan. The honest catch is the traffic: to detect a modest improvement, each version needs thousands or tens of thousands of visitors, which most small business websites do not get in a year. If yours is one of them, there are better uses of your time, covered at the end.

Do you have enough traffic to A/B test?

Answer this first, because it decides whether to start at all.

The number of visitors you need depends on two things: your current conversion rate, and the smallest improvement you want to be able to detect. A low conversion rate and a small improvement need a very large sample. The table uses the usual settings (95% confidence, 80% power) and shows visitors needed per version; double it for the total.

Current conversion rateTo detect a 10% lift20% lift50% liftDoubling
1%about 163,00042,7007,7502,320
2%about 80,70021,1003,8301,140
5%about 31,2008,1601,470440

"20% lift" means relative: a 2% rate rising to 2.4%, not to 22%.

Now apply it. Say your main service page gets 3,000 visits a month and 2% of visitors enquire, which is 60 enquiries. To detect a 20% lift you need about 42,200 visitors in total, which is 14 months of traffic. To detect a doubling you need about 2,300, which is under a month. In other words, a site at that size can only detect very large changes, and the button-colour tests that fill conversion blogs are out of reach. When a client asks for A/B testing on a website design and development project, this sum is the first thing we run together, and it often changes the plan.

Your testing tool's calculator may give slightly different figures depending on its statistical method, but the order of magnitude will be the same. Use the conversion rate calculator to get your current rate per page before you start.

How do you run the test, step by step?

If the traffic is there, the steps are short. Most of the work is in the first three.

Pick one page and one goal

Choose the page with the most traffic that leads to an enquiry: usually a service page or a campaign landing page, sometimes the homepage. Decide the single action you will count: a form submission, a click on the WhatsApp button, a quote request. One goal per test. Counting five goals and picking the one that moved is how false winners are made.

Write down one change and what you expect

"Replacing the headline 'Integrated facility solutions' with 'Office cleaning and maintenance for Dubai offices of 20 to 200 staff' will increase quote requests." One change, a reason, an expected direction. If you change the headline, the image and the form at once and the new version wins, you will not know which change did it. That is acceptable for a big redesign test, as long as you know that is what you are testing.

Set the sample size and end date before you start

Use the table above or your tool's calculator. Write the number down, and work out the end date from your traffic. If the end date is more than two or three months away, stop here and use the alternatives at the end of this post.

Split the traffic with a testing tool

Google Optimize, the free option many small sites used, closed on 30 September 2023, as Google's Optimize sunset notice confirms. As of October 2026, alternatives range from commercial testing platforms to open-source tools such as GrowthBook; check current pricing and limits before choosing. On a custom-built site, a developer can also split traffic on the server, which avoids the flicker some browser-based tools cause when they swap content after the page loads.

Whichever you use, the split must be per visitor, not per page view: a visitor who returns tomorrow must see the same version.

Check both versions work and are tracked

Open both versions on a phone and a desktop. Submit the form on each. Confirm the conversion appears in your reports against the right version. A test where version B's form was silently broken will "prove" that B is worse.

Run it for whole weeks without stopping early

Run for at least one full week, ideally two, so weekdays and weekends are both covered. Do not stop when one version pulls ahead. Early leads are common and often disappear; checking every day and stopping at the first "significant" result makes false winners far more likely. Stop at the sample size you planned.

Read the result against the plan

At the planned sample, compare the conversion rates and look at the significance your tool reports. If the difference is not significant, the answer is that the change made no difference big enough to detect, which is a real result: keep whichever version is simpler or cheaper to maintain. Do not slice the result by device, country and day of the week looking for a segment where B won; with enough slices one always does by chance.

Record it and ship the winner

Write down the page, the change, the dates, the sample, the rates and the decision. Make the winner permanent and remove the test code. A test log stops you running the same test again next year.

How do you keep the test from hurting your search rankings?

Google's guidance on website testing and Google Search sets four rules that matter for a simple test:

  • Do not show Googlebot something different from visitors. That is cloaking. Googlebot should be treated as any other visitor.
  • If the versions are at different URLs, put a canonical tag on the variant pointing to the original. Google prefers this to noindex.
  • If you redirect visitors to the variant, use a 302 (temporary) redirect, not a 301.
  • End the test promptly. Google warns that a test left running long after it is needed can look like an attempt to deceive.

What should you test first?

Changes big enough to detect at your traffic. On a small site, that means changes to what the page says and asks, not how it looks.

What can you do instead if your traffic is too low?

Most company websites are in this group. These methods find problems without needing statistical significance:

  • Watch five people use the page. Ask five people who match your buyer to find out what you do and request a quote, while you watch without helping. Problems that trip up three of five are real.
  • The five-second test. Show the first screen for five seconds and ask what the company does. Our homepage structure guide describes it.
  • Session recordings. A free recording tool shows where real visitors stop scrolling, which fields they abandon, and what they tap that is not a link.
  • Ask the people who enquired. "What nearly stopped you getting in touch?" on the first call produces better ideas than most tests.
  • Before and after. Make the change, then compare equal periods with the same traffic sources. It is weaker evidence than a split test, so only trust large differences, and check nothing else changed. Traffic but no leads covers what else to look at.

For a small site, the test that pays is usually getting the basics right in the first place: one goal, a clear headline, a short form, real proof. Those are steps 1 to 5 of our guide to planning a website, and they do not need a sample size.

Was this guide useful?

Questions

Frequently asked.

How long should an A/B test run?

Until each version reaches the sample size you calculated before starting, and for at least one full week, ideally two, so every weekday is covered. Stopping as soon as one version looks ahead is the most common way tests produce false winners. If the sample would take months, the test is not worth running.

Can I run an A/B test with Google Analytics 4?

GA4 does not split traffic between versions; a testing tool or your developer does that. GA4 can then report on each version if the variant is sent with the visit. Google Optimize, Google's free testing tool, closed on 30 September 2023.

Is a before-and-after comparison as good as an A/B test?

No, because anything else that changed between the two periods, such as season, a campaign or a competitor, is mixed into the result. It is still useful when traffic is too low for a split test. Compare equal periods, keep traffic sources steady, and treat small differences as noise.

Sohail Hussain

Founder, MSH Technologies

Sohail has designed and built websites, web apps and AI products for over 12 years, for clients from Singapore-based industrial groups to early-stage SaaS startups. He writes about what actually worked.

Keep reading

Related guides.

All posts

Your next move

Like what you see? Tell us about it.

A process that takes too long, a website that undersells you, or an idea ready to take shape. Tell us where you are and we will suggest a useful next step.

India · Working with clients worldwide

Sohail, Founder, replies within 1 working day (Mon–Sat, IST). Free and no obligation.