Tecof • September 9, 2026
A/B Testing in Ads: How to Run a Split Test
In brief: what is A/B testing?
A/B testing means running two versions of an ad or a page at the same time, splitting traffic randomly between them, and measuring which one produces more conversions. Half of your visitors see the existing version, called the control. The other half sees a version with one thing changed, called the variant. If the gap between them is too large to be explained by chance, the winner goes live for everyone. The method exists for one reason: to settle which headline, image or button actually earns money with measurement rather than opinion.
A store spending 40,000 TL a month on ads that lifts its click-through rate from 2.1 percent to 2.7 percent with a single headline change brings roughly a quarter more visitors to the site without touching the budget. Same money, more carts. Gains like that rarely come from big ideas; they come from the habit of measuring the difference between two sentences.
The most expensive decision-making method in an ad account is following the instinct of the most senior person in the room. Your creative team argues for one image, your performance team defends a different headline, and both sides have internally consistent reasons. As long as nobody measures, that argument reopens every month.
A properly built test turns the argument into three weeks of data collection and leaves you with two things: a winning version and an explanation for why it won. The second one usually matters more, because you can reuse the same insight in email subject lines, product descriptions and shipping messages.
Control, variant and the mechanics of a test
Every test has four parts: a hypothesis, two versions, randomly split traffic, and one success metric chosen in advance. Miss any of them and what you have is not a test but an observation. Observations mislead.
Start with a hypothesis, not a variant
A good hypothesis follows this shape: “If we move the delivery date directly under the price on the product page, add-to-cart rate will rise, because shoppers do not want to commit before they know when the parcel arrives.” It contains a change, an expected effect and a reason. A variant with no reasoning behind it will not hand you your next test even when it wins; it fixes one screen and stops there.
The fastest source of hypotheses is complaints. Live chat transcripts, return reasons, the question and answer sections on Trendyol and Hepsiburada, the messages people send about shipping. Whatever buyers keep asking is the information your page is missing.
One variable at a time
Change the headline, the image and the button together and you will never know which one moved the number. The rule is one variable per test. Creative testing bends it often, comparing two completely different video ads. You can do that, but then your output is “video A performed better”, not “short video performs better”. Isolating the variable is what turns a result into knowledge you can generalise.
Significance is a question of patience
On day two your variant might be 30 percent ahead. That tells you almost nothing. A 95 percent confidence level means the chance of the observed difference being random noise sits below 5 percent. Google Ads and Meta run this calculation for you. Neither of them will be patient on your behalf, and whoever looks early usually decides wrong.
| Method | What it does | Traffic needed | When it fits |
|---|---|---|---|
| A/B test | Compares two versions of one variable | Low to medium | Sites above roughly 10,000 sessions a month |
| A/B/n test | Three or more versions of the same variable | Medium to high | When the creative pool is wide |
| Multivariate test | Tries combinations of several variables | High | Above 100,000 sessions a month |
| Staged rollout | Releases a change to a small slice of traffic first | Low | Infrastructure and checkout changes |
For small and mid-sized stores the practical answer is almost always the first row. Multivariate tests sound more scientific, but traffic divided across eight combinations never reaches significance in any of them, and three months later you still have zero decisions to show for it.
Setting up an experiment in Google Ads
Tests live in two separate places inside Google Ads, and confusing them wastes a lot of time. If you want to try campaign settings, a bid strategy or targeting, you use the Experiments screen. If you only want to try ad copy, Ad variations gets you there much faster.
Campaign experiments
Open Campaigns in the left menu, then the Experiments tab. The plus button offers options depending on campaign type: search campaign experiment, Performance Max experiment, video experiment. Pick a base campaign and Google creates a copy. Change exactly one thing in that copy, leave the traffic split at 50 percent, and set a start and end date.
While the experiment runs, both campaigns bid from the same pool, so your total spend does not increase. Set the end date at least four weeks out. In search campaigns, smart bidding strategies re-enter a learning phase in the first days, which makes the first week of data misleading almost every time.
Ad variations
For copy tests, go to the Ads screen inside Campaigns and open the Ad variations tool. It lets you build a bulk test of the “find this phrase across the account and replace it with another” kind. You could swap “Free Shipping” for “Same-Day Dispatch” across every headline, apply it to 50 percent of traffic, and read the click-through difference ten days later.
What is worth testing here
- Benefit or reassurance in the headline: “Free delivery over 10,000 TL” against “30-day returns”
- Showing a price range in the description line or leaving it out
- Bid strategy: Target ROAS against Maximise conversions
- Splitting brand searches into their own campaign or keeping them together
- Sitelink extensions pointing at category pages or at promotion pages
A concrete example: a furniture retailer ran two headlines in a search campaign. The control read “Modern Bedroom Sets”, the variant read “Bedroom Set With Free Assembly”. The assembly promise won on click-through rate, but cost per click rose as well. The decision was made on profit per order, not on click-through rate. Which metric misleads you and when is covered in the guide comparing CTR, CPC, CPM and CPA.
Running the same test on Meta Ads
In Meta Ads Manager there are two ways in. You can tick the A/B test option while building a new campaign, or select a live campaign from the list and press the A/B test button above it. The second route is safer when you want to try an idea without disturbing a campaign that already performs.
Your test variable can be creative, audience, placement or optimisation event. Meta keeps the two groups separate, so the same person does not see both the control and the variant. Compared with the older habit of duplicating ad sets and letting them cannibalise each other, that alone changes the quality of the answer.
Duration and budget
Meta suggests four days as a minimum; in practice seven to fourteen days behaves better. Weekday and weekend behaviour diverge sharply in Turkey, and so does the first week of the month against the last, because payday shifts spending. Build tests in whole weeks.
Split the budget evenly between the two groups and leave it alone until the test ends. Looking at a test you interrupted halfway is like reading a spreadsheet with half the rows deleted.
What to try in creative tests
- A square image of the product alone against a vertical video of it in use
- Leading with price in the first three seconds against leading with the problem
- Captioned video against uncaptioned, for the audience watching with sound off
- Customer-shot content against studio photography
- On the audience side: broad targeting against a lookalike built from product page visitors
A cosmetics brand set this up on Instagram: the control was a product shot on white, the variant a fifteen-second application video. The video earned far more engagement, yet add-to-cart rate came out nearly identical in both groups. The lesson was that the video attracted attention without giving a reason to buy, so the next test added price and return terms to the final frame. How to divide budget between the two platforms is worked through in the Google Ads and Meta Ads comparison.
What a test costs: visitors, lira and days
Most tests end without a verdict because the difference you are chasing is smaller than the data you can collect. A rough approximation for the sample you need per variant is 16 × p × (1 - p) ÷ (expected difference)², where p is your current conversion rate. The formula assumes 95 percent confidence and 80 percent power.
For a store converting at 1.5 percent and paying around 4.50 TL per click, the numbers look like this.
| Target conversion rate | Relative lift | Clicks per variant | Total cost of the test |
|---|---|---|---|
| 3.0 percent | Double | About 1,600 | About 14,400 TL |
| 2.25 percent | 50 percent | About 5,200 | About 46,800 TL |
| 2.0 percent | 33 percent | About 11,000 | About 99,000 TL |
| 1.8 percent | 20 percent | About 28,800 | About 259,000 TL |
The table makes one thing plain: small improvements are expensive to prove. A store on a 50,000 TL monthly budget would need months of paid traffic to demonstrate a 20 percent lift. That is exactly why smaller accounts should test bold variants. Not the button colour, but the offer itself: lowering the free-shipping threshold, pushing instalment options forward, rebuilding the product bundle.
Stores without the traffic have another way out, which is to move testing off paid channels. Email, SMS and on-site notifications carry no cost per visitor, so you can try the same hypothesis there in three days and carry the winning message into your ad copy afterwards. That route is cheaper, not slower.
Duration has two floors. One is the sample, the other is the calendar. Even when the sample fills early, never run shorter than a week, because the people who shop on a Tuesday are not the people who shop on a Saturday. Avoid opening tests during 11.11 or Black Friday weeks; you cannot apply that data to a normal month.
Before you declare a winner, look at return on ad spend. A variant that raised click-through rate while lowering average order value has not won anything. The walkthrough of how ROAS is calculated is a good place to work out the threshold below which a variant is quietly losing money.
After the click: page and checkout tests
Ad tests bring the visitor to the door. Money is made inside. Find a headline that performs 30 percent better and fail to repeat that promise on the product page, and you hand the extra clicks straight to your bounce rate.
Four things worth testing on a product page
- Where delivery information sits: under the price or next to the cart button
- Showing the instalment table openly against hiding it behind a link
- Image order: the product itself first, or the product in use
- How visible stock levels are during size or colour selection
Cart and checkout
Cart abandonment hovers around 70 percent according to industry averages, and every point you win back here is cheaper than the same point won on the ad side. The things to try are concrete: enabling guest checkout, showing shipping cost at the first step, moving cash on delivery higher in the list, repeating the order summary before the 3D Secure screen.
One example specific to Turkey: many stores park bank transfer at the bottom of the payment list. On high-value baskets, moving that option directly under credit card and printing the discount next to it is a variant worth measuring. Because the commission differs, the same revenue can leave more profit behind.
Testing inside checkout comes with one unbreakable rule: start with a small slice of traffic when the flow is live. A variant that throws an error on order creation gives back everything it earned in a single afternoon. For ideas about where to start on the page side, look at the ways to lift ecommerce conversion rate and the guide on turning a website into a sales machine.
Which tool
Google Optimize shut down in 2023 and no single successor replaced it. VWO and Optimizely are the common choices for page testing, while Unbounce gets you live faster if you work mainly with landing pages. Measurement can sit in Google Analytics, and marking conversion events server-side as well reduces the data loss caused by ad blockers.
Deciding what to test first
Your test list will always be longer than your capacity. Order it with a simple score instead of a feeling: impact, ease, evidence. Give each one a mark from 1 to 5 and start from the highest total.
| Test idea | Impact | Ease | Evidence | Total |
|---|---|---|---|---|
| Add a delivery date to the product page | 5 | 4 | 4 | 13 |
| Enable guest checkout | 5 | 3 | 5 | 13 |
| State the free-shipping threshold in ad headlines | 4 | 5 | 3 | 12 |
| Change the colour of the add-to-cart button | 2 | 5 | 1 | 8 |
| Rebuild the homepage navigation | 3 | 1 | 2 | 6 |
Evidence is the column most teams skip. Do you have an observation supporting this change? Does the heatmap show people hunting for shipping details, and what share of support tickets mention delivery time? Ideas with no evidence behind them go on the waiting list rather than into the first round.
When you fill in the impact column, ask how many people will see the change and whether seeing it genuinely affects the purchase decision. A banner on the homepage is seen by many but sits far from the moment of choice. A single sentence in checkout reaches fewer people and stands one step away from the decision.
Notice the button colour row. It is the easiest test to build and half the case studies online are about it, yet its effect is usually small. Let ease alone drive your order and you will spend three months on colour experiments and learn nothing.
Review the ranking monthly. One result changes the score of three other ideas: the day delivery information proves itself, the ideas about repeating that message in the cart and in reminder emails gain evidence points too. The list is a living document, not a plan written at the start of a quarter and forgotten.
Keep the outcome of every test in one place. A spreadsheet with the date, the hypothesis, a description of the variant, the sample, the result and the decision taken. Six months in, that sheet teaches a new hire in one sitting what would otherwise take them weeks.
Mistakes that break a test
A badly built test is worse than no test, because it dresses a wrong decision up as data.
Stopping early
Closing the test on day three while the variant leads is the most common error by far. When conversion counts are small, rates swing wildly from day to day. Write the end date at the start and stop opening the dashboard every morning until then.
Testing with too little traffic
On a page that gets 50 visitors a day, the time needed for a meaningful result is measured in months. Test on the category page that carries real traffic instead, or skip testing there and simply apply the well-established improvements.
Changing something else mid-test
Launching a shipping promotion, updating prices or sending a campaign email while the test runs contaminates the result. Open your test calendar next to your marketing calendar.
Ignoring the device split
Most ecommerce traffic in Turkey arrives from mobile. A layout that wins on desktop can lose on mobile because the cart button falls below the fold. Segment results by device at the very least.
Counting the same person twice
A visitor who arrives on a phone and returns the next day on a laptop can land in both groups, which shrinks the measured difference. If you can tie group assignment to a logged-in user id rather than a cookie, do it. If you cannot, read the result knowing the gap is understated.
Never shipping the winner
The test ends, a report is written, the file goes into a folder and the site stays the same. Every test needs an owner and a date for implementation. A test you never apply is spent budget.
Watching the wrong metric
Click-through rate may be up while returns are up too, which makes the net effect negative. Choose the success metric before you build the test, and make it a revenue or profit metric whenever you can.
Forgetting the season
School terms, religious holidays, discount weeks and payday cycles all change behaviour. Testing a variant during a holiday week and applying the outcome in February will mislead you.
What to account for when testing in Turkey
Building a testing setup involves legal groundwork as well as technical work. Under KVKK you need explicit consent for the cookies that track visitor behaviour, and visitors who decline never enter your measurement pool. Your test sample will therefore be smaller than your real traffic. Know your consent rate and size your sample against that number, not the raw one.
For email and SMS variants, İYS registration is mandatory. When you test a subject line, you send to the recipients who appear as opted in on İYS rather than to the whole list. Since permission status changes over time, splitting groups by permission would skew them, so split randomly and then filter.
Marketplaces work differently. On Trendyol and Hepsiburada you do not control the page layout, but title, image order, price and shipping promise are still yours. Listing the same product under two titles in two different periods and comparing sales velocity is not a true A/B test, because time enters as a second variable. Carrying a title that won on your own site over to the marketplace still beats carrying an untested one.
Payment and shipping variants
Instalment tables from local payment providers differ from bank to bank. Which bank's instalment plan appears first in the cart is a decision you can test on its own. The same applies to logistics: making the fastest carrier the default may lift conversion while raising cost per parcel. Put the two numbers side by side before deciding.
Your platform sets the pace of all of this. If every variant needs a developer, you will run one test a month; if you can edit page sections from an admin panel, you will run one a week. On the Tecof ecommerce platform product page sections and the cart flow are editable from the panel, so building a variant takes minutes rather than a sprint.
Here is the job for tomorrow morning. Open the three highest-spending campaigns of the last thirty days, copy the ad copy with the most impressions, and write a second version that changes one promise. Set the test up from the Experiments screen in Google Ads or the A/B test button in Meta, put the end date fourteen days out, and leave the dashboard alone until then.
Frequently Asked Questions
Is there any difference between A/B testing and split testing?
In practice both describe the same method. Some sources reserve split testing for setups where two pages live on entirely different URLs, and use A/B testing when content changes on the same address. Ad platforms never draw that line. When you see either term, you can assume the same thing is meant.
How long should a test run?
Seven days is the floor, because the test has to cover the behavioural difference between days of the week. The ceiling is set by your sample; most ecommerce tests finish somewhere between fourteen and twenty-eight days. If you use smart bidding, treat the first seven days as a learning phase and keep that data out of the decision. A test heading past thirty days usually means the difference you are chasing is too small.
How many visitors and conversions do I need?
As a rough threshold, aim for at least 100 conversions per variant. The exact figure depends on your current conversion rate and the size of the difference you want to detect. Measuring a jump from 1.5 percent to 3 percent needs roughly 1,600 visitors per variant, while measuring a move to 1.8 percent needs more than 28,000. Smaller accounts should therefore test bolder changes.
What does statistical significance actually mean?
It tells you the observed difference is unlikely to be random. A 95 percent confidence level means the probability of the gap appearing by chance is under 5 percent. That is not a guarantee: roughly one test in twenty will crown a false winner. For high-stakes decisions you can raise the bar to 99 percent, though the data requirement climbs noticeably.
Where do I set up a test in Google Ads?
Under Campaigns in the left menu, on the Experiments screen. Press the plus button, choose the campaign type, pick a base campaign, make one change in the generated copy and leave the traffic split at 50 percent. If you are only trying ad copy, the Ad variations tool on the Ads screen is quicker. Results and the significance indicator appear inside the experiment card.
How do I start a test in Meta Ads?
In Ads Manager, select a campaign from the list and press the A/B test button, or tick the A/B test option while creating a new campaign. Choose creative, audience, placement or optimisation event as the variable. Split the budget evenly between the two groups and run for at least seven days. Meta keeps the groups apart, so one user never sees both variants.
Can I test several things at once?
Technically yes, but not if your traffic cannot support it. Two versions of three variables means eight combinations, and each combination has to reach significance on its own. Below 100,000 sessions a month, stay with single-variable tests. Sequential tests try fewer combinations in the same period but give you a much clearer answer.
What should I do when a test ends in a tie?
A tie still tells you something: the change you tried did not matter to buyers. Retry the same idea in a bolder form, or move to the next hypothesis on the list. If you keep landing on ties, what you are testing is probably too small, and it is time to touch the offer itself, the pricing or the shipping terms.
How much budget should a small store put into testing?
Somewhere between 10 and 20 percent of monthly ad spend is a workable share. For a store running on 30,000 TL a month that means 3,000 to 6,000 TL, enough to run one serious test each month. On tighter budgets, starting with email subject lines or product page tests teaches you faster, since there is no cost per visitor there.
What happens once the winning variant is live?
The new version becomes your control and the next test builds on it. After shipping a winner, watch the conversion rate for two weeks; there are cases where the gain measured in the test does not repeat one to one in production. Write the outcome into your test log and list the other screens where the same insight could apply.