The Ultimate Guide to Split Testing in Google Ads

Anecdotes are dead. No-one cares about your hunches. Show us some cold hard facts and be gone with you.

Just kidding. Sort of.

In a world increasingly driven by performance marketing, making decisions based on data is essential for an effective marketing strategy.

So, whilst you may think that £49.99 is a more enticing price point than £50.00, you don’t know that unless you’ve tested it.

And when I say tested it, I mean split (AB) tested it.

Split testing is probably what attracts so many people to digital marketing. It essentially means you can act on hunches and anecdotes and prove yourself right.

Or, more interestingly, wrong.

Below is a full guide to split testing in Google Ads, but if you’re in a rush, here are the 5 Golden Rules:

1. Only vary one aspect between the control and the test – so you can determine what has actually driven changes in results.

2. Test until you have reached a level of statistical significance that you feel is appropriate – otherwise, you’re not really testing. 

3. Base your tests around a metric that you want to improve – that way you upgrade your account whilst widening your understanding of your customers.

4. Always run the test at the most granular account level that you can – creating new campaigns every time you test something will quickly make your account a nightmare to manage.

5. You should always be testing something – accounts are dynamic and improving performance is iterative; don’t fall into the trap of believing you’ve found the peak.

What actually is AB testing?

First things first, AB, A/B, and split testing in this context can be used interchangeably (and I’m going to do so – got to get that SEO index up).

AB testing allows you to examine performances in differences between two variants of the same variable by randomly splitting traffic between the two.

A very common example of this in Google Ads is using two different ads in the same ad group and assessing how CPCs, CTR, and CvR vary between the two.

Split testing is definitely not limited to ads, however – we’ll come onto what you can (and should) test.

The most common alternative to AB testing in Google Ads arguably is before-and-after testing. As the name suggests, instead of splitting traffic between two variations, this involves running one variation after another.

Wherever possible, you should be split testing to avoid confounding results from other variables, but we’ll outline some examples of where before-and-after might be more appropriate.

I’m listening. What should I be A/B testing?

Almost everything, but focus initially on whatever will have the largest impact on your account. For example, if your CTR for shopping is poor despite a strong IS, consider addressing your feed before testing Search ads.

Or if you are struggling for volume, think about testing responsive ads with a significant number of headline variations (we’ve seen great results from this).

Below is a list of suggestions to get you started:

  • Search & GDN ads
    • 2 versions of a headline or description, including or not including:
      • Price
      • CTAs
      • Different USPs
      • Brand name
      • Synonyms
      • Different adjectives from the search term
      • Different cases (e.g. title vs sentence)
    • Having third headlines and second descriptions, or not (Search only)
    • Landing pages
    • Using responsive ads
    • Using DKI
    • Countdown vs static sale ads
  • Creatives / images
    • GDN:
      • Product only shots vs lifestyle images
      • Different colours
      • More or less text
    • Shopping:
      • Products only or featuring people
      • White or coloured backgrounds
  • Feed content
    • Order of content in titles and descriptions
    • Including or not including adjectives in titles and descriptions
  • Campaign structure
    • Search
      • Single keyword ad groups vs multiple keyword ad groups
    • Shopping
      • Smart shopping vs not
      • CSS vs not
      • Single product product groups vs multiple product product groups
  • Bid strategies
    • Using one vs manual bidding
    • One strategy vs another
    • Different targets

Remember, when AB testing anything, make sure to only vary one element. For example, do not test one whole ad against another whole ad – you’ll never know whether it was the CTA in the headline or the USP in the description that led to the increased conversion rate.

Well, not without split testing it that is. Just do it first time round and save yourself a load of hassle.

As mentioned, there are some parts of your Google Ads account that aren’t suitable for split testing. The reasons for this are usually that the infrastructure you’d need to build to test would be unsustainable, and the reason you are implementing the changes has already been based on data.

For example, we would not recommend split testing:

  • Weather based bidding
  • Location modifiers
  • Demographic modifiers
  • Device modifiers
  • Audience modifiers
  • Networks

In these instances, it makes more sense to apply your bid adjustments and monitor performance for the specific segment.

There are other cases which you could arguably A/B test, but it might not give you as accurate a picture as before-and-after testing would.

For example, you might run a test that consists of monitoring ROAS across both paid and organic search if you stop bidding on brand terms in the paid channel.

You could have your paid brand campaigns on every other hour, and see what the impact is, but that doesn’t actually reflect the situation if you weren’t bidding.

To test it, you need to just turn the bidding off. Then, you’ll need to look at the cost saved and revenue generated across both channels before-and-after, trying to take into account seasonality.

It’s not very scientific, but if you look year-on-year, and compare to any other benchmark you might have (other territories for example), factoring in account changes that might impact performance, you should have a fairly decent idea whether it was a good move or not.

We’d always recommend A/B testing if feasible, but if you’ve already made a data-driven judgement or the test won’t actually be that meaningful, no pressure.

Well I’m convinced! How do I set up a test?

There are two main concerns when setting up a split test: what level are you going to test at, and what mechanism will you use to test with.

Levels

As mentioned in the golden rules, you should try to set up any experiments at the most granular level possible. This avoids creating a messy account with thousands of redundant campaigns.

Usually, this boils down to testing at either an ad level or a campaign level. The list below outlines a selection of things you’d test at either:

  • Ad level testing
    • Ad copy
    • Landing pages
    • DKI and responsive ads
  • Campaign level testing
    • Structure
    • Bid strategies
    • Shopping campaign types
    • Feeds

It’s worth a note on testing feeds here. We would 100% recommend optimising your shopping feed if you are in e-commerce and haven’t done so previously. To test the optimised feed though would involve setting up a new Google Merchant Center (so a bit more than a new campaign is required).

Legitimately you could also before-and-after test this, but as above, it’s definitely not as scientific.

Mechanism

So you know what you are testing and where you are going to test it, but how do you actually set up the test?

There are a few options open to you (in order of our preference):

1. A script that runs using labels, and will alternate one labeled entity with another, every hour

Scripts would always be our preference, as they are very easy to set up and read the results from. They also much more reliably split traffic relatively evenly, which can’t always be said for Google’s internal mechanisms.

Usually, they run on anything that can be labeled, which is most entities in the interface.

My colleague Wesley wrote a great article here about different tech tools including those that can help with testing – check out 35, 65, and 74 in particular.

2. Drafts and experiments in the Google Ads interface

Drafts and experiments is a good choice in particular for bid strategies. This is because split testing scripts usually involve turning one variant off and another one, which can play havoc with a bid strategy’s testing period.

Drafts and experiments work at campaign level, and you can find Google’s instructions on how to use it here.

3. Google Ads settings

There are some settings within your campaigns that could in theory A/B test for you. For example, you can choose to “optimise: prefer best performing ads” at a campaign level.

In our experience, this rarely splits traffic evenly, and is a classic case of Google black box in terms of what constitutes “best performing”.

When do I know when to conclude a test?

That’s an easy one – once it has reached a level of statistical significance that you are happy with.

Strictly speaking, we’d aim for at least 90%, but on low volume accounts, you may choose to set the bar lower.

Here is an example of a statistical significance calculator that is straightforward to plug your results into for CTR and CvR and get an answer.

How does this all fit with automation?

Given the ever-increasing push towards automation within Google Ads, it can be hard to navigate what should be left to Google to do, and what should be AB tested.

As a general rule of thumb, we’d recommend testing automation versus the manual equivalent. For example:

  • Bid strategies vs manually bidding
  • Responsive ads vs not

We love automation, but ultimately only if it’s driving good results for an account. Bid strategies for the main part can work well, but it’s not unknown to see negative results either.

All in all, split testing is a powerful tool and can unlock some really interesting and powerful findings about your target audiences and within your accounts.

Hopefully you will have found some inspiration here, but do get in touch with us if you need ideas on how to improve performance within your account.

Amy Hawkins
About Amy Hawkins

Amy joined Clicteq in March 2019 as a Paid Search Account Manager. She has a wealth of experience in managing enterprise retail and lead generation paid search accounts including BMW, Hive, Secret Escapes and iRobot.

Leave a Reply

Your email address will not be published. Required fields are marked *