5 Crucial Components of Every Successful App Marketing Funnel

Knowing how to market your mobile app is essential in order to stand out from the crowd and build your customer base. In this article, we'll discuss two of the leading models for mobile app ...
Continue Reading
All Posts

How Much Should You Really Spend on A/B Testing?

A/B testing is one of the most powerful tools in a digital marketer's toolbox. While it's easy to speculate about which logo looks better or which email subject lines get the most clicks, it's hard to argue in the face of cold, hard statistics.

By seeing which versions of your web page or marketing materials have the highest performance in terms of conversion rate, you can make smarter, data-driven decisions about the best way to bring in customers.

While it's difficult to deny the utility of A/B testing for your business, however, it can be tempting to go overboard in the effort to tweak and fine-tune your advertisements until perfection.

Unfortunately, marketing budgets aren't unlimited (especially for small businesses), so you need to figure out how you can get the most bang for your buck.

In this article, we'll discuss how much you should be spending on A/B testing, so that you can strike the right balance between receiving valuable results and ruining your return on investment (ROI).


The Return on Investment of A/B Testing

The average cost per click of a Google AdWords campaign is between $1 and $2, and some keywords may cost upwards of $50 for a single click.

Google AdWords Cost Per ClickWith the associated expenses so high, you need to make sure that your digital marketing campaigns are well-oiled machines designed to convert as many people as possible – and you can do that with A/B testing.

Done right, A/B testing has resulted in some amazing victories for marketers.

When President Barack Obama was running for office, his digital marketing team used A/B testing to pick among four different buttons for user signups. The team estimates that the experiment helped pick up 2.8 million additional email addresses, or roughly $60 million in extra donations.

There's also the (in)famous story about Google manager Marissa Mayer (later CEO of Yahoo), who used A/B testing to decide between 41 different shades of blue to use for Google's navigation bar. Today, Google's simple interface with bright primary colors is one of the most recognizable brands in the world.

Despite these successes, you need to make sure that A/B testing is right for your own company's situation.

It's estimated that only 1 in 8 A/B tests actually result in significant change for the organization.


How Much Your Company Should Spend on A/B Testing

In order to decide how much you should be spending on A/B testing, you first need to settle on a set of metrics and KPIs to monitor, and then weigh them to ensure that the benefits of testing outweigh the costs.

A/B Testing Metrics to Monitor

A/B Testing Metrics

There is no single set of metrics and KPIs that will work for every organization when doing A/B testing. However, some of the most common metrics are:


Bounce rate:

The percentage of visitors who leave the web page without visiting the rest of the site.


Conversion rate:

The percentage of visitors who perform a desired action, such as making a purchase or signing up for your mailing list.


Return rate:

The percentage of visitors who return to your website at a later time.


Engagement rate:

The percentage of visitors who engage with a given website feature.


Time on site:

The average length of time that a user spends on your website.


Lifetime value:

The average predicted net profit that a new customer will generate for your business.


Evaluating Your A/B Testing Metrics

Once you've chosen the metrics that are most meaningful to your business, you can do a little math to figure out whether A/B testing makes sense for you.

For example, suppose that you've selected conversion rate and lifetime value as your two most important metrics:


• By performing A/B testing, you find that one of the website designs improves conversion rates by 2 percent, and you estimate that 100 new customers will sign up as a result.


• What's more, you project that these customers will have an average lifetime value of $200.

This means that you can spend a maximum of 100 x $200 = $20,000 on A/B testing to ensure it's still worth your while.


When A/B Testing Isn't Worth It

Pros outweighed by ConsAlthough there are clear benefits to A/B testing, there are also certain cases where it's simply not worth the effort. Below, we'll describe two of these situations.


Not Enough Visitors

Every unique impression on your website counts, which means that you might feel pressured to A/B test all of your visitors. If your sample size isn't large enough, however, your results won't be statistically meaningful.

Generally speaking, you want to have a p-value of less than 0.05 in order to say that a test proves or disproves a given hypothesis. This depends on both the sample size of your survey and the conclusiveness of your results.

If you find that 55 percent of your first 100 visitors prefer a given logo, for example, that's likely not enough users to come to a definitive conclusion yet.

If 95 percent of them prefer the logo, however, you may be able to conclude which logo is preferable without having to survey hundreds of additional users.

What's more, as you test more minor elements, the traffic you need to arrive at a conclusion should increase.

Deciding whether a button should be green or blue will require a great deal more testing than picking between two landing pages for your website.


No Hypotheses

A/B testing is an application of the scientific method: you come up with a hypothesis, run some tests, and then modify your assumptions if need be.

For example, you might notice that an unusually high number of visitors are abandoning their online purchases in their shopping cart. After making this observation, you hypothesize that this is due to a confusing checkout process that causes people to lose interest.

In order to test this hypothesis, you can use A/B testing on your customers: one group sees your original checkout screen, and the second group sees a new screen that you've tried to streamline and make more clear. The results of this experiment will help you decide whether you should change the process.

Without a hypothesis to test, however, you'll be flying blind, testing without a greater purpose to guide your efforts.

As a result, you'll be more likely to make rookie mistakes such as making too many changes at once, or not adding a control group to the experiment.


Final Thoughts

A/B testing is a powerful tool that can provide you incisive insights into your customers' most intimate thoughts and behaviors. Like all tools, however, A/B testing is only as good as the person who's currently using it.

By learning which situations A/B testing is best suited for and applying it in the right ways, you can cut your marketing costs while improving your ROI. Most importantly, remember that A/B testing should be combined with other tools in your marketing toolbox.

Russell Chua
Russell Chua
Content Marketer at Creadits

Related Posts

5 Crucial Components of Every Successful App Marketing Funnel

Knowing how to market your mobile app is essential in order to stand out from the crowd and build your cu...
Continue Reading

Do People Spend More Money on Father's Day or Mother's Day?

Every year, there is a battle between Mother's Day and Father's Day on which day consumers will spend the...
Continue Reading

How Small Businesses can Succeed with Facebook Advertising

Facebook advertising is undoubtedly a popular and powerful way to promote a business.
Continue Reading