5 Multi-Arm Bandit Tests to Increase Your Monetization

5 Multi-Arm Bandit Tests to Improve Your Website Monetization

If you’ve ever seen a grandma with seventeen inch biceps, then she likely runs multi-arm bandit tests. She uses it to increase her gambling revenue at the local casino. You’re going to use it to improve your website’s monetization.

There are a lot of different multi-arm bandit tests, or bandit tests for short, you can perform on your banner ads but since you should get testing asap we decided to focus on 5:

  1. Above the Fold vs. Below the Fold
  2. Left Side vs Right Side
  3. Different Ad Sizes
  4. Embedded vs Adjacent to Content
  5. Mobile vs Desktop (Separate setup)

Before you go off googling pictures of seniors who body build, let me explain the analogy. Let’s start with where bandit tests got their name from, what they do, and how they will help increase your ad earnings.

Did you ever see the slot machines at a casino? You put in your money, pull a lever, and wait to see if you win something. Those handles are the “arms”. And the machine itself is the “bandit”. It’s not much of a secret that these machines rob you blind – unless you own them.
the grinch that stole your quarters

The “pros” who play them – quite often little old grandmas – have a strategy. Standing in front of a row of seemingly identical machines, they need to decide which lever or arm to pull, how many times, and in which order. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls[1]. They go through many machines, pulling levers, trying to see which one will pay out the best. I’m sure all that level pulling is quite the workout. Hence our depiction of grandmas with such strength they could be arm wrestling champs. Ba-dum ching!

But how do they know to which money making machine to turn to? That’s the root idea of the multi-arm bandit testing strategy. Here at AdNgin we have created a proprietary bandit algorithm that allows online publishers to quickly understand which of their ad setups will generate them more ad revenue.

We’ll outline 5 bandit tests you can create now to increase your ad revenue. But, before we do that let’s clarify the difference between bandits and A/B tests.

How is bandit testing different from A/B testing?

multi arm bandit vs a/b testing

Courtesy of Conductrics

Let’s keep it simple. With A/B testing, you create two or more versions that you want to test. You send equal amounts of your traffic to each. At the end of the test, you pick a winner. From there, you direct all traffic to the winning version. They call that “exploring” and then “exploiting”.

Still not crystal? Need some more visual aids? Conductrics really has some great info on this so why not share some more of it with you:

There is a little problem in here though. During the course of the test, the losing variations are getting lots of your traffic too. That means they are wasting some of it.

So a bandit test still compares different versions. But it directs most of your traffic to the version performing best from the the outset.

You get to test out new variations like a/b testing, but you reduce the amount of traffic sacrificed.

The eventual losers get less of it to waste, and the likely winners get more of it to turn into ad revenue.

“Exploring” is the testing variations part. “Exploiting” is a bit harsh perhaps. Because what it really means is leveraging your traffic in the best way possible.

Stand by for another negative label. They call the bandit algorithm a “greedy” approach. But that’s just an industry term. It means we want to focus our resources wherever we think we’re going to get the best results.

A/B tesing is a great tool for testing out images and content but unlike A/B tests, bandit tests work faster and are therefore most suited for testing elements that have a direct effect on revenue streams.

If you want to learn more about when to bandit test and when to A/B test read this post from conversion XL.

What are we measuring in our bandit test?

why use bandit tests
Ok, so let the grandmas put their faith in their lucky rabbit foot charms at the slots. For your bandit tests, we need actual KPIs (key performance indicators). Yes, we need to collect some metrics.

Now some of the obvious ones are CTR and Ad RPM (advertising revenue per 1,000 impressions). We need them to go up for obvious reasons if our goal is to earn more ad revenue.

But we caution you to not lose focus on your user experience (UX). Pay close attention to your bounce rate. As the bounce rate increases, it’s more than just lost ad revenue. Not only are they not clicking more ads, they are abandoning your site. Find the sweet spot where monetization and UX balance out.

Our 5 recommended bandit tests for banner ads

Since we’re focused on advertising with our testing, what are some of the factors (i.e. arms) to test?

Bandit test 1 – above the fold vs. below the fold

Mobile users scroll. It’s a fact that’s baked into the way we use mobile devices. This has led certain experts to claim that there is no fold. But don’t believe everything you read. Test it for yourself.

True of False: Everything important on a page has to be visible without scrolling. Test it.

True or False: Your ad groups perform differently when above and below the fold. Test it.

Bandit test 2 – left side vs right side

Many UI designers will talk about F and Z patterns. This relates to how a user reads your page. A lot of research happens here. Researchers want to know where they focus. And they use eye or mouse movement tracking to figure it out.
heat maps for multi arm bandit testing ad revenue

Most web templates provide navigation or advertising on the left or right side of the page. So this also becomes a good testing opportunity for your ad units.

Where are visitors looking at the most? Where are they clicking ads the most?

Bandit test 3 – different ad sizes

multi arm bandit test ad unit size

Size might matter. But we just don’t quite know which size without testing. Almost every ad network has many different ad unit sizes and dimensions. There is quite a variation. Compare traditional banners to skyscraper formats or video embeds.

So the type, size, and dimension of ads are also important factors for testing.

Bandit test 4 – embedded vs adjacent to content

locate your ad units correctly

The traditional ad setup was to include advertising in vacant areas of the page. They could be within your header, footer, and side navigation. It was an attempt to keep them away from your main content. Beyond clutter, it also promotes a clear separation between content and ads.

Smart ad optimizers started to think more in depth than just F and Z patterns. If we expect the user to focus on the content, then why not bring our ad units within the content space? Embed them.

There is also adjacency. That is more like traditional advertising placement. But the key here to explore embedded content and other creative ways to place your ad units.

Bandit test 5 – mobile vs desktop (Separate setup)

create separate ad setups for desktop and mobile
This is the hot zone here. Mobile versus Desktop. Mobile usage is exploding – as we all know. That means no serious web publisher can do well long term without factoring in mobile traffic.

Separate apps are a bit out of scope here. But we can focus in on testing either responsive or separate mobile versions of your site.

With responsive, the design and structure is built to recognize mobile devices. When the smaller screen sizes are in use, the page layout and content can change. The layout may change. Some content may even disappear.

By contrast, a separate mobile site setup is another version especially for mobile devices. With this method, you design a version that is only for target mobile devices. It’s not tethered to the responsible web version. It’s built specifically for mobile.

Putting these 5 bandit tests into action

So now that you know how bandit tests work and have a few suggestions to get you started, We want you to use AdNgin for free, create a couple of bandit tests, and see how it impacts our ad revenue metrics.

Remember that the bulk of your traffic is prioritized only to hit your top performing version so your risk is minimal.

Loss of revenue while testing has a real and obvious cost. This is why we use a bandit algorithm at AdNgin to make sure our users only increase revenue. The bandit tests can help you get insights into what changes might work. And they do it without jeopardizing your ad revenue.

Remember: In the casino, the house wins. In the ad world, testing wins. Now go set up your bandit tests and do a few bicep curls with your grandma.

I’m Head of Marketing Operations at AdNgin. Before coming to AdNgin, I was a marketing professional focused on SAAS business models. When I’m not working, which is rare, I sail and hang out with my son, Jonathan, and wife, Meital.
Start optimizing Page Views
  • Kevin K

    What attracted me to this site is that it’s in words that I understand. I know nothing about this stuff and I’m trying to learn it for my site. This site explained it clearly to me and they don’t use my lack of knowledge against me. They are a very honest company.

  • Taylor H

    I like how this website lets you test it out first. I was able to see what was working and what was not. You never know what’s going to catch people’s attention and this site is very useful when you want to get the best results for your business.

  • Nicky Smith

    I have heard of A/B testing before but never tried it. I didn’t see the purpose. Then when I wasn’t getting results I decided to try it. I came across this site and I had no idea about all the different test they could do with my ads. It’s helped me immensely.

  • Grace J.

    I’ve always did desktop ads and since finding this site and discussing options, I am now doing mobile ads. I’ve seen a big improvement with my site. I’ve never had a company work with me and have my best interest in mind.

  • Shelby

    After long hours of searching online for something to help out my husbands website, I came across this site. This site has helped us learn about ads and what works best for our site because it’s different for everyone. This site has really taken the guess work out of it for us! Thanks again!

  • Pingback: My “Must Try” List of A/B Testing Tools - Compared | AdNgin()

  • Pingback: 10 Tips to Improve Your Stuck AdSense Revenues | AdNgin()