hero ab testing tools compared

My “Must Try” List of A/B Testing Tools – Compared

A/B testing has become so pervasive that data nerds (that’s us) are using it to optimize their dating profiles. Let’s see if the profile picture with me wearing my favorite blue shirt while petting my dog gets more clicks and emails. You know, in direct comparison with the beach picture where it looks like I took a bath in tanning oil. It was only half a bottle! There were cervezas involved. No kidding. It happens. Don’t judge.

ab testing tool comparison2

Long before people started doing conversion optimization to get more dates, bloggers got really good at using A/B testing tools too. More sign-ups, sales, and CPC ad clicks.

The results are too good not to try. And if you didn’t, then you’ve been losing out on conversions and earnings.

As A/B testing exploded in popularity (like the picture of me and my kitten wearing matching New Year’s hats), more tools started to emerge. So let’s explore some of the top tools out there.

Here’s a complete list of the best ones on our must-watch list. If you haven’t tried at least one of them already, you may need an intervention. It’s time to get started!

FYI Although we don’t like to think of ourselves as an A/B testing tool, We round out this list with AdNgin (that’s us).  Our specialty is in testing for the best ad placement and ad network configuration, optimizing and earning you more.

So far, sounds pretty A/B testy, right?! But, our core algorithm is a multi arm bandit algorithm so what AdNgin does is in fact bandit testing, which is like A/B testing on steroids (more on bandit testing can be found here).

A quick warning. There are a couple examples we included that require some coding. If you’re not a coder, just hold your breath, peek through your fingers, and scroll to the next A/B testing tool on the list… really fast. All will be ok. We promise.

ab testing tools comparison 2

What features should every A/B testing tool have?

  1. Baked in A/B testing algorithm
  2. Rapid set up
  3. Easy upload of HTML/CSS/ JavaScript
  4. Segmentation
  5. Third party integration
  6. Cross device testing
  7. Analytics and reporting
  8. WYSIWYG editor

The below tools have all of the above.

Our 12 favorite A/B testing tools

  1. Unbounce
  2. PageWiz
  3. VWO
  4. Google Analytics Experiments
  5. Convert
  6. Truconversion
  7. A/Bingo
  8. Five Second Test
  9. Optimizely
  10. HotJar
  11. AB Testy
  12. AdNgin

1. Unbounce

For starters, what a great name. Unbounce is a complete landing page creation, management, and optimization platform. It provides us a variety of professionally designed templates to create a landing page. They are responsive designs that work for mobile too. And the A/B testing functionality is built in.

So that means you’ll be creating testable variations and then seeing which ones bring the most conversions. They are more geared towards landing pages. If you’re want to test different landing page variations, are building an email list or selling a product or service, Unbounce is a great option.

Pricing: Starts at $49 per month.

Best for: Campaign managers, online marketers, entrepreneurs, bloggers.

2. PageWiz

pagewiz list of ab testing tools compared

If you like Unbounce you’ll love PageWiz. PageWiz is not as well known but it’s an up and comer that’s worth checking out. Their templates are some of the nicest and they also offer a built-in and automated A/B testing solution.

Pagewiz integrates with third parties like SalesForce and Mailchimp and also provides real-time analytics so you can put those leads to quick use. If you run a marketing agency you can also use their interface to manage various clients through one account.

Pricing: Starts at $29 per month

Best for: Campaign managers, online marketers, entrepreneurs, bloggers.

3. Visual Website Optimizer by Wingify

VWO calls itself the “A/B Testing Software for Marketers”. Of course, if you’re using any A/B testing software, whether you like it or not, you’re acting like a marketer. Don’t worry. That’s a good thing. Marketers get results.

Wingify’s team built a full gamut product with features that go far beyond the basics of A/B testing. If you’re new to conversion optimization and testing, start out with the basic A/B testing components first. You can take advantage of the rest as you get more familiar with VWO.

Unlike Unbounce, there is more versatility with VWO. You’re not just creating custom landing pages. You can actually adapt new versions of your website and test the variations to see what works best. There are also additional features like heatmaps, targeting options, and third party integrations available.

Pricing: Starts at $49 per month.

Best for: CROs (conversion rate optimizers), webmasters, content marketers, user experience experts.

4. Google Analytics Experiments

There are some undeniable advantages of being a $400,000,000,000 company like Google. One of them is the complete freedom to hire as many geniuses as needed. And it results in Google software at or near the top of lists like these. Maybe a little bit like the USA and the olympic medal count, or Canadians who dominate slapshots and beaver tattoos.

The caveat here with Google Analytics Experiments is that it’s a part of the overall Analytics package. It’s not the core focus of Analytics. Likewise, Google’s known to be hit and miss with their user interface. It’s probably slightly superior to the USS Enterprise knobby controls that Captain Kirk used back in 1969, but only slightly. If you use Google Analytics and are still hesitant to get started with the other options, give Experiments a try.

The setup isn’t as complex as you might expect. But it’ll probably just whet your appetite for a better A/B testing tool set, like the ones showcased here. If you’re comfortable with Google Analytics already, we think Experiments is a good place to get started.

Pricing: Comes with Google Analytics so if you don’t pay for GA you won’t pay for GA experiments.

Best for: A/B testing noobs.

5. Convert

Another great name. No beating around the bush here. There’s no mistaking the objective with Convert. However, they are targeted at the Enterprise market and won’t be suitable for some lower trafficked blogs. Convert boasts flicker free tests. This is actually an important feature.

Flickering sometimes happens during A/B testing. When the test modifies your page content in real time, it could show flickers or blips on the screen as the test renders. That’s not a deal breaker, but it isn’t a good thing either. Technically, it could introduce a contaminant into an otherwise controlled test. That’s why the Convert team works so hard to eliminate it.

There are also some good code integrations. However, we think this one is best for more advanced teams. A higher learning curve is expected to take full advantage of all that it has to offer.

Pricing: Starts at $125 per month

Best for: Large eCommerce

6. TruConversion

truconversion ab testing tools list

Opportunity! We like to keep a watchful eye out for new products. TruConversion is one of them. It’s (very) similar to one of our other products on the list (HotJar) and bids itself as an “all-in-one analytics and feedback application”. The opportunity here is if you can get in on their beta you’ll receive the full enterprise package free for a whole year. Not bad.

Their marketing doesn’t suck either. Even though they’re only in beta, they’ve managed to gain a following of 20K email subscribers and publish a lot on their blog.

Though their strength is in understanding how your users interact with you site (including conversions), they combine many of the best features into a single streamlined package. And then tie directly into some other major applications and platforms. They include heat maps, recorded visitor sessions, conversion funnels, micro surveys, and form analytics. All of which, you can A/B test through the system.

Looks like there’s some serious backing behind this tool so we predict it will gain popularity quickly once out of beta.

Pricing: Free while in beta

Best for: CROs, Online marketers.

7. A/Bingo

a/b bingo ab testing tools

If you’re not a coder, you might want to skip over A/Bingo. However, it’s growing in popularity as its a plugin written for the highly popular Ruby on Rails development framework. If your site is built on top of Rails, there may be some potential here for you.

One of the key advantages of integrating A/B testing directly within code is speed. The plugin is very cool and open source. So we’ll forgive them for the 15 year old webpage design. It gets the job done and shares a cool tool – and that’s what matters. None of your designer snobbery! If you’re interested in A/Bingo, you should also compare with Vanity.

FYI, the creator of A/Bingo recently sold it so I’d grab the code quickly before it becomes unavailable.

Pricing: Free

Best for: Coders who like no frills software.

8. Five Second Test

Five seconds may seem like a short time, but in fact it is more than enough time for a website visitor to determine if there is enough quality in your website to stay, or to leave potentially never to return.

It’s a novel way to collect user feedback. You prepare a mockup of your site or page, ask some questions, and then get feedback from real users. They have 5-seconds to see it. Then they are presented with questions. And Five Second Test probes them for answers to understand what they recall and what left the biggest impression.

What can I learn in 5 seconds?

  • Who you are
  • What product or service your provide
  • Why should I care

So a 5 second test can be a great starting point to test your entire conversion funnel. First, do I pass the 5 second test. Second, everything else.

Pricing: Basic is free and a pro package costs $99 per month.

Best for: Technically anyone but online marketers will probably gain the most from it.

9. Optimizely

 

Just in case we startled you above by talking about code, we’ll bring you back into the A/B testing comfort zone. Optimizely is probably the best known A/B testing solution of the bunch. Best of all, for most users, it’s free for life with a pretty decent set of features that will do nicely for most A/B tests.

Optimizely has been, and still is, a great starting point for most A/B testers while also providing the necessary features for more advanced users.

Pricing: Free with most features. More robust packages are most likely priced according to page views.

Best for: CROs, online marketers, UX experts, webmasters.

10. HotJar

hotjar list of ab testing tools comparison

Not branded as an A/B testing tool but the functionality is there among some other really great features like heatmaps, visitor recording, conversion funnels, and more.

One thing you need to keep in mind with HotJar is that the data you will be provided is sampled (just like Google Analytics started doing recently). This allows HotJar to offer very competitive pricing but you don’t get all of your data and need to rely on HotJar’s algorithms to provide you with accurate sampling.

By the way, if you’re running an agency, HotJar offer whitelabeling, AKA custom branding, options for your reporting.

Pricing: Basic package is free for life. Paid packages start at $29 per month for users with over 10,000 daily page views.

Best for: CROs, Webmasters, Online Marketers, UX experts.

11. AB Tasty

A good user experience is one born through continuous testing and AB Tasty try to provide their users with all the tools necessary not just to convert but also to create a great user experience that really connects with end users.

So obviously AB Tatsy is more than your nuts and bolts A/B testing, although there’s that too. You can create communication messages through modules that pop up or slide in in real time to segmented audiences. As a marketer, you gotta love the potential.

They also offer a DMP (data management platform) integration to really put your data to good use.

Pricing: Starts at $34 per month.

Best for: Online marketing and UX professionals.

12. AdNgin (shameless plug)

adngin ab testing tool list comparison

If you’re a digital publisher, we’ve saved the best for last. We’re going to have to suspend a little modesty just this one time. While the other A/B testing tools we’ve showcased are more focused on content optimization, AdNgin is unique. We’re fully focused on maximizing your ad earnings through continuous testing.

That’s right, AdNgin does not modify content. Instead it allows you to experiment with Google AdSense layout like placement, size, device, and much more. It isn’t just about changing the layout of your ads though. We also help you test across different ad networks. We want to find the absolute best combination of ad layouts, sizes, types and networks to increase your earnings.

Pricing: Free for life up to 10,000 daily page views. Paid packages start at $250 per month.

Best for: Individuals and companies who use Google AdSense.

Always be testing

Between you, me, and the internet. We don;t really care which A/B testing tool you choose from above (as long as you use AdNgin:)). The most important thing to remember is that the best time to start A/B testing was yesterday. Don’t drown yourself in endless research on finding the perfect tool. Start testing now cause a day without testing is a day you’re losing money.

Did we leave out your favorite tool? Tell us how many of these A/B testing tools you’ve tried. How many are on your list of todos.

I’m Head of Marketing Operations at AdNgin. Before coming to AdNgin, I was a marketing professional focused on SAAS business models. When I’m not working, which is rare, I sail and hang out with my son, Jonathan, and wife, Meital.

Start optimizing Page Views
  • Great post Eyal. Thanks for mentioning Pagewiz in your list.

  • Robin Rue

    Unbounce sounds like a great site. Heading over to check it out now 🙂

  • Mardene Carr

    Wow, this is a whole lot to get through. I will have to save it and go back through when I have more time

  • Pingback: My “Must Try” List of A/B Testing T...()

  • Unbounce sounds great I am checking them out as we speak.

  • Jennifer Rollins

    Great post! I will be checking out all that you have mentioned.

  • Amber Ily Lampi

    Wonderful post thank you for the information. Unbounce sounds great and I’ll for sure have to check them out. Thank you!

  • Maddie

    I love this post. I’ll have to check some of these out.

  • Lots of valuable information and some great sites to check out!

  • That’s very detailed and informative. I don’t think I’ve been using any of these or even doing a/b testing. I should definitely get ahead and start with it.

    • You’ll love the results.

  • Lexie Lane

    Unbounce sounds interesting. This blog is very interesting and full of information. Thanks for this.

  • Tua Mukherjee Chatterjee

    AB Tasty looks interesting. Will go through the entire thing definitely.

  • Claudette

    You are most certainly taking us to the next level with this one! Thanks so much! 🙂

  • Michelle Mink

    i have never heard of a/b testing tools before but since i have google analytics i will be checking into it. thanks

  • I have no clue what this is. I have to go google it. Perhaps you could have provided even a short definition for first time visitors to your blog.

  • I have never heard of this and how to use a/b testing tools. I have to look more into this.

  • Liz Mays

    This is an excellent resource list. I need to really sit down and digest all of this.

  • The Google Analytics and UnBounce ones seem like they would help my website more 🙂

  • This is all new information to me! I will definitely be looking more into this!

  • Pingback: 10 Tips to Improve Your Stuck AdSense Revenues | AdNgin()