Split Test Results: This Simple Page Boosted Response By Over 900%

blog post thumb - 04.01

By Kat Von Rohr, Writer for LeadPages

Wow. You don’t often see split test results like this.

Last Veterans Day, LeadPages user, Jess Wilkinson — aka “Prospector Jess” from Hunting4Gold — ran a quick test to see which LeadPages template would boost conversions on this limited time offer.

One of the LeadPages templates you see below boosted response by over 900%.

Take a look and see if you can pick out the winner below…

Veterans 2 with border resized
Choice A: This first page has your standard, out-of-the-box background from a Super Basic Squeeze Page inside LeadPages.
Choice B: As you can see, this page features slightly tweaked copy and the standardized background from the Basic Centered Squeeze Page inside LeadPages.

And the Winner Is…

Choice B.

This second variation increased response for this Veterans Day offer by 952% according to the data from inside LeadPages below.

Also, Choice B dominated in this split test with a 100% probability. As a rule, we only declare a split test like this one statistically valid if it has a 95% probability or greater that this result was NOT due to chance.

So in this case, our LeadPages split testing tool calculated this particular result was NOT a fluke. In other words, it’s statistically valid.

A close-up look at the data from inside LeadPages.
A close-up look at the data from inside LeadPages.

The Genius Behind this Test (and the Entire Campaign)

Personally, I love this split test for several reasons.

First of all, the entrepreneur behind this test, Jess used the same out-the-box template backgrounds that anyone can use inside LeadPages.

We rigorously test all the designs in LeadPages, so we can give all our users the highest possible conversions. That means both the white background in Choice A and the blue sky background in Choice B are proven designs.

So testing both these proven pages to your audience is like challenging two prize fighters to a grudge match — and seeing which one wins. (Not a bad strategy, right?)

Also, as you can see, this is not a simple background split test.

If you look closely, you’ll notice that the copy has been slightly tweaked on both pages. I like that, because the copy clearly matches the design for each page. But at the same time, both pages still convey the same information and speak to the same target audience — veterans.

(Of course, that begs the question: Was it the slightly tweaked copy or the completely different design that clinched the winner here? I’d love to hear your thoughts in the comments.)

The Holiday Factor

Finally, I like this test because it features an urgent limited-time offer that’s tied into a holiday. Most marketers would never consider using a largely unsung holiday like Veterans Day to build their list like this.

Also, if you opt-in for this page, you immediately receive an email that asks you to contact the GPAA by November 17th to claim your veteran-only discount. That’s a fantastic way to inspire urgency — and in this case, to honor veterans who have served their country.

Quick tip: As you’re planning out your marketing for 2014, you might consider using the lesser-known holidays in your country to run special offers like this one. Depending on your business, you might even try targeting specific groups of potential customers as they did here.

A debt of gratitude goes out to Jess Wilkinson and his team at Hunting4Gold and DoradoVista, Inc. for sharing their results with us. To learn more about what Jess and his team are up to, check out http://hunting4gold.com/.

The Fastest Ways to Run This Split Test in Your Own Business

Again, the two pages you see above are available inside LeadPages for you to use as is. So without changing backgrounds or making any significant changes, you could have a similar test up and running in minutes.

If you currently have a control page that you’re using to grow your list, I recommend you try testing it against a radically different opt-in page like our user did in this split test.

You never know – you just might find your next page that boosts your conversions by triple-digits like Jess did above.

If you’re already a pro customer, you can start testing in minutes with our built-in split testing tool. If you haven’t tried it yet, I recommend you start by watching our three-minute tutorial video on how to use this tool first.

Even if you’re not a pro customer, you can still split test your pages inside LeadPages using any other split testing tool available today. (Fair warning: Using other split-testing tools is not as easy as using ours — but they still work.)

That’s it for today’s split test. Before you go, why do you think choice B won this split test? Was it the design? Was it the slightly tweaked copy? Let me know in the comments below.

  • The differences I see between the two ads:

    1) First ad is stating you will be “claiming” the offer by entering your email. Second ad is indicating you’ll get more details by entering your email address. Difference in commitment.

    2) First ad has no deadline while second ad is providing a “buy now” with a specific “buy by” date. Big difference in urgency presented.

    3) First ad is very vanilla in terms of design. Second ad is easier on the eyes, more appealing aesthetically. That’s subjective, yes, but they’re VERY different layouts.

    4) First ad is also stating a newsletter comes with, while again 2nd ad is only saying more details comes next. Another difference in commitment.

    Those are large differences from a marketing/sales perspective and I would expect a significant difference in the split test. Nine times, though, is…well, ginormous!

    • Kat Von Rohr

      Agreed. Thanks for the in-depth analysis Stan. 🙂

    • Roman

      900% increase is 10 times increase, not 9.

      For example, a value increased from 2 to 6. That’s an increase of 3 times (6/2). However, the percentage increase is calculated this way: ((6-2)/2)*100. This gives you 200%.

      A simpler example that everybody would understand: a value went up from 1 to 2. That’s an increase of 2 time but only 100%.

      • eduleadership

        Depends on whether you’re talking a 900% _increase_, or 900% _of_ the baseline value. The former is `10x, the latter 9x.

        • 2 to 6:

          It increased *by* 2 times (200%) the original value (increased by = added)

          It’s an increase *of* 2 times (200%) the original value (increase of = added)

          It increased *to* 3 times (300%) the original value (increased to = multiplied)

          When Stan stays:

          “That’s an increase of 3 times (6/2)”


          “a value went up from 1 to 2. That’s an increase of 2 time but only 100%”

          If we really have some fun with semantics, perhaps the “of” in each of those statements should be changed to “to” and “by”, respectively, to make the statements correct?

          TBD. 🙂

          • Josh Wilkinson

            you guys crack me up 🙂

        • Roman

          True, but since this article says, “…boosted response by over 900%…”, it refers to the former (in your comment).

  • John Levengood

    “A” tells you the price up front. “B” tells you the discount and requires further action to find the actual price. As a consumer I would click away from the page if I knew the price up front and did not know any benefits yet.

    • Kat Von Rohr

      Good point! Thanks John.

  • Armen

    Copy ‘slightly tweaked’? Are you serious?

    Honestly, examples like this tell us very little. There’s such a vast difference that it becomes pure guess work in figuring out the reason for the dramatic difference in the results.

    I appreciate what you’re trying to do and I don’t want to sound too hard or ungrateful, but your articles would be immeasurably more helpful if you drilled down and gave hard data that would actually help people.

    • Kat Von Rohr

      Fair enough. I appreciate you sharing your opinion. 🙂 Yes, as a copywriter myself, we could probably debate what constitutes “slightly tweaked” vs. “entirely overhauled” for copy for days on end. 🙂 But instead, I’ll just say that I consider this slightly tweaked because it’s the same idea, with a subtle but important differences in the wording. (

    • Completely agree with you, Armen. And you should be harsh. Perhaps this was an experiment more than a split test. Because we both know that this ain’t the way to run A/B splits, and someone should point that out. I am glad you did.

    • I wouldn’t consider it only “slightly tweaked” either. Way too many factors changed to fit that description. However, I do think it’s a good test as long as there is a hypothesis behind each of the changes. Some follow up testing with the winning version set as the control, and manipulating individual factors from the original could help zero in on what it was that contributed to the lift most significantly. This is a time limited campaign though, so it’s hard to test factor by factor with a fixed volume of traffic.

  • EtienneJ

    Yeah, this test isn’t very helpful.

    And why don’t you post the raw metrics? To get a 900% increase, the control must have had terrible conversion.

    If control had 1% and new version has 9%, they’re both still bad.

    • Kat Von Rohr

      Fair enough. Generally, we only post the metrics that our customers have given us access to. But I appreciate your position and will certainly keep it in mind for future split testing posts. Thanks for the comment. 🙂

    • “Finding the control” – I like it. Great way to think of these initial multi-factor tests.

  • Uri Lederman

    I don’t really think that THESE commenters are getting the point here.. this is a GREAT example of FINDING the control…

    initial a/b testing should very well be two, three, even 4 different landing pages to find which resonate better with your audience…

    then tweak..

    again, great example of testing to find a control..

    although at 9%, i am assuming they WILL go back to the drawing board or find better targeted visitors.. 🙂

    • Kat Von Rohr

      Very true. Thanks Uri. 🙂

  • Maggie Percy

    Choice A may be ‘clean’, but it’s boring and has no emotional pull. Choice B has a picture that leads one to feel they are on the road somewhere, etc. Choice A has too many words bunched up. My observation with our list of over 6500 is that people do NOT want to have to read and process a lot these days. Choice B has simpler lines of easier to read text. Choice B also emphasizes the savings and the urgency with the end date. Neither really is hyping a transformation or benefit, but I assume the market he addresses already knows the benefit of subscribing.

    • Kat Von Rohr

      Agreed. Thanks for the insight Maggie. 🙂

  • Maggie Percy

    We did a Memorial Day sale to our list last year to raise money to go see Nigel’s 90 yr old Dad in the UK (we just didn’t have the extra cash). His Dad fought in WWII and was at Anzio, so we called it the War Hero Sale. We approached the list openly with our goal and gave them a great value for $47. We used the long form Lead Pages sales page. It was a big success. I agree that using lesser known holidays the right way for your market could be a real boon.

    • Kat Von Rohr

      Very cool! Thanks for sharing this Maggie. 🙂

  • kc

    Cmon…both squeeze pages have to say the same thing in order to make a valid comparison that will demo to potential buyer that the designs LP offers are great and high converting in comparison to the typical DYI designs…

  • Jim

    The better question is this 9-fold increase an increase of something like .01% to 1%? If not, how much traffic was sent to this page during the sample period? If we’re talking hundreds and not at least thousands, or we’re talking about a large gain over a tiny spread then this is all kind of irrelevant to anyone except the owner of the page. Looking at the two it is easy to see why the one did better, but not easy to see why it would have beaten a page getting reasonable results already by 9x, which makes me think the net is tiny but the % is a lot. It would be nice to know the conversion % on these.

    • Kat Von Rohr

      Thanks for the feedback Jim. Generally, we can only release the figures we get from our customers. But I’ll certainly keep this in mind for future posts. 🙂

  • ChinoBradley

    Changing the button copy from “Get Instant Access” to “Show me the Details” is a split test. This is a whole landing page rebuild. Way too many things changed to see what exactly triggered the massive conversion boost.

    • Kat Von Rohr

      True – that could be a whole separate test. Thanks for the feedback Chino. 🙂

  • martin

    Yes agree with both of you’s Armen and Michael. The text on both pages are not exactly the same so saying that choice B’s Template made the difference is totally incorrect. Love for some to do a proper A/B split test ones where text is exactly the same on both template’s and see which one wins out.

  • Josh Wilkinson

    This was a case where we had a new opt in page (choice A) that did very poorly (4.5% optin rate) The 2nd page (choice b) converted 26.7% opt in rate and was the page we stayed with for the remainder of the promotion. Also to be noted was this promotion was very experimental, we had not run this offer before and it was to a new traffic source.

    The experiment was to try out the super basic squeeze page template and test it compared to the basic centered page (which we have used elsewhere with success) with the goal of keeping the copy as close as possible on each page, which is impossible to do perfectly due to template area restrictions.

    The hypothesis was that the basic centered page would work better because of more color and was more engaging for this demographic, but the hope was that the super simple page (choice A) would work better because I find it easier to write copy in and its simpler and easier to deploy. As it turned out, the basic centered page (choice B) won out to my disappointment.

    I believe the copy probably had some effect on conversion rates but it is not so different that it would cause a 9x difference. We have found button text can equate to a couple % bump but never more than 10%. If you have, please post the button text you use :). As Uri said in the comments, we were finding the control.

    • Josh Wilkinson

      I forgot to mention, We do have a few pages that are at 70-80% conversion rates using the basic centered squeeze page with the default background. Its a pretty solid template.

  • Anthony Tran

    Wow that is some amazing results… I’ve never seen split test results that clearly dominated one over another. I agree the copy is cleaner on “B” and the fact that they didn’t talk about the price… just the savings of $40 felt less “Salesy”

  • Linda

    without even reading the copy, I would have clicked on the 2nd one just from a esthetic perspective. It feels more defined and concise. That may not be an analytical answer, but it is a first thought from seeing lots of opt ins. I’m a newbie myself so I’m seeing this as a ‘look and feel’ perspective.