LeadBoxes Split Test: How James Schramko Boosted His Opt-ins by 50%

Split-test-thumbnail_5-19

Today’s split test features a pair of LeadBoxes belonging to James Schramko.

If you don’t know James, he’s an online business and web traffic consultant and the founder of SuperFast Business.

Recently, James started linking his website banners to LeadBoxes instead of traditional landing pages on his site. James then ran the split test below to find the exact copy and colors that would boost his opt-ins even further for his LeadBoxes.

The majority of traffic for James’s split test came from his website, SuperFastBusiness.com, where 65% of visitors are repeat visitors.

The split test consists of two dramatically different LeadBox variations, with changes in both color and copy. Here, the winning variation boosted opt-ins by a relative 50%.

Before we delve into the numbers, take a look at the LeadBox variations below and see which was the winning variation.

Original/Control: This pop-up LeadBox features longer headline, a privacy policy and a yellow button that reads, “Send My Training.”
Original/Control: This LeadBox features a longer headline, a privacy policy and a yellow button that reads, “Send My Training.”
Winner: +50.5% Improvement: This LeadBox features a shorter, bolder headline, no privacy policy and a purple button which reads “Send My FREE Training.”
Winner: +50.5% Improvement: This LeadBox features a shorter, bolder headline, no privacy policy and a purple button which reads “Send My FREE Training.”

Purple Takes The Day

By a margin of just over 50%, the second variation is the winner.

The original yellow-button variation converted 32% of visitors, while the purple-button LeadBox converted nearly half of its 124 visitors for a conversion rate of 48%.

After running the stats, the second variation’s probability of outperforming the original variation is 96.54%.

Keep in mind that anything over 90% is considered to be statistically significant when split testing. In other words, you can declare a winner for your split test after the probability hits 90% or more.

With these results, James will expect to see an average of 16 more registrants for every 100 visitors by switching the design of his LeadBox.

Below are the full results of the analysis.

Here are the exact results James saw when he ran this split test within LeadPages. From left to right: Conversions are divided by Visitors to create a Conversion Rate. These numbers are compared in the Percentage Change and analyzed for statistical significance in the Probability of Outperforming Original metric.
From left to right: Conversions are divided by Visitors to create a Conversion Rate. These numbers are compared in the Percentage Change and analyzed for statistical significance in the Probability of Outperforming Original metric.

How James Interpreted the Results

When I spoke to James, he wasn’t surprised by these tests results. However, the magnitude of the difference between the two LeadBoxes was definitely unexpected.

Here’s why James was expecting the first LeadBox to lose:

“I was concerned that the CTA button was looking washed out/blended too much with the site. When I created the new treatment I also considered that many people will not be attracted to building an asset because they don’t yet know what that means.

“On top of that I was curious if a privacy policy would help or hurt conversions since it is probably blind to most people in my market.”

Why do you think the second version so significantly outperformed the previous version? Share your thoughts in the comments below and let’s get a discussion going.

Split Testing Your Own LeadBoxes

As I mentioned before, James didn’t realize how significant his changes impacted conversions until he actually sat down and ran the split test.

That’s the power of split testing — it can reveal things you had no idea were factors until you crunch the numbers.

LeadPages offers dozens of free templates that make it easy for you to customize and deploy your own split test in minutes. Yes, minutes.

Here James tested for how a change in copy and color affected his results, but almost any design change is in play. When you’re using LeadBoxes, you can easily test adding an image, different headline, button color, or even the privacy policy as James did here.

If this if your first time running a LeadPages split test, check out this awesome three minute tutorial to get started.

All Pro users of LeadPages have access to all of our in-house split testing tools, but any account has the option of using another split testing tool within our system. Check out the video below for more about how to split test using LeadBoxes.

Thank you again for James and his team at superfastbusiness.com for sharing this split test.

Why do you think the second version so significantly outperformed the previous version? Have you run a split test with similarly surprising results? Let me know in the comments below.

  • Jason Ephraim

    If that is indeed an email field, the placeholder text shouldn’t say “enter your email below”.

    • Tom Johnson

      You’ve got a sharp eye, Jason. Do you think the e-mail field text impacted the overall results?

      • Jason Ephraim

        Thanks. Yes, I think the bounce rate wold decrease dramatically (depending on how the validation is set up to handle errors.). Change “below” to “here” is a good start, and I’d set the next test to play with the placeholder to test messaging like “What email can we send your free course to?”, and so on.

        • thanks for the find. I am testing ‘below’ versus removing that word to see if it makes any difference. (I know it is incorrect however I am curious)

          • Jason Ephraim

            No worries. Great idea! Afterall, always be testing. I’d also suggest testing some more “exciting” headlines: “Boost Your Traffic & Sales Overnight”, “Mind-Blowing Leads & Sales”, “Unbelivable”, “Sky-Rocket”, “Capture”…Anything to pique the visitor’s interest. I also think you might see an increase with a small explanation of the actual “offer” (course). In my mind, it goes from the “Get this” to “by giving us this”, without the “with this”. Hope this helps you think up some future tests. Best wishes for a continued success!

          • Thanks Jason I appreciate you giving me all these ideas. I did used to have long sales type opt-ins (One was written by John Carlton) so it is a good reminder to try the old ones with the new technology.

          • Jason Ephraim

            To address your point, did you try any imagery along with your message? You say you are giving something which (along with the idea that they will receive it via email) alludes to a product. Even among Leadpages’ many blog posts, their examples along these lines usually includes something like a rendering of a physical book or pamphlet. “Get This Free” might work better if you include a visual representation of your course (even if it’s just a book-jacket rendering).

          • that is what the Leadpages tests were all about. Video versus image versus no image. For my audience (mostly repeat visitors) these treatments do not increase conversions. They seem to like punchy with the leadboxes

          • Jason Ephraim

            Wow, interesting results!

          • Jason Ephraim

            I also checked out your website. I really liked the direction the “products” section of your site takes to try and target visitors based on their needs. Based on my experience, this technique can really improve conversion rates. I’d suggest incorporating some symbols preceding the text on each button’s label along with unique colors for each one. You want these choices to be no-brainers that can be easily navigated. Therefore, it makes sense you apply the same techniques that work on callouts like lead pages to these choices.

          • Work in progress – we just updated our web services page to a new format today and the rest will fall into line. We smushed together all of our sites a few months ago into this supersite so it is working well.

  • Angeline Plesek

    I love these types of tests…you never know what’s going to happen.

    I think the purple button won out just because of the copy–adding the word “FREE.” I agree that the purple button does look a little washed out.

    It would be interesting to see how the yellow button would perform with the copy from the purple button OR if the yellow button had a black font with the same copy as the purple button.

    My guess…yellow would win out. BUT here again, you just never know until you test, test, test. 🙂
    Thanks for this great post!

    • Tom Johnson

      Calling out that something is “FREE” does tend to help!

      You’re definitely right that testing each variation again and again is the only way to be sure what’s significant. I’ll keep my eyes open for others test that swaps the colors and copy as you suggest — those would be some very interesting results to consider.

  • Gerrie

    I couldn’t access the Rapid List Building Worksheet through the link, a gift for tonight’s webinar with Justin L. How else can I access / download it? Thank you.

    • Kat Von Rohr

      No problem. Your worksheet is available on the bottom of the thank you page that you were directed to when you signed up for the Rapid List-Building webinar. If you scroll down on this page you’ll see it on the bottom: http://lp.leadpages.net/thanks. Hope that helps.

  • Samantha

    Agree with Angeline completely. We love tests like this because they are very unpredictable and keep you guessing.
    Nice article and great examples/content! Plus– Love how you added the word FREE. It’s crazy how many people click and choose things based on FREE these days!! convertify.io/blog/