LeadBoxes Split Test: How James Schramko Boosted His Opt-ins by 50%

Today’s split test features a pair of LeadBoxes belonging to James Schramko.

If you don’t know James, he’s an online business and web traffic consultant and the founder of SuperFast Business.

Recently, James started linking his website banners to LeadBoxes instead of traditional landing pages on his site. James then ran the split test below to find the exact copy and colors that would boost his opt-ins even further for his LeadBoxes.

The majority of traffic for James’s split test came from his website, SuperFastBusiness.com, where 65% of visitors are repeat visitors.

The split test consists of two dramatically different LeadBox variations, with changes in both color and copy. Here, the winning variation boosted opt-ins by a relative 50%.

Before we delve into the numbers, take a look at the LeadBox variations below and see which was the winning variation.

Original/Control: This pop-up LeadBox features longer headline, a privacy policy and a yellow button that reads, “Send My Training.”
Original/Control: This LeadBox features a longer headline, a privacy policy and a yellow button that reads, “Send My Training.”
Winner: +50.5% Improvement: This LeadBox features a shorter, bolder headline, no privacy policy and a purple button which reads “Send My FREE Training.”
Winner: +50.5% Improvement: This LeadBox features a shorter, bolder headline, no privacy policy and a purple button which reads “Send My FREE Training.”

Purple Takes The Day

By a margin of just over 50%, the second variation is the winner.

The original yellow-button variation converted 32% of visitors, while the purple-button LeadBox converted nearly half of its 124 visitors for a conversion rate of 48%.

After running the stats, the second variation’s probability of outperforming the original variation is 96.54%.

Keep in mind that anything over 90% is considered to be statistically significant when split testing. In other words, you can declare a winner for your split test after the probability hits 90% or more.

With these results, James will expect to see an average of 16 more registrants for every 100 visitors by switching the design of his LeadBox.

Below are the full results of the analysis.

Here are the exact results James saw when he ran this split test within LeadPages. From left to right: Conversions are divided by Visitors to create a Conversion Rate. These numbers are compared in the Percentage Change and analyzed for statistical significance in the Probability of Outperforming Original metric.
From left to right: Conversions are divided by Visitors to create a Conversion Rate. These numbers are compared in the Percentage Change and analyzed for statistical significance in the Probability of Outperforming Original metric.

How James Interpreted the Results

When I spoke to James, he wasn’t surprised by these tests results. However, the magnitude of the difference between the two LeadBoxes was definitely unexpected.

Here’s why James was expecting the first LeadBox to lose:

“I was concerned that the CTA button was looking washed out/blended too much with the site. When I created the new treatment I also considered that many people will not be attracted to building an asset because they don’t yet know what that means.

“On top of that I was curious if a privacy policy would help or hurt conversions since it is probably blind to most people in my market.”

Why do you think the second version so significantly outperformed the previous version? Share your thoughts in the comments below and let’s get a discussion going.

Split Testing Your Own LeadBoxes

As I mentioned before, James didn’t realize how significant his changes impacted conversions until he actually sat down and ran the split test.

That’s the power of split testing — it can reveal things you had no idea were factors until you crunch the numbers.

LeadPages offers dozens of free templates that make it easy for you to customize and deploy your own split test in minutes. Yes, minutes.

Here James tested for how a change in copy and color affected his results, but almost any design change is in play. When you’re using LeadBoxes, you can easily test adding an image, different headline, button color, or even the privacy policy as James did here.

If this if your first time running a LeadPages split test, check out this awesome three minute tutorial to get started.

All Pro users of LeadPages have access to all of our in-house split testing tools, but any account has the option of using another split testing tool within our system. Check out the video below for more about how to split test using LeadBoxes.

Thank you again for James and his team at superfastbusiness.com for sharing this split test.

Why do you think the second version so significantly outperformed the previous version? Have you run a split test with similarly surprising results? Let me know in the comments below.