Split Test Results: How Timing (Not Layout) Boosted Response on this Page by 87%

By Kat Von Rohr, Writer and Editor for LeadPages

I have a quick but important split test to share with you today.

If you’re currently running videos on any of your lead generation or sales pages, you’ll want to pay attention to this one.

In December, LeadPages user, James Grandstaff ran a short-lived split test to see if he could increase the leads he generated on this Video Squeeze Page below.

James began by testing when to ask his viewers to join his list.

On one variation, his lead gen box appears as soon as you open the page, so potential subscribers can join his list instantly.

On the second variation, the exact same lead gen box appears two minutes into the video, so potential subscribers can view his content before he asks them to opt-in.

Testing the timing of this lead gen box ended up boosting his response by 87%. But which variation won out here — the instant lead gen box or the delayed one?

Take a guess and then scroll down for the answer.

The original: As you can see, this Video Squeeze Page opens without a lead gen box. The lead gen box doesn’t appear until more than two minutes into the video.
The original: As you can see, this Video Squeeze Page opens without a lead gen box. The lead gen box doesn’t appear until more than two minutes into the video.
Variation 1: This variation is the same in every way -- except the lead gen box appears immediately when you visit this Video Squeeze Page.
Variation 1: This variation is the same in every way. The only difference is the exact same lead gen box appears immediately when you visit this Video Squeeze Page.

And the Winner Is…

Variation 1: The instant lead gen box.

Immediately introducing this lead gen box increased opt-ins by 87%.

This page also won out with a 100% probability that this result was not due to chance. (Keep in mind: Anything over 90% – 95% is statistically significant.)

James originally set up this Video Squeeze Page as part of two-video series to promote his Facebook marketing course in early December. This short-lived split test only lasted two weeks to promote this limited time offer.

Throughout those two weeks, James only drove cold traffic to this split tested page. That means potential subscribers who were not familiar with James’ work were still more likely to opt-in when James gave them the chance to opt-in immediately.

Even with the cold traffic, James hypothesized that asking for the opt-in instantly would increase his conversions on this page. In the end, he was proven right.

Further Proof that this Instant Lead Gen Box Won Out

Like many LeadPages users, James didn’t stop with just two variations. As you’ll see in the numbers, James also tested a third variation of this Video Squeeze Page.

What I found interesting: The third variation of this video squeeze page also had a “no fade-in” immediate lead gen box. The only difference on this third variation is James also tested removing the video’s intro.

But in both cases, the instant lead gen box increased response.

Both “no fade-in” variations introduced the lead gen box immediately when a new visitor came to this page -- and both increased response with statistically valid results.
Both “no fade-in” variations introduced the lead gen box immediately when a new visitor came to this page — and both increased response with statistically valid results.

A big thanks to James Grandstaff for sharing his results here. If you’re curious about the video on this page, you can see the full video here.

Also, to see what James is working on now, check out his blog.

How to Run a Similar Split Test on Your Own

Once again, James used the Video Squeeze Page to run this split test.

You can see how your own audience responds to an instant lead gen box like this, by running a similar split test on your own pages.

Here’s how you would set up this split test…

  • Step #2: On this first variation, click on “fade-in box” and set the timer for when you would like the lead gen box to appear during your video. (For example, you might set your lead gen box to appear 30 seconds into your video.)
  • Step #3: Once you have this page exactly as you want it, enable split testing on the page by clicking on “A/B test.”
  • Step #4: Add a variation to this page by clicking “add variation” from the drop-down menu under “A/B test.”
  • Step #5: Choose to “copy and modify” your original variation. This will give you the exact same Video Squeeze Page to modify for your second variation.
  • Step #6: On this second variation, click on “fade-in box” and change your current time to zero minutes. This will make your lead gen box appear instantly when potential subscribers visit this page.
  • Step #7: Save it and you’re done. Ready to split test.

If you’ve never run a split test like this before inside LeadPages, I recommend you check out our video on split testing first.

Also, if you’re not currently set up to split test, you can find out more about how to enable split testing in our video below.

That’s it for today. Before you go, what did you think of this split test? Have you ever ran similar tests on your own pages? If so, did you have similar results? Let me know in the comments.

  • Be the first to comment

    • Kat Von Rohr

      Haha – thanks for the encouragement to comment here. Appreciate it. 🙂

  • Harris Fellman

    I’d be curious to know which group was more responsive actually. What happened ‘after the opt-in’? Did he get a higher sales conversion rate? Higher or lower refund rate? etc.

    • James Grandstaff

      Hey Harris, this is James. I found the “quality” of leads were better when the box faded in at the Call To Action time. Lower conversions but higher quality. I don’t have hard numbers to share for this test. But I did spent about $90,000 on Facebook Ads when the site first launched and more than doubled my ROI. The opt-in form was set to fade-in the entire time and confirmation was required. Hope this helps.

      • Kat Von Rohr

        Nice! Thanks for the additional info here James. 🙂

  • Debbie Kennedy-Crook

    Is there any chance one day that someone on here can tell us which variation led to the most subscribers actually staying on their list.

    I just don’t get why so many people rave about their high conversion rates when surely, in the long term it’s about quality, relevancy and
    above all , trust.

    • Todd Barth

      That’s 2 different animals. People stay on your list if you give good value and they like you. 1st you must get them on your list, then you can share your knowledge. Take care

      • Kat Von Rohr

        Thanks for the feedback Debbie. And thanks for weighing in on this Todd. Generally when you run a split test, you’re only looking at this split testing data for a limited timeframe — from when you begin the split test to when you declare a statistically valid winner. After that, marketers generally don’t track how specific variations translate to time spent on their list, which is why we don’t report on this data.

        Also, strictly speaking, the goal of any split test is to increase conversions — not necessarily to increase the long-term value of those particular subscribers. (Although if you can both inspire loyalty and get high conversions with one lead generation page, that’s fantastic.) As it is, most marketers only use lead generation pages like this to collect leads, and then use their follow-up sequences and other high-quality content to inspire long-term loyalty and trust, as Todd mentioned. Again thanks for the comment – truly appreciate it.

        • Debbie Kennedy-Crook

          Hi Kat. Thanks for breaking it all down.I do feel that creating some trust with your landing page helps in the long run. It’s often the first point of contact after all. That’s just my view and I realise different things work with different types of audience. I appreciate your response

      • Debbie Kennedy-Crook

        Of course that’s true Todd. My comment was a bit vague and wasn’t directed at James’ page. I’ve watched the video on James’ page and it is really good. He gives alot of information and creates immediate trust. If I wanted this information I would have no hesitation handing my email address over.

        But there are alot of people that, unlike James’, achieve high optin rates from squeeze pages based solely on the curiosity factor.

        Then people subscribe to satisfy their curiosity. Then they realise they’ve got a ton of other marketers hitting their inbox everyday with the same stuff and they unsubscribe or just don’t bother opening them anymore.

        There have been a few case studies on here based on pages like that.

        So that’s what I meant really. It would be nice to see a case study on here that showed some the longger term results. Not just the optin conversion rates.

        • James Grandstaff

          Hi Debbie, see my comment to Harris below. 🙂

  • In the test I ran with order buttons the instant order button always beat the delay. I know the popular ‘gurus’ promote delays (and no player controls etc..) almost all of the time my results are the opposite. It may be down to warm traffic with trust or some other factor. I think the best answer is to question it with a test of your own funnel. Great post

    • Kat Von Rohr

      Nice! Thanks for sharing your results James. Truly appreciate it. 🙂