How to A/B Test Donation Pages to Drive Recurring Gifts

5 min
William Schmidt
Will Schmidt

In order to grow any metric, you need to test, experiment, and determine what will impact donor behavior. The same principle applies to your recurring giving strategy.

When you test elements like your donation form and calls to action, you might find that little changes can make a big difference in the number of people who sign up to make a monthly gift. To that end, Classy rolled out an optimization that allows organizations to choose how the “donate monthly” option appears on their donation form.

We chatted with Terry Breschi, product designer, and Jackie Lam, senior product manager, to discuss how you can creatively test with this update to ultimately drive up recurring gifts.

But First, Why Do We Test? 

In any context, testing is important because it informs which new changes, processes, and ideas will be successful. In other words, you can’t make an informed and confident decision about something based purely on your assumptions.

For example, you might assume your audience wants to get weekly emails from you with multiple, quick stories about your impact. The reality could be that they want quarterly emails with one in-depth, long-form story about impact.

There’s really no way to know for sure without testing. And basing decisions like this off assumptions can end up hurting you more than helping you. When you know what works you can confidently plan a strategy that capitalizes on your learnings and data from the test.

Perhaps the most common form of testing is the A/B test, where you test one version of an email, webpage, or other item against another version, present them to users randomly, and see which performs better using statistical analysis.

For example, you could test different subject lines for an upcoming email campaign to see which one gets more opens. “A” would be the type of subject line you usually send, “B” the new version.

A/B Testing Your Recurring Giving

Before you start any test, you have to first establish your reason for doing the test in the first place. Are you trying to prove to your board that you need to shift your fundraising strategy away from one-time donations and rely more heavily on recurring gifts?

Defining the “why” is important because it will guide everything you do for the test. That, in turn, will help you sort through the piles of data you collect to determine what’s worth paying attention to and if your initial hypothesis is successful or not.

For the purposes of this article, we’re going to say the reason for testing is to find out if your audience engages more with the optimized recurring giving option or the classic option.

Create the Campaign

To start, create a fundraising campaign. Classy’s recurring giving optimization is compatible with all campaign types: peer-to-peer, crowdfunding, registration with fundraising, or donation pages.

recurring gifts
The different campaign types you can create with Classy.

After you fill out the details for the campaign—duration, amount you want to raise, images, headlines, and copy—you’re going to duplicate it to create a version “B.” Go back to the fundraising dashboard in Classy and duplicate the campaign:

recurring gifts
Duplicating a campaign.

Essentially, you’ll be running two parallel versions of this campaign. One will have the recurring giving optimization enabled, and the other will have it disabled. Make sure you clearly label which campaign is “A” and which is “B.”

recurring gifts
Name your campaigns clearly.
Pro Tip
Remember, when you create two campaigns like this, you’re going to also split your audience into two random groups. If you’re a smaller organization, pay attention to how many supporters you have in either group. Without a big enough audience, you might not get enough data to produce statistically significant results, especially for recurring giving.

Engage the Recurring Giving Optimization

After you create, build, and duplicate your campaigns, you have to change the recurring giving optimization for each one. On the “Donation Page” tab, under the “Donation Settings” section, you will see the note about optimization:

recurring gifts
The optimization notification.

When you choose to participate by clicking “Yes,” you allow the option to give once or monthly to show up above the suggest donation amounts:

recurring gifts
Monthly giving options with the optimization enabled.

When you click “No,” the option to give once or monthly shows up as radio buttons below the suggested giving amounts, next to the donation amount:

recurring gifts
Monthly giving options with the optimization disabled.

You can also choose to have the monthly recurring donation or one-time donation pre-selected when people land on your donation page.

Send the Campaigns

Once everything is all set, you’ll have two individual campaigns that are the same in every area except the recurring giving optimization. Now, you have to send them out to your audience.

While Classy doesn’t have an A/B functionality that automatically presents your two campaign versions randomly to users, you can get around this by manually creating two “random” groups from your audience that will see either version.

Combine all of your email lists and split them into two halves, group “A” and “B.” Then, take the link from campaign “A” and send it to email segment “A” and the link for campaign “B” and send it to email segment “B.” From there, your job is to monitor the different campaigns and see which one people engage with more.

Make sure that if you send any other emails to your segments about the campaign that you don’t mix up the links, and that you give each campaign the same level of promotion. If you send too many emails to group “B” and don’t match them to group “A” you risk throwing off the results of your entire test.

Calculate the Results

Once you’ve reached the end of the campaign, go back into the Classy dashboard and begin analyzing the data for each one. To read test results correctly, you need to keep in mind that clearly defined reason for doing the test in the first place.

Since we were looking to see which recurring giving option engaged the most people, we won’t need to pay attention to things like amount raised for each campaign. Instead, pay attention to how many supporters signed up for recurring gifts on each campaign.

The last step is to calculate if your results are statistically significant. This is the most important part of the test, because it will tell you with absolute certainty which campaign you should move forward with, or if you don’t need to make any changes.

Pro Tip
There are free tools to help calculate the statistical significance of your test, like this one provided by Kissmetrics. This is what we use to determine the success of some of our own A/B tests.

The framework for this A/B test doesn’t only have to apply to recurring giving. You can test many other things with your campaigns as well, like different headlines, copy, and images for the same fundraising initiative.

Have fun with it, but remember to follow the guidelines so you don’t accidentally disrupt your test and all your hard work. And if you haven’t set up a recurring giving program, use the guide below to find out how you can produce a sustainable source of monthly revenue for your organization.


Resources


Where social entrepreneurs go to learn and grow

Join over 20,000 leaders just like you who get their weekly dose of technology, innovation, fundraising ideas, and the latest industry trends.

Subscribe to the Classy Blog