Testing 101: A great testing strategy starts with analysis

Site testing is one of the pillars of growing revenue through incremental improvement. If one of your goals for 2017 is to start a testing program for your organization it could seem like a daunting task.

Testing is a very broad topic, one for which we could write multiple blog posts on, and include information on topics such as targeting, segmentation, technology, testing parameters and program structure. So, for the purposes of this blog post, I’d like to share with you just the basics- i.e. “Are we ready to start testing, and if not, how do we get ready?”.

Reporting is not Analysis

Take a  critical look at your monthly reports. You should ask yourself, “Is this reporting, or is this analysis?”. You can answer this question by taking note of whether your organization is spending countless hours generating manual reports from the data versus spending time interpreting that data to pull out useful findings and recommendations. This will help you to understand if your testing is going to be meaningful, or if it is just testing for the sake of it.

Shifting to Analysis

If you are not getting meaningful insights from your data, then you’ll need to take a step back and find out why. Helpful questions to ask in order to determine this include:

1. “Why are we up/down month over month for registrations/transactions/engagement?”
2. “What is contributing to this and how do we influence it?”
3. “What has changed and why did it make an impact?”

If this sounds like your current process, then you are well on your way. However, if you’re still stuck on habitual reporting, we can help. This is what we do for clients everyday, it’s kind of our thing. Again, this is a broad area for discussion, but more simply if you set out your analysis in a way that requires you to think about what to do about the data you are seeing, then this is a good start.

1. Observation – what did we see in the data? e.g: A trend, a dip, or an anomaly.
2. Implications – what does that mean? e.g: An affinity, a tracking issue, or a drop in performance.
3. Actions – what should we do to address? e.g: Test audiences, test content, investigate tagging.

Pull test ideas from your analysis

A testing idea which stems from understanding campaign performance has a much larger business impact than testing something arbitrary like the color of buttons.

With this approach you can get further with each analysis that you do, and you’ll generate powerful, data driven recommendations that will help you reach your goals for any given program or campaign.