Strategy Before Code: A Process for Valuable Analytics

 In this blog, originally published by Marketing Land, Dave Booth, the Digital Analytics Association’s Practitioner of the Year for 2014, makes some bold statements about the processes needed for a trustworthy and accurate analytics implementation. Read on to find out if you need to take your implementation back to the drawing board.

---

I’ll start with a bold statement: In 2015, you can no longer afford to live with a web analytics deployment that was slapped on by your website developers or IT department as an afterthought. I know, I know; I can already hear the protesting… “But I’m getting that hit report every month! Everything must be set up right!”

The truth is that these days, your data is an organizational asset – perhaps the most valuable one you have. And some big generators of your data in this day and age are very likely to be your online web properties, mobile apps, and digital advertising platforms.

Here’s another bold statement: If you’re not using web analytics data to drive decisions from the C-suite on down, it’s been deployed wrong, and it’s time to start over. Only this time, make sure to get it right by following a business objectives-driven approach to deploying your web analytics measurement strategy. This starts by understanding and formalizing exactly what those business objectives are.

Business Requirements Gathering

In well over a decade of working with web analytics tools, we’ve seen many, many web analytics deployments “done backwards,” meaning that the emphasis was placed on the Javascript first, and the end result is an awkward attempt to back a whole lot of meaningless numbers into something that’s actually important to the business.  Backwards.

A sound web analytics deployment will begin with the business objectives and design the technical solution accordingly. We’ve found the following process to work extremely well for enterprise-level deployments, and no matter who you are or what you do, this process can be adapted to fit most any organization out there.

1. Talk To Your Stakeholders. Scary, I know, but in the end, it’s the stakeholders that are going to use the data, so you better start with them. Ask them what their group’s objectives are and what targets they’ve signed up for to make sure they earn their bonus. I guarantee that you will not hear “If we could only get our bounce rate down this year, that would make all the difference!” What you will hear are the same themes that businesses have been working on for centuries: Lower costs. More revenue. Higher margins.  Better customer satisfaction. And unfortunately, none of those are canned reports in any web analytics tool I’ve ever seen.

2. Develop A Framework. This is where it starts to get fun. We’re going to have to translate those business objectives into the metrics, dimensions, and segments that make up the language of the web analytics tools; and to do this, frameworks can really be helpful. There are many that you can use, but one of my favorites goes back a few years and is explained really well by Avinash Kaushik. The framework provides a way to tie back all the data you can collect from the web (and other data sources) to Net Income, by way of measuring things that indicate success or failure in an organization’s ability to improve margins and market share. Now, that’s the vocabulary of the business stakeholders!

3. Create Your Key Performance Indicators (KPIs). With the business objectives and framework to guide you, the translation can begin. This is where you’ll need to bring in some experience to ensure that you know what is possible to measure with the tools that are out there and what’s not. By way of a quick example, let’s say that you uncovered that customer retention was a big business goal for your organization. While there’s no single metric that will tell you that you’re doing a good or bad job with this, you can absolutely find 5, 10, 15 or maybe even 20 KPIs that collectively will give you an indication of how well you’re doing here. If “repeat purchases,” “number of positive post-purchase social shares,” “average repeat purchase frequency,” “number of repeat visitors,” “weekly email blast click volume,” and a handful more are all going up and to the right month over month, you can clearly see that you’re doing a good job with your organizational customer retention goal. Your KPIs will be unique to you, but spend some time here to ensure that they’ll help you truly understand the business goals!

4. Map Your Data. Once you’ve got your KPIs in place, you need to figure out how to feed them. This means you’re going to have to break those KPIs into the metrics, dimensions, and segments that you’ll need to collect in order to properly and accurately calculate the numbers that will tell you if you’re doing well against your business objectives. It’s also important to understand that not all of this data will come from your web analytics tool! Map it anyway. Even if you don’t have a tool to measure it (yet), this is how you plan and budget for it next round. Even for the simple KPIs listed above, in addition to a web analytics tool, we’ll very likely need to be pulling data from a number of social platforms, email marketing platforms, back-end e-commerce platforms, and likely our customer relationship management (CRM) system.

Technology Selection

Now that you’ve got your data mappings and a clear way to tie it all back to the things your business actually cares about, it’s time to start thinking about what technology is required to drive this data collection and which items you’ll invest in both now and in the future.

• Web Analytics Tools. There are a number of web analytics tools out there that can be configured to collect the data you’ll need and integrate with other tools and platforms, and among the most popular are Google Analytics (and its Premium version), Adobe Analytics and the larger Marketing Cloud,  IBM’s Coremetrics and the larger Enterprise Marketing Management suite, and WebTrends. There are of course many more, and your data requirements and budgets will dictate the best choice for you.

• Tag Management System (TMS). At this point, you better have a really, really, really good reason for not using tag management. Otherwise, you’ll need to pick one of the many solutions out there. If you’re going with a Google or Adobe web analytics tool, they both offer free tag management solutions through Google Tag Manager (GTM) and Dynamic Tag Management (DTM), but if you need additional features, there are a number of great paid solutions out there including Ensighten, Tealium, Signal, and more.

• Specialty Tools. Will you need visual analytics solutions like heatmaps of where people’s mice are moving and what they’re clicking on?  Voice of customer (VOC) data? The ability to watch playback of specific sessions?  Specific data for web apps? Social media monitoring or social media analytics data?  You may end up requiring additional third party tools, and there are many.

• Integrations. You’ll very likely need to integrate your data collection with a number of tools and systems you already have. Think of your content management system(s) (CMS), customer relationship management (CRM), email marketing platform, marketing automation platform, advertising platforms (search, display, retargeting, programmatic, etc.), back-end financial systems, third party data and data management platforms (DMPs), and more. Knowing where all your data is coming from can help you develop the architecture required to collect and trust all of this data and make sure the right systems are talking to each other in the right ways.

Technical Solution Design

Whew. That’s a lot of work. But now, you’re ready to design your technical solution. Please note that we’re still not ready to start sticking Javascript anywhere yet, this is the design phase. Here’s where we’re going to look back at all of that data we need to drive our KPIs (that drive our business objectives), and filter that list to the things we’ll be collecting with the tool we’re designing the solution for.

For example, if we’re going to use Google Analytics as our web analytics tool, it’s time to document every account, web property and view we’ll need, and for every one of those, we’ll document every setting, filter, and configuration as well as every goal, what type of goal it is and what will trigger it, any funnels we want to see leading up to that goal, every single event and associated category, action, label, and value, every single custom dimension and metric along with the slots that will be used, each and every integration that will be configured, and about 50 more items that you’ll need to carefully map out and write down.

If you’re using Adobe Analytics, you’ll need to figure out every report suite that will be required, every traffic variable, success event, and conversion variable, each… wait, I’ve probably lost you. And that’s just fine.

The point is that this is where the technical folks will really earn their keep by thinking through and creating this living document that outlines every single aspect of your deployment. Many issues will be uncovered and solved here before a single line of code is written, and this document should include a formal change log so that at any given time, anyone can go to this documentation and figure out exactly what is being tracked, how it’s being tracked, and where to find the resulting data.

Installation

OK. For any developers that are still reading this far down, it’s (finally) your turn. Here’s where you’re going to get to put some JavaScript on some pages (or potentially add some SDK’s and update code for app deployments) and write some code in your staging or development environment; and these days, there are really just three high level steps to this process:

1. Strip Out Any Old Code. If you’re like most organizations, you’ve probably got a legacy of previous tool versions and scattered analytics code all over the place. It’s time to get organized, so the first thing you need to do is figure out where all the code is and get rid of it. There are some great tools out there that can help you do this, like the Web Analytics Solution Profiler (WASP), that even has a crawler that can create a nice little report of every page on your domain(s) and every single tag that’s on it.

2. Pop In The Tag Management Container. Tag management is a beautiful thing. All that nasty code you just stripped out is going to be replaced with two items, the first of which is the container code. This will allow the tag management system to inject whatever it needs into any page of the site (assuming that container code is there, so you better make sure you’re not missing any pages!).

3. Data Layer. This is the second part of tag management, and it’s essential for accurate data collection. Using the solution design as your guide, certain pages will need to expose certain data to the tag management system that can be used in the tracking tools that are deployed through the container. Take, for example, an e-commerce “thank you” page. Your analytics tool (and probably all of your digital advertising conversion pixels) will need to know what was purchased (categories, item names, SKUs, options, etc.), how many, how much, shipping cost details, and more. All of this is written to the data layer, and then it becomes available to be used in the tag management configurations.

Quality Assurance

Quality Assurance (QA) is a necessary step for any successful software project, no matter what your developers tell you (myself included once upon a time).

Formally creating and executing test cases to ensure that each and every component of your solution design has been implemented according to plan may take some extra time up front, but it will save you much more in the end. At this stage, the goal is to ensure that all the required code has indeed been put in place in the staging or development environment.

Tool Configuration

With the code in place, it’s time to go into the tag management and web analytics tools and begin the configurations, again, in the staging or development environments. The exact processes will depend upon the tools you’ve selected, but at a high level, the tasks involve:

• Account Setup & Administration. An administrator will set up the appropriate accounts and high level settings, assign specific levels of access to specific users, and things of that nature.

• Tag Management Configuration. Each tool varies, but generally you should include data definitions and validation, loading of tags into the system, configuration of tools to be deployed through the TMS, creation of the trigger events that will fire the various tags, and development of any additional code required to perform specific required logic.

• Web Analytics Tool Configuration. Once again, each tool is different, but typically administrators will need to log in and configure the web analytics tool to accept all the data that will be sent to it, classify it in the proper way, and ensure that it’s stored in the right place. There are also a number of settings and configurations that each tool requires in order to report the data required to feed the KPIs that feed your business objectives, and once again, the solution design will be your guide. Many tools additionally offer the ability to create custom reports, custom segments, and custom dashboards, and these are often required (and often third party data visualization tools as well) to surface the data required by the business stakeholders.

Quality Assurance (Yep, Again)

With the tools configured, it’s time to QA the data that’s being collected in the tools against the solution design. Again, a formal test plan complete with test cases and the execution of that test plan will ensure you’ll be getting what you need when you push this thing into your live production environment.

Production Deployment

This is where it all comes together, and this step includes that extremely stressful moment where you point everything at production accounts and push your solution out into the wild.

Depending on your organization’s development cycles and production processes, you’ll simply want to slot this in as you would any other update or change, and in the case of mobile apps, this will involve the re-publishing of a new version to the appropriate app store(s).

Quality Assurance (Not A Typo, One More Time)

Contrary to popular belief, there are some things that can go wrong when shifting from a perfect deployment in a staging environment to the production environment, so it’s a good thing to QA the production deployment one more time. The same test plan should be executed and the results should be documented.

If any errors are found, you’ll simply follow your organization’s process for production fixes or possibly roll-backs to ensure that you’re not disrupting the user experience.

Knowledge Transfer – Closing The Loop With Stakeholders

With all this technical stuff done, it’s time to collect some data. We typically recommend waiting a week or two so that you have some meaningful data to look at, and that’s usually at least how long it takes to schedule time with those stakeholders you worked with in the first step of this process.

Knowledge transfer is about showing those stakeholders how to find exactly what they need to make the decisions they need to make based on the framework everyone agreed on at the beginning of this process.

Remember that example about the KPIs to tell us how we’re doing with customer retention? Well, the purpose of this transfer of knowledge is to show the people responsible for customer retention how to log into what, navigate to where, and find exactly the data and KPIs they agreed would tell them if they’re doing a good or bad job with this organizational objective.

Ongoing Maintenance

Like a garden, once you’ve planted it, the work has only just begun. Technologies are constantly evolving, business objectives will change, and you probably won’t fit everything you wanted into your first deployment. That means that you’ll have to plan for a cycle of ongoing maintenance and improvement:

• Ongoing Quality Assurance (QA). Your web properties and apps are constantly changing, and you need to make sure that your analytics data isn’t being impacted. Fortunately, this can be automated to a large extent with a host of QA tools (including WASP as mentioned earlier) and help you make sure that new pages, new frameworks, and changes aren’t impacting your data collection.

• Updating Your Business Goals. This is where good data governance comes into play. Having a formal plan and committee to ensure that business conditions are consistently aligned with data collection, reporting, and goal setting is what will drive updates to your business objectives and the KPIs that will measure their success or failure.

• Updating Your Implementation. Whether it’s keeping up with new features or making changes to reflect a changing business environment, updating the solution design and documentation and then making the changes to code and configuration is a need that you’ll need to plan for on a continuous basis.

• Adding To Your Evolving Data Stack. You’ll find that as your organization becomes more and more mature with respect to data and analytics, you’ll require more integrations and more new components to your larger data stack. Whether it’s adding new data sources as discussed in integrations earlier in this article, adding more efficient and effective data visualization tools to the mix, automating data hygiene and preparation processes, or simply adding new functionality to your toolset, you’ll be well served to plan and budget for growth in this area.

Now What?

Hopefully, at this point you know if you need to do some work on your web and app analytics programs, and this has helped to provide a process to ensure that the data you’re collecting is accurate and trustworthy. But more importantly, use this process to activate the right data that can drive the right KPIs for the right people in the organization who can start using it to make the right decisions.

 

Originally posted by Marketing Land on March 2, 2015.