A couple weeks ago at the Internet Marketing Conference here in Vancouver I had the great opportunity to ask a handful of questions to Gary Angel. Gary is an expert in the field of web analytics and is the President and CTO of Semphonic, a web and search engine marketing analytics consultancy.
If you enjoy the interview and don't want to miss out on anymore of the knowledge that Gray regularly imparts, then be sure to head over to his analytics blog and subscribe to his feed.
Here is the interview.
What is the most common mistake you see people making when analyzing analytics data?
Two things came to mind when I thought about common mistakes in web analytics data. One is that there is a prevailing myth in web analytics data that you can trust the data, in particular, you can trust the data if you trend it. That is simply not true. Trending is not a solution for a data quality issues. Lots of times I see people say: well, my visitor count went up and I know that my visitor counts are not really accurate, but because I have trended it that's meaningful. The problem is that data quality issues often drive trends, so if you look at trends and you think that because you are looking at a trend therefore the data is trustworthy, it's just not the case. A lot of times what you are actually doing is tracking data quality issues and their trends inside your system. I think that's an extremely common mistake and one that's been really widely perpetrated in our industry.
I think the other common mistake I see is believing that because two things happen a lot in association on your website, they must somehow be causal. There is a really strong relationship between them. But the truth is that websites have a strong navigational structure and lots of times pages are linked together in ways that make it highly likely that if you visit one, then you will visit the next. I think a classic example of this is, if an order button is only on one page then everyone who orders is going to look at that page, it doesn't mean that that page made them order. So I think a lot of times mistaking correlation for causality is a big issue in website analytics.
What advice would you give a company that is completely new to analytics and looking to get into it?
I think a lot of this depends on what kind of company you are and I think that you need to look at your culture and size of your company and where you want to go with web analytics. Before you make a lot of decisions about how you are going to invest in tools and people.
I think for large enterprises that are maybe coming to web analytics but know that they need to be serious about it over the long run, I think they need to get into web analytics in a much more formal process. They need to do it vigorously; they need to invest in the right technology because they are going to own those technologies for a long time. The larger the organisation, the harder it will be to shift the technology.
If you are small company and if you are agile, if you have one or two people working on web analytics, I think it is a lot easier to let your web analytics grow organically, to get some free tools, to start experimenting, to play around with it because it is a lot easier for you to retrain yourself and re-invest.
So if you are getting started with analytics, I think that the first step is to figure out where you need to go with it. Are you the kind of company that can grow organically, that can learn as you go, that can change quickly, that can sort of figure out what you need? Or are you the kind of company that really needs to put a formal effort behind it and figure out, in one or two years where do we need to be?
How do we determine which analytics solution is best for a particular site?
There are a range of web analytics solutions and it's probably fair to say that most of them are pretty good. There are unique aspects to each of them that can make them particularly appropriate for certain types of sites and certain types of problems. I think that the key things to look are what your data integration needs. There's lot of differentiation in web analytics tools between how easy it is to get data into the tool and get data out of the tool.
If you need to integrate a lot of data, have a lot of marketing campaigns going on, if you have a lot of back office data, those are things that really drive your solution. A lot of organisations have particular privacy or IT concerns and there are two major classes of web analytic tools. Tag based solutions and log based solutions. A lot of times if you have unusual privacy restrictions, if you have unusual restrictions about whether you can cookie a visitor, if you need your data to be behind your firewall, that will drive a lot of your thinking about the tool.
So a lot of this is about sitting down and looking at “DO I have particular needs? I am looking at the tools and saying which of those tools really fit those particular needs. I think one thing that's nice is by and large these days; most of the web analytical tools in the market are really strong tools and do the core jobs for almost anybody.
Can you speak about the importance of tracking in regards to search engine marketing?
I think it's search engine marketing that really drove the demand for web analytics. When you start to spend real money on your website, and that's what search engine marketing is. When people started doing pay per click, it really upped the interest in web analytics and in fact I think that search engine marketing has been probably the single largest driver behind web analytics. It's really, really important.
With search engine marketing tools, a lot of times you can evaluate how good a job you are doing getting people to your website, but web analytics can really expand that view about how successful those people are on the website, where they are failing and where they're succeeding. So web analytics is a key component of search engine marketing, and it's the place where your marketing dollars, where the rubber really meets the road. So if you're going to take advantage of web analytics then search engine marketing is certainly one of the best places to start.
When would you suggest, if ever, that more than one analytics solution be installed on a particular site?
We see this happening quite a bit these days, much more commonly than a year or two ago. To some extent that's because of free solutions like Google analytics and Index tools which have made it economically more feasible to put more than one solution on your website.
The question is, why would you want to do that? There are a couple of reasons. First of all, sometimes you get unique capabilities out of the tools. You may have a tool that does almost everything you want, but is still missing one or two kinds of analysis. In addition, people put tools like Google analytics on their website because they want to take advantage of other tools that are related like the Google site optimizer. If you want to use the multivariate testing suite from Google you may put Google analytics on your website even though you have another analytics tool.
We also sometimes see people putting multiple tools on their website so they can get a better sense of how the data reconciles and whether there are data quality issues. Sometimes it is nice to have two different tools as you can look at the numbers and get a sense for how your organization is doing and whether there are significant differences. If there are significant differences, it may alert you to data quality problems. That being said, there are also drawbacks in having two tools on your website. It can create confusion. Any time, particularly in a large organisation, if you have two sets of numbers circulating around the organisation, there tends to be disagreement about which is right. People spend a lot of their time arguing about things that in the end don't really matter. One thing we've seen is almost any time you have two different data processing solutions, they are always going to produce different answers. Sometimes to extents that are somewhat surprising and maybe a little worrisome.
I think for an organization looking to deploy multiple solutions, if you have a good strong reason, if you have a particular capability you need to get, then that's great. I think there are risks in just having two solutions out there, because I do think that it can create confusion in the organisation.
What do you believe will be the major changes in the field of web analytics in the next five years?
Sometimes the pace of change in Analytics is frustratingly slow. I remember getting into this field not web analytics, but the database marketing analytics more than twenty years ago and the truth is that the types of analysis that most of us are doing right now are no more sophisticated and in Web Analytics in many ways are less sophisticated than what we were doing twenty years ago. That is somewhat surprising and also disappointing, although with web analytics, we took on a whole new set of challenges because the quantity of the data which we have to analyze is so vast.
That being said, I think that probably the biggest direction in Web Analytics is going to be increasing interest in data warehousing web analytics information and joining it to customer information, really opening up that information from an IT perspective or platforms that are not just reporting platforms, give people a wide variety of analytic, modeling and statistical tools. The sophisticated organisations that we are working with that have sort of gone through the process of mastering Web Analytics, have stuck one or two years in getting a basic infrastructure in place, getting good at Web Analytics reporting. Where they are going now is warehousing that data, joining it to their customer information, opening it up to a whole new set of tools, a whole new set of analysts, a whole new set of reports. I think longer term that's where the industry is eventually going to go.
How can a company track PPC conversions that are not happening online? For example phone conversions.
It's a really common problem. It's not just a problem related to pay per click. The vast majority of clients we work with are multi channel. It's not unusual for them to have a wide variety of conversion methods, sometimes by phone, sometimes by retail. It's difficult in a lot of respects to bring that information in. From a PPC perspective, one of the things that you can do is to use the technique called Conversion proxies. Basically a Conversion proxy is a measure of engagement. It's tracking the behavior on the website, in terms of its quality prior to, for instance, submitting a lead or on the website itself to get a sense of how different PPC campaigns are performing. That can get you a long way beyond what you do if you optimize just in terms of how many people you are bringing to your website.
One thing I always say is that there's no worse strategy in PPC than optimizing to the number of clicks you get. That's a terrible strategy for almost every PPC program because by looking at the subsequent behavior of the people on your website; you will see striking differences in the quality of visitors, the different key words and ad groups and the campaigns resourcing from Pay per click. So even if you can't tie all the way back to offline, even if you can't make an absolute connection between whether people came to your website and bought, you can do a lot a of optimization in terms of Pay per click by looking at what people actually did do on your website.
What are the most important factors to track in a PPC campaign?
There are a couple of things that I can think of that come to mind, and maybe I'll talk about one as well that may not be the absolutely most important thing but I think is often overlooked. We see a lot of people when they deploy creative in Pay per click simply measure the effectiveness of Creative by how much click through it drives. In fact, there are tools out there like the Google Ad rotation tool that really encourage you to do that. The problem is that when we actually look at the quality of visitors being driven by Creative, we see that it is strikingly different just as the quality of visitors is different by keyword.
Most people know that they will probably do a lot better with certain kinds of keywords, maybe their brand keywords, or keywords that are highly focused or keywords that are to oriented toward the buying process as opposed to keywords that say “Get a free quote” if you are a stock broker. We all sort of know that keywords drive highly different quality of visitors. What I think is being ignored by a lot of PPC buyers is that Creative has the same effect. Some advertisements and some creatives drive much less qualified visitors. They look great because the Clickthroughs are good on them. They are actionable, they are aggressive and they get people on their website, and you know what, the people on the website don't do anything. So it's often overlooked that Creative just like keyword has to be measured in terms of the quality of the visitors that are coming in.
I think that the other thing and this is really important just in terms of the broader Pay per click campaign is understanding the overall value of the people you are bringing to the site. We see a lot of clients make a fundamental mistake about PPC thinking that all the people who come through their PPC channel are actually new prospects. That is just not so. Over and over again, we see that customers are coming from PPC sources for customer support; they are coming for truly branding purposes, they're coming even though they have been to the website ten or fifteen times before. They'll still use the PPC ad to come to the site.
So if companies look at PPC as just a prospect vehicle. They can often (1) overstate the impact of how many prospects that it's bringing to their site and (2) understate it's conversion efficiency because lots of people that are coming through PPC aren't really good prospects at all.
So it's really important that when you measure PPC performance, to look at the types of visitors coming to your site, to segment them out to see what they are really doing on your website and not just assume they're all a bunch of prospects coming in the door. That's a very common mistake we see in PPC measurement.
How is a company able to measure the effectiveness, with analytics, of its branding efforts?
There are two aspects to this, both of which are really interesting. One, I believe truly under looked value in web analytics measurement is that they can give you tremendous insight into how effective all of the rest of your mass media efforts actually are.
These days, the web is an integral part of consumer behaviour and what we really encourage clients to do is to use the website and web analytics not just to track the performance of the website, but use web analytics to track the performance of their mass media. Using web analytics, you can actually see and understand the performance difference between things like television ads that you are running in different media markets.
You can use the web analytics to see how much additional traffic to your web site those different creatives and those different mass media channels are driving and what the quality of the visitors they are driving. It's like the world's largest focus group and for a lot of mass media efforts that are all about branding; there's been very poor ways to measure those. A lot of times people do follow up opinion research and do simple surveys like do you remember the brand, are you impressed by the brand, would you buy from the brand. Well, looking at the actual behaviour of subsequent consumers, did it drive more consumers to your website? Were those consumers the people you wanted to come to your website based upon the things they did there? It's a really powerful technique for optimizing the rest of your branding efforts and I think that branding people have looked upon the web as just another channel to push the brand out, to have it rewrite a tremendously powerful channel for looking at how well all of the other branding efforts are really doing. We are encouraging a lot of our clients that do a lot of mass media spending to do that.
Then there is the second side of this question. My website, part of what it's supposed to do is branding and part of what it's supposed to is to get my brand across to people. Now how can I measure that and that's a really, really tricky thing to do. Almost all of our clients from the very biggest to the small and midsized guys are struggling with this issue.
There are a couple of techniques that I think are helpful. The one interesting technique that we sometimes use to measure brand impact is to run simultaneously surveys of people who are entering the site and people who are leaving the site. We make sure that we are not surveying the same people. We are actually surveying two different populations randomly, people as they enter the site and people as they leave the site. We are asking them a very short set of questions relative to their impression of the brand and by running those two surveys simultaneously; they can a much better sense of “did this site really have much of an impact in the way people are thinking about my company”.
I think that when people use opinion research, use voice to customer, use online surveys, there's been a tendency not to use them very flexibly. A lot of times we will just deploy them, but deploy them against a random sample of people, deploy them all in the same place and the only thing they look at sometimes are the overall satisfaction score for the website. But opinion research and online surveys are tremendously flexible and powerful tools. They can be used creatively in a lot of different circumstances to understand the psychological impact that your site has had on people.
So what I think is the biggest thing that I would say to people looking at measuring the brand impact of their site. First of all it is important; branding is a big part of what most websites are about and secondly, you can't compare it with GRPs and mass media. What is really different about the web is that the experiences are so dramatically different. Sometimes people engage very deeply and sometimes their experiences are very shallow, whereas with a TV commercial, everyone gets the same 30 seconds. There is no corollary. You can't take visits, you can't take time on sites and there's no direct corollary to the types of measurements we're used to in mass media. That being said, if you look at where people are going on a web site and look at the functions they are doing. If you carefully survey them in terms of what the impact was on the people who looked at the areas, you can get a pretty strong sense of how powerful each area of your website is. I think that it is important to understand that a website is made up of a lot of different pieces some of which have a much stronger branding impact than others.
So it's important not just to look at things like time on site, but to look at the functional components of your site, the tools, and the core areas of your site, how much time people spend on those and then to look at the subsequent impact in how people talk about your brand and remembered your brand and expressed their opinions about your brand. I think if you do that, it's really hard to quantify brand measurement, but if you do those things, I think that you will still be way better of than you are in the non online world, whereas right now I think in a lot of ways traditional mass media people have a much better sense of the brand impact than we do online. So that's something that you can definitely remedy. You are never going to get it completely right, but you can get much closer to the truth about how important that brand impact is.
What tools do you recommend to your clients for tracking their PPC campaigns?
First of all, and I think that I touched on this earlier. PPC campaigns have been kind of siloed. A lot of times when they are implemented by agencies and even when organisations do them in house, they are not well integrated with the rest of the web measurement effort. I think that's a mistake.
One tool I definitely recommend is a web analytics tool. It doesn't matter whether it is Omniture, Google analytics or anything else, but the web analytics tool will tell you things about your PPC program that things like Double Click, Spotlight tags or your Google reports will not tell you. At the same time, web analytics data is not the complete story either because it doesn't bring in the impression information and the cost information that's obviously vital to understanding what's going on with a PPC campaign. There are a lot of good bid management solutions out there. Our clients range all over the map, from just using the tools provided by the major search vendors, they go into Google or go into Yahoo. If you are primarily buying on one engine, most people who are buying on one engine will just use the tool from that engine.
I think where it gets trickier is if you start to do multi engine buys, it starts to be operationally difficult, not only to do the buys on multiple engines but to track the buys on multiple engines. That's where you need to start thinking about an additional tool to help with tracking and not just the web analytics side, but the cost and impression side of PPC. There are tools out there, one from Commerce 360, Click Equations is real interesting and these help not only from a bid management and operational side, but in terms of bringing that information together from multiple engines and really helping you track what is going on. I think that gets to be a challenge for people. If you are only on one engine, my experience is that a lot of people can get by with the tools the engines have. It's hard when you are on Google, Yahoo and MSN and you have to jump from one platform to another and they all have different reports. Then you have to take those reports and put them all together which can be a big hassle and that's when having the right tool can save you a fair amount of time.
