Tips Tricks, Traps and Tools: #4.1 of many: Understanding Direct Visits in GA & Web Analytics : UDVs

Direct visitors may be your most valuable. Keeping track of them makes sense. Advertisers, for example, may be more interested in your direct traffic than your paid traffic. And then there are the New Direct Visitors. That requires some expensive advertising or valuable word-of-mouth. If you are doing any off-line marketing, direct traffic KPIs rule (along with some fellow megalomaniacs). So it makes sense to identify direct visits, to segment direct visitors and to track conversions within that segment. Post #4, GA User Defined Variables (UDVs) Revisited described the Scenario “Improved Attribution” requiring that direct visitors status be recorded so as not to be overwritten by subsequent traffic sources. We delve more deeply here, distinguishing between direct and “external” visits and between direct visits and direct visitors. We also provide two code snippets below. While this post caters to Google Analytics into account, many of the concepts and explanations apply to Web Analytics generally.

How Traffic Sources really work. What is a real direct visit?

There are really 3 different points of view as to whether a visit is direct or from an external visit source:

• Browsers and the internet (let’s call that HTTP). HTTP is not concerned with visits and their origin. (In fact without cookies and other mechanisms browsers and web servers cannot remember visitors and sessions). A referrer to HTTP, is the page used to request another page. The two pages may be on the same or different domains.
• GA and some other vendors’ tools begin with that definition but muddy the distinction:
  • • Visits that would normally be direct are considered external if the “campaign cookie” (__utmz) contains a campaign or referrer from a previous visit.
  Like HTTP, referring sites are those from which a visit originated, but only if they are not designated search engines. Eg google.com in the Referring Sites Report does not refer to organic search but to other Google sites such as Gmail
• Visits from non-HTTP sources (eg emails, address bars and bookmarks) are direct unless they contain campaign tags
• Site owners will want to distinguish between direct and external visits differently, depending on their requirements. To track direct visitors more closely, they would define direct visitors in the same way as HTTP. This post will show how to implement that requirement.

In this post:

• We use the UDV to segment visitors as having arrived on a direct visit al la HTTP. No change is made to the way GA tracks visits nor how it credits external traffic sources for visits.
• Direct Visits are based on the HTTP referrer. If there is no referrer (no originating HTTP page) the visit is direct and the visitor will be segmented as having visited directly.
• If the visit comes from a tagged email one may want to define it differently. By default, HTTP would regard that as direct but it did originate via an external source. In this post they will be considered to be a special case.
• External visit sources are those which come from AdWords links, banner ads and other referrals.
• ga.js uses the “Referrer” attribute of the browser’s request for the landing page to determine the source of the request.
• Clicking on a link in a search engine, generates a request for a page on your site and reports the originating URL as the referrer.
• If there is no Referrer, the visit is a direct visit.

Campaign Cookie

If the visit originated externally, ga.js sets the external traffic source cookie (__utmz) at the start of the visit to record how the visitor got to your site. The __utmz cookie persists for 6 months to allow attribution of conversions to the traffic source. The __utmz cookie is also known as the Campaign Cookie. It is not a misnomer if one considers campaigns to consider all efforts to externally drive traffic to one’s site. Campaign is also a fair term since all involved in marketing campaigns will agree its a war out there. However, calling it an external traffic source cookie promotes understanding and helps one remember that “direct” is only recorded as a default when no external source is already in place. With that in mind, the following scenarios are easier to internalize:

• a visitor, who’s last visit(s) were from external sources, returns directly. GA does not overwrite the traffic source cookie and the traffic source is still credited with the visit.
• a visitor, who’s previous visit was recorded as direct, returns via an external visit source will have their External Traffic Source cookie overwritten to record the (new) external traffic source.

How Traffic Sources don’t really work. A GA AdWords Conspiracy?

I’ve read and heard it said that GA designed their Attribution around favoring external visit sources just in case that source may be an AdWords link. To add to the theory, external visit sources are clearly afforded some priority over direct. Does that suggest a conspiracy or even some bias? I doubt it. The AdWords folks have other ways of showcasing AdWords conversions. That preoccupation serves only to confuse one’s understanding of the concepts involved and creates mnemonics that are better forgotten. Getting real, external traffic referrals are mostly the result of costly efforts and the Attribution Model is designed to track success of what is being spent, whether on AdWords, Google’s competitors’ products, affiliates, email campaigns, etc. Direct visits are usually a consequence of that spend and so, those too should be attributed to the originating external sources. Direct visits are also more nebulous. Returning via one’s browser history or from memory or seeing a billboard or bus ad. Tracking direct visits is by no means unimportant. Its just not the aim of Campaign and Traffic Source tracking.

Identifying Direct Visitors

			// before var pageTracker = _gat._getTracker...
	var isDV = 	(!document.referrer) && 
			(document.cookie.search(/(__utm[cb]=)/g) == -1 ||
			 document.cookie.match(/(__utm[cb]=)/g).length <= 1
			);

	...
	
		// after  var pageTracker = _gat._getTracker and 
		// after  any pageTracker._set*() function calls ... 
	if (isDV) pageTracker._setVar('Direct Visitors'); 

That’s it. The snippet checks that there is no referrer. If there is a referrer, this page is either not the landing page or the visit came from an external traffic source. So, the last 2 lines check that there are no visit cookies and confirms we are on a landing page. On subsequent pages there are referrers from the current site but the visit cookies would already have been set. So now we know it’s a direct visit so all that’s left is to set the cookie. If the visitor returns in a later visit from an external site, the __utmz cookie will be updated to reflect that source. Our UDV cookie will remain intact. This visit would be counted among those that direct but which can also be attributed to the particular external source. But for one missing piece of functionality, we are now able to see which traffic sources result in visitors who return directly.

Who’s on 1st? Direct or External?

We don’t know, yet. How many, previously direct, visitors returned via an AdWords campaign? Since GA designates such visits as external, that fact is lost. We extend our snippet to compare the two. Add the referring source/medium as per the “Campaign Cookie” to the UDV value so:

	var strCookie = document.cookie;
	try {
		var strSource = strCookie.match(/utmcsr=(?!\(direct\))([^;|]*)/)[1];
		var strMedium = strCookie.match(/utmcmd=(?!\(none\))([^;|]*)/)[1];
		var strSourceMedium = strSource + '/' + strMedium + '|';
		}
	catch (e) {
	   var strSourceMedium = '';
	} 
		// before var pageTracker = _gat._getTracker...
	var isDV = 	(!document.referrer) && 
				(strCookie.search(/(__utm[cb]=)/g) == -1 ||
				 strCookie.match(/(__utm[cb]=)/g).length <= 1
				);

	...
	
		// after  var pageTracker = _gat._getTracker and 
		// after  any pageTracker._set*() function calls ... 
	if (isDV) pageTracker._setVar(strSourceMedium + 'Direct Visitors'); 
				

The resulting UDV would still only be set in a direct visit and would look so:

• A visitor who first arrived via Google organic search returning directly: “google/(organic)|Direct Visitors”
• An originally direct visitor returning via AdWords: “Direct Visitors”