7 Days of Urchin 7: Day 7 – Filters, Lookup Tables, & Customizable Geo-data

Finally, the last post in the series! It’s taken a lot longer for all these posts to go out, but in the end, it’s been well worth it since I now know a lot about the capabilities Urchin 7 has to offer.

So, in this final post, I’d like to talk about the new ability to order your filters (finally!), new external lookup tables, and the ability to choose what areas of the world you are interested in and only include detailed reports about those areas.

Filter Order Matters

In Urchin 6, the order in which filters were applied was determined by filter type. If multiple filters of the same type were applied, then the order the filters were linked in determined the order. The first stage of ordering could affect profile processing time in cases where multiple filters were applied then you excluded certain data (ie. IP addresses). On those pieces of excluded data, Urchin would waste processing time by applying all the filters on data that wouldn’t be in the reports anyway. The second stage of ordering really affected multi-step transformation filters. You could only link the filters to the profiles one at a time, and you would need to make sure you linked them in the right order. Otherwise, you would need to unlink your filters and re-link them. Well, in Urchin 7, they have now added the ability to control the order in which filters are applied. So you could create all your filters and apply them, then worry about the order.

External Lookup Tables

Lookup tables are tab-delimited files that allow you to search a field for a value and replace and/or populate other fields with values. Usually, people use them for master tracking codes, but other situations where lookup tables are useful are cases like doing friendly URL naming, populating user-defined or custom fields based off of username etc. Prior to Urchin 7, these files had to reside in the folder lib/custom/lookup tables. With Urchin 7, you can now specify a lookup table as external. An external lookup table isn’t as sexy as it sounds though. If you are thinking you can specify any data source and have act as a lookup table (perhaps maybe a dynamically updated lookup table by querying a database?) you would be wrong. What external lookup tables allow you to do is specify lookup table files outside of the lib/custom/lookup tables directory. In Windows, you can specify UNC files (ie. a file shared over the network). But in Linux, you can only specify files local files. Here’s a screen-shot of how to create an external lookup table in Linux. In Windows, you will have additional settings to specify the username/password to use for UNC files.

Now, there are some reasons why you would want an external lookup table. A couple of reasons I can think of are:

• the table file resides on another machine on the network and you don’t want to download the file on a scheduled basis
• you automatically update the file on a schedule, but you don’t want the process that creates the file to have access to other folders in the Urchin directory for security reasons

Regardless, external lookup tables are a cool new feature. Here’s hoping they can improve the feature by allowing you to connect to a database or other types of external sources like files via FTP or HTTP!

Customizable Geo-data

The last feature I’d like to go over is more of a performance feature than functionality feature. This feature will really help out for very large Urchin installations or for installations that have limited RAM. By default, all profiles include a host of information about the geographical location of visitors and their network information (ie. domain, and connection speed). Urchin does this by loading a large database file into RAM that maps specific IP addresses to geo-location and network information and then looks up the IP address of each request and maps out the data into the other fields. Now as you can imagine, this file is quite large, and before Urchin 7 the entire database would have be loaded. But with Urchin 7, you can now specify the granularity of the information you want to see either at a global level or on a per-profile level. Urchin allows you to configure the database in two ways: the geo information to include and geo database to use.

For geo information, you can choose to include or exclude

• network locations
• domains and
• geo locations/connection speed

According to the documentation, each additional piece of information you add on will increase the memory used by the profile processing engine. Following is an idea of what kind of memory increase to expect with each option:

• network locations – ~350 MB
• domains – ~100 MB
• geo location – ~20 – 150 MB depending on the geo-location database you choose

As you can see, if you are not using certain pieces of geo information, you can free up a fair bit of RAM which would allow Urchin more RAM to use for processing.

The geo database setting allows you to choose which geo-database to use when processing the profiles. Again, you can set this on a global level or a per-profile level. The various geo-databases available provide different levels of country/region/city detail information. So if you aren’t interested in regional detail level for all countries in the world, you can select a geo-database that provides the detail level you need for your analysis and improve your profile processing speed by decreasing the amount of RAM dedicated to the geo-database. For example, if you have one profile that tracks only US visitors and one that tracks European Union visitors, you could tell Urchin to use the US regional database for the US visitors profile and the EU regional database for the EU visitors profile.

Urchin 7 ships with the following geo-databases to select from

• • World – Contains detailed country/region/city information for the whole world. Use this database if you need detailed information for all countries.
  • World light – Contains country-level information for the whole world. Use this database if you only need to know which country visitors are coming from
  • Regional databases for United States, European Union and Japan – Contains country/region information for the region selected, and country-level information for countries outside of the region.

The selection of regional databases is somewhat limited at the moment, but Google is working hard on providing databases for other regions.

If you were to select the European Union geo database, the drill-down symbol will still appear beside non-EU countries, but drilling down will just result in the valu “(not set)” as in the screenshot below.

So that’s it. There’s my in-depth review of Urchin 7. Hopefully you have found this series of posts useful and informative.

If you would like to read more about Urchin 7, please visit http://www.google.com/urchin.