Bad analysis is a result of bad design pt. 1 – The Rant

Finding meaningful insights in data means knowing how to combine data and make useful comparisons. Insight is in the details. All data in aggregate is “crap”. Everyone keeps saying this.

So why is it that reporting suites give you such useless metrics by default?

Let's take the first two screens some one who has just logged into a Google Analytics account will see: the profile screen and the dashboard. On the profile screen there are three metrics: visits, ATOS, bounce rate and completed goals. The dashboard, on the other hand, populates with visits, pageviews, average bounce rate and time on site, and a nice looking, but less-than-useful, total-goals overview.

What does it communicate to people new to analytics when the most repeated metrics are visitors, avg. bounce rate, and ATOS? It communicates that these are important metrics. It communicates that they should be looking at aggregates.

The default reports aren't much better, requiring intense filtering and sorting to find any really useful information.

This is why we see so many marketers that use Google Analytics as a “hit tracker”: interfacing decisions communicate importance of particular statistics, and in the case of most reporting suites, those statistics are unsegmented data. Meanwhile important data requires digging, enigmatic custom report creation (why can't I compare both visits and total goal completions by page title?) and an inquisitive state of mind.

Bad analysis is a result of bad design. But it's easy to complain about design. It's much more difficult to design solutions.

So how could we go about building an analytics interface that would encourage new users to search for meaningful information? How could you improve reporting to encourage better data? I'm afraid that's going to half to wait for part 2…

In the mean time, I want to hear your recommendations, ideas, and rants about reporting interfaces. Send them to kentclark+analysisbydesign@vkistudios.com, message @ordinarychap on twitter, or leave a comment below.