|
|
A Different Approach To Good Reporting In Web Analytics
By Gary Angel
Expert Author
Article Date: 2007-12-03
I've written before that Management Reporting is one of the most difficult tasks in all of web analytics. It's an area where - as a company - we've struggled mightily.
Nor is it the case that when I see other reporting styles and implementations I'm very impressed. Mostly they are worse, often much worse, than Semphonic efforts that I judge failures.
I've also written previously about why I think so many efforts at reporting are misguided. They tend to over-focus on identifying individual KPIs instead of describing real-world situations. And when they do describe a real-world situation, they generally lack the intelligent context to make clear to the decision-maker what's driving change.
We've done some work recently that I think addresses both these concerns and has produced some reporting that is significantly more interesting.
I'm going to illustrate this work with a report on Traffic. Yes traffic.
Traffic is nothing to sneer at. Visits to a site may not be the type of KPI that you are supposed to focus on. But in my experience, it represents a crucial real-world fact about your online business opportunity. In addition, I get more questions about changes in traffic than almost any other single variable.
Suppose your report set includes a nice chart like this:
When a marketing manager sees the July to August, August to September, or September to October change, a perfectly reasonable question is generated. Why?
It's possible that the consituents of an answer are already present in the report set.
Changes in visit traffic may well be driven by improvement or decline in various sources. If the report set also contains a report on marketing channels, the seeds of an answer might be there:
A chart like this contains a chunk of the data necessary to undertand the traffic variation shown in the 1st chart. But it certainly doesn't make it easy to answer the question. Nor is all of the necessary data contained in this one place.
We can see, for example, that the Direct Traffic is mirroring the overall pattern - two months of decline and then a recovery. Perhaps it's the causal cuplrit. But what's causing the Direct Traffic to change?
To understand that, a manager might look for a different chart somewhere else in the report. Making the assumption that direct traffic is driven by offline marketing and repeat visits, the manager might look for something like this:
We can see that there was indeed some decline in visits per visitor in August. But that September actually went up instead of down. So changes in Loyalty may have been responsble for some (how much?) of the decline between July and August but can't possibly have contributed to the decline between August and September.
Going back to the source chart, we can see that the decline from August to September looks to be biggest in Organic Search. So perhaps we would drill down into a detailed report on Search to see a comparison of August and September.
Google and Yahoo are both down - so that may be the driver for the monthly change. But it's interesting to note that some other engines are actually up - forming a small counter-trend.
Now, we just have to explain the upward tend in October!
You can see that this is getting to be a lot of work. You can also see that it would require lots of charts and tables and fair amount of smarts to get an answer about what's driving traffic change. That will lead to reporting bloat - and most users will wonder why they are drowning in so much apparently useless data.
To combat this, takes a very different approach. The essential idea is to build an analytic model of a concept like Traffic (or Conversion or Satisfaction or Efficiency or Engagement) and then embody that model in a report.
To do this we dump all of the relevant data for the model into a working spreadsheet. Then, we create a (fairly complicated) Excel Macro that processes the data using the analytic model. It then populates the report spreadsheet with the core numbers being tracked along with the analysis of the key causal factors and counter-trends.
The beauty of this approach is two-fold. First, a good deal of analytic complexity and intelligence can be built into the model. This can prevent a decision-maker from misunderstanding or missing the contribution of key factors (like visitor loyalty) on traffic. Second, it allows the report to encapsulate all of the key information in one simple presentation that provides actual immediate and well thought-out answers to the inevitable questions.
Here's an example of the output for the Traffic Model from this situation:
The model captures not just the trend in traffic; it immediately identifies the factors that drove the trend, the extent to which they contributed, and any counter-trends that were also in-place. To get the same intelligence from any non-model based report set would take big chunk of work and a fair amount of knowledge about web analytics. The two things that a good report set is supposed to make unnecessary!
The more I work on reporting, the more I think that the distinction between analysis and reporting is meaningless. Because unless you embody good analysis in your reporting, you'll never deliver a good report set.
Comments
About the Author:
Gary Angel is the author of the "SEMAngel blog - Web Analytics and Search Engine Marketing practices and perspectives from a 10-year experienced guru.
|
|
