Actionable Insights. Do They Really Exist?

I'm Going to Optimize!
Suppose you find a page with a high bounce rate.  What are you going to do?

Does web analytics data really contain any actionable insights?

We love data, but what about those "actionable insights" we web analysts are supposed to come up with?

What to Do When You Look at Google Analytics

1.  Find problems and propose an A/B test.

For example, as you scan down a list of landing pages, search phrases or traffic sources, look for ones that perform poorly - the bounce rate is high, the conversion rate is low.  For each problem, you will not be able to come up with a clever, creative solution that's guaranteed to fix the problem immediately.  You are not a psychic.  Don Draper is a fictional character.

Instead, your "job duties" are to find problems.  Then you alert people. Someone - perhaps you, perhaps someone else -  must come up with ideas that might solve the problem.

Since no one knows whether a proposed idea will be an improvement, the solution is an A/B test.  The test will provide data showing whether the new idea produces better results.

For your important web pages, such as your home page, you should always have an A/B test running, so you are continuously optimizing your marketing.  You can A/B test AdWords ads and email campaigns.  If lots of people see it, or lots of resources are required to create it, A/B testing should be part of the mix.  The result is that web site visitors are happier and your company sees revenue increase.

2.  Find Content Mismatch Problems.

Some people want to read about Five Features You'll Want in a Digital Camera.  Other people want to buy one right now.

Search queries have query intent.  Often you can guess what someone is looking for.  They have a question and are searching for an answer.  They have a problem and are looking for a solution.  A technique that works well is to provide what people are looking for.

Similar differences are apparent in traffic sources.  Some websites have links to your site, and those people tend to be looking for something specific.  Other visitors are looking for something else.

When you look at Google Analytics, segment the data.  People arrive at your web site by typing search queries.  They click on links they find on other web sites, blogs, Twitter messages and in emails you send.  In each case, why?  What must these people be looking for ?

Problems with high bounce rates and low conversion rates happen for a variety of reasons.  One reason is content mismatch.  As you are dreaming up ideas to test, consider whether you are offering what people are looking for.  If you are not, and you can address this problem by an A/B test that presents the content people want, you'll often have an A/B test that shows much improved results.

Where to Find Actionable Insights 
  • Do they appear on our analytics reports and charts?
  • Are we supposed to pull them out of thin air?
  • Or what?
Let's sharpen our focus:  We love data, but are the "actionable insights" we suggest really backed up by data, as so many web analysts seem to suggest?

I recently discussed this with Gary Angel - who has a "Ph.D. level" understanding of analytics - asking if he's ever even seen a real "actionable insight."

His reply is controversial and it shows how to increase revenue at your company.

I said:
  • "Actionable insights" are difficult to sell to others because, really, we often have no data to back up our great ideas.
  • For example, if a search phrase has a bounce rate of 85%, you can guess why that might be.  You can guess what you might do on a page to make things better, but really you often have no data to support your claim that things will get better if only the company makes the change you - the analyst - are advocating.
Most marketing campaigns begin as guesses.  Same for most suggestions from web analysts.  Until the data rolls in, you don't know how well you are doing and you can't optimize.

Here's How it Works
The fact is, your web analytics application is a measurement workbench.  It shows what's working and what's not working.  Some search phrases send visitors to your site that convert into customers, while other search phrases bring visitors but few turn into customers.

If certain search phrases, email campaigns or paid campaigns work well, you'll see that happening.  You want to do more of that.

If certain search phrases, email campaigns or paid campaigns work poorly, you'll see that happening.  You'll want to try out alternatives, perhaps using an A/B test, to see if you can come up with something that works better.

Web analytics can become complex, yet no matter how much you look you'll never find a button you can click that says "Optimize" or "Suggest an Actionable Insight."  You or someone at your company can take this as an opportunity to try something new to see if it works better.

Even when something is working well, you can still try out new ideas to challenge the champ and see if you can do even better.

HIPPOs Explained
The HIPPO problem, which has been described in blogs and discussed in papers appearing in peer reviewed journals, arises out of the frustration web analysts feel when their ideas are rejected by people with fancy job titles.  When people have different opinions, which opinion will carry the day?  Supposedly, it's the Highest Paid Person's Opinion.

Those HIPPOs just get on my nerves!  ; )

Is this HIPPO idea really true?  Should we so fervently blame other people for our problems?

If you have proof that your idea will increase revenue, people will jump on it - even highly paid people who have lots of opinions.  The problem is, we often have no proof.

Analytics applications only show data about what's happened in the past.  As for our fancy ideas about how to improve engagement metrics, bounce rates or conversion KPIs, we usually have no data to back up what we are suggesting.  We can pivot an Excel sheet as much as we want, but often we still lack data showing that our ideas will work.

  • Everybody has ideas
  • Without data our ideas are no better than anyone else's

Two Great Suggestions
Gary has two great suggestions for actionable insights.  First, here's his opening comment:
"Greg, I couldn’t agree more. I’ve long fought the idea that there is something directly actionable in a metric – any metric. There never is and the whole search for “actionable” KPIs is deeply misleading."

Uh, say what?

Want Action?  Here's Tip #1 from Gary Angel
If you want to get some action, and increase revenue at your company, here's the first suggestion Gary has:
"I think of analysis as generating two kinds of action – one is to figure out what is actually a problem – so that you set the table for testing."
I love that phrase, "set the table for testing."  If you want to do some real analytics, here's what I think Gary is suggesting.  First, click around on that web analytics application for a while.  You do that all the time anyway, right?  As you look at lists of data, scan for something that's out of the ordinary.

Perhaps you have a form on every page of your site that people can use to sign up for your newsletter.  Now, are you adept enough to segment your conversion rate for newsletter signup by web pages?  If so, you may see that the average conversion rate for all pages is 2%, and most pages have a conversion rate that's pretty close to the average.

But wait!  What if you spot a page with a near zero conversion rate?  ("I think of analysis as generating two kinds of action – one is to figure out what is actually a problem....")

Can you improve this?  Well, you can find out a lot about that page:  Number of visitors, where they were immediately before they arrived at the page, search queries they used, etc.  Something is going on here.  What?

I shall figure this out....

See if you can discover what's going on.  Why do people who visit this particular page almost never fill out that form?

Now, while we're doing all this figuring, let's step back and get some perspective.

  • You, yes you, may be able to figure out what might be wrong.  Maybe the page does not attract the type of person who would be interested in your newsletter.  Or, maybe you can describe the newsletter in a way that's appealing to the people who would read this page.  Or, maybe you should bail and try a different offer on that page.
Let's change the call to action to:
"Buy Now!"

  • No, wait!  You have a lot of ideas, but do you really want to get involved in this conversion mess?  It depends.  As Gary Angel notes, it may be smart to get the "creative" folks involved:
"Analysis doesn’t necessarily generate action. The analyst has to do that – and, in fact, it can be tricky because the analyst isn’t always the best person. We, for instance, aren’t creative folks. So it’s often hard for us – when we’ve identified a problem – to suggest a compelling solution."
You now have an opportunity.  You've discovered a problem.  Pat yourself on the back, smile a little smile, and consider your next step.  You can get the creative people and business analysts involved.

Whether you come up with an idea for improvement, or you get ideas from others, the key thing is data.  It's data that's going to solve your problem.  It's time for ... A/B testing!

You've discovered a problem that no one knew existed.  That's your job.  Realistically, this problem is an opportunity for improvement.  Alert people and solicit suggestions, but be sure to set the table.  Pitch it as a request for their expertise and tell them you'll set up an A/B test for their best ideas, one by one.

Want Action?  Here's Tip #2 from Gary Angel
This second tip works well.
 "The second type of web analytics analysis "is closer to targeting – if you can identify content/tools that work better for a particular segment, it’s close to (but still not quite) a slam dunk to make recommendations. 
You still have to answer questions about how to steer that content to those users and it’s not as if the data provide the answers."

Here's what I think Gary is talking about, by way of an example.  You want to get content to a particular segment, because they convert well when they see that content.  So, let's first discover a problem:  Look at a list of the search queries that are bringing you organic traffic from the search engines.

Do you see any search queries in the list with an unusually high bounce rate?

The bounce rate is 85%.  Disgusting!

Now, using that search query, go through these steps:
  1. Figure out the searcher's intent.  If the query is "San Francisco Florist," it's likely that lots of these people are looking for a florist in San Francisco.  Just a wild guess.  Maybe they want to buy some flowers and have them delivered to someone in San Francisco.
  2. Run that query in Google.  Since your web analytics application is indicating you get a lot of visitors who have typed that phrase, it will be no surprise if you see a page from your site on the first page of the search results.
  3. Click.  This will let you see what's happening with so many people who visited your site.  When you click on the search result, you'll arrive at the same page these visitors are all seeing.
  4. Figure out why they bounce.  It's easy.  Every searcher is looking for something, and people who bounce are not finding what they are looking for.  As a general rule, you have about four seconds for them to find the content they want, or a link to the content they want.  If that doesn't happen, they bounce.
  5. Offer content that caters to their search intent.  Perhaps you could put a large button on the top of the page that includes their search phrase:  "San Francisco Florist."  Now they have something to click on, and you can take them to a page that has the content or offer they are looking for.
Problem solved?  Heck no.  Set up the new treatment of the page in an A/B test.  Get some data to show whether bounce rates go down and conversion rates go up.  Remember, you're not trying to convince anyone that your idea will work.  You are proposing to test an idea and find out if it works.

You do A/B testing all the time, right?

There you have it!
Actionable insights don't just jump out, but if know how, you can enhance conversion rates and increase revenue.
  1. Use web analytics to discover problems.  You can dream up solutions yourself, and ask others for ideas.  Be sure to test ideas using A/B testing, so you have real data showing how successful your enhancements really are.
  2. Use web analytics to discover which content works for various segments of users, and again, use A/B testing to measure your success.

Love web analytics?
... then you'll want an Avinash Analytics Tag Cloud coffee mug.  Take a look.
/* ******************* hello bar code follows - greg **************** */ /* */