Measuring User Engagement, with Examples from Yahoo

If you own a web site, how do you measure the way that people interact with your site? What data do you look at, how do you analyze it, and what do you do with that analysis?

The topic is becoming a popular one on the Web, and I have some links below to some articles on the subject that I thought were pretty interesting. I was inspired to collect those links after looking at a patent filing from Yahoo that describes some of the methods that they might use to try to understand how engaged people are upon their web properties.

The patent application is Techniques for measuring user engagement, and the listed inventors are Francesca M. Soito and Nitin Sharma (who appears to have now moved to Google).

User Engagement Variables

There are much better ways than using “page views” as a metric for measuring user engagement. In a fairly general way, this document explores some other techiques that could provide meaningful analysis of the way that people interact with their web sites. It looks at user engagement variables in three areas for different services and products that it offers:

• Number of sessions with a product,
• a time spent interacting with a product, and;
• a user’s actions involving that product.

Based upon those interaction, it could create a user engagement score, which could be segmented in different ways – based upon time, viewed as a frequency over time, and so on.

The authors use an email program as an illustration of how this process is used. I’m wondering whether this kind of process is really something that is patentable, but regardless, it’s an interesting inside look at how a couple of search engineers from Yahoo might suggest measuring how people interact with a site.

They do provide a big list of things that they might count as “user actions” within the context of that email illustration. Let’s consider a different example, and come up with our own list. If this process were applied to Yahoo’s photo site Flickr, what kind of information would you look for as a measurable user action, to tell how well the service is working?

Here are some:

1. Free Signups
2. Premiums Signups
3. Photos uploaded
4. Photos uploaded by users of the free, and the premium services
5. Information about captions created for photos
6. Information about tags used for photos
7. Comments left on photos
8. Groups created
9. Growth of groups
10. Percentages of people joining groups
11. Contacts made
12. Pictures geo-tagged

In other words, I would try to identify all of the different activities that people can take on the site, and find different ways to measure that activity. Looking at that activity, plus information about user sessions, and time spent on the site could possibly provide me with some interesting information that I can act upon.

For example, if memberships in user groups start to slow significantly, I will want to explore why. If the number of images uploaded by premium members drops significantly, but the number of pictures added by free members increases significantly, I might want to uncover possibly reasons for the change in activity by the two groups.

Demographics and User Engagement

The patent briefly brushes upon the topic of looking at demographic information, if any is available for the user population, in conjunction with user engagement scores. It’s possible that this kind of information might be helpful in developing strategies for improving user engagement.

They note that if the “low engagement segment of the user population has a high proportion of users corresponding to a particular demographic, new product features targeting that demographic may be introduced in an effort to increase the engagement of those users with the product.”

Demographic information could be broken down into a lot of different types of information. Here are the examples that they provide:

Gender,
Socioeconomic status,
Tenure,
Online behavior metrics,
Property usage (e.g., page views generated on other properties),
Age,
Which country the user is in,
etc.

User Engagement Articles

If you’re not looking at and trying to measure how users engage with your website, you may be missing out on some valuable information that could be used to make your web site more effective. These are some of the articles on the subject that I found interesting:

• User Engagement is a Depth, not a Breadth, metric
• Social Media White Paper “Tracking the Influence”
• Web 2.0 Metrics: The More Things Change…
• An Eric Peterson five part series on the subject of User Engagement:
  • How do you calculate engagement? Part I
  • How do you calculate engagement? Part II
  • Calculating engagement, part III … social engagement and relative content grouping
  • The engagement metric, defined (part IV in a series)
  • Measuring social activities online using my visitor engagement metric (Part V in a series)
• What Is Web 2.0′s Magic Metric?
• Standard Metrics Revisited: #3: Bounce Rate
• Rethink Web Analytics: Introducing Web Analytics 2.0