BYOMSPM

In this post I go over two articles that discuss how to improve metrics, and I added some questions posed to myself to provoke critical responses in bold.

Grade I gave myself for this assignment: 89/100 (it’s short but I will definitely use some of these strategies)

First I read an article about how to choose better metrics to measure your UX product, which argued that metrics like daily average users (DAU), monthly average users (MAU), or total page views aren’t actually indicative of the product working well (Elman). Elman suggests figuring out a more specific metric that better measures how many people are actually using your product in the way that you want them to be using it. Can you give me an example of a more specific metric? For example, instead of measuring daily average users, measure how many users navigate to their own profile page on a given day and track that over time.

This was reiterated in another article I read, which suggested that big picture metrics like page views or DAU also aren’t very helpful for evaluating the impact of small changes in a bigger product (Rodden). Can you give me an example of a feature change that likely wouldn’t have an impact on a big picture metric? A metric like daily average users isn’t really going to tell you whether changing a the placement of a button on the UI was successful (unless it immediately plummets); you would want to be more specific about the page you were trying to drive more traffic to or the change you were hoping to see.

Rodden suggests that to more holistically measure the quality of the user experience, use the HEART method:

• H = happiness/how satisfied users are with the product; collected through user surveys or user interviews; metric might amount to a NPS or perceived ease of use
• E = engagement/level of user involvement; collected via more complicated metrics, such as number of blog posts per month or average time spent on a page
• A = adoption; measured with metrics focused on new users or adoption of new features, such as number of users signed up in the last week
• R = return rate (as opposed to churn, or when a customer does not return); measured as a metric over time, such as percentage of users who returned after a month
• T = task success; measures efficiency, effectiveness, or failure rate of a specific task

The argument is that more complicated metrics and measures of success give an idea about not just the usage at a point in time but how the usage is changing over time.

Rodden also outlines the Goals-Signals-Metrics framework for determining more informative metrics to center around:

1. First, define your big picture goals of your product or initiative. Be as specific as you can about the impact you hope to have on customers.
2. Next, figure out what user behavior will signal to you that you have met your goals. This isn’t as specific as a metric, so it might be that the time users spend on a given page increases. To analyze which signals you want to use, consider which are the easiest to measure and which will likely be the most impacted by success or failure towards your goal.
3. Lastly, convert your favorite signals into specific metrics that you can actually track or use to test.

Can you walk through an example of a product initiative and how you would use the Goals-Signals-Metrics framework?

Fictional product initiative: adding an FAQ page to a website

Goal: reduce amount of calls that come into call center to ask questions

Signals: number of calls to call center reduces; total time spent by customer service representatives on the phone with customers reduces

Metrics: the average number of calls received by the call center per day

Thanks for reading.

---

Works Cited

Elman, Josh. “The only metric that matters.” Medium. 2 November 2012. https://joshelman.medium.com/the-only-metric-that-matters-ab24a585b5ea.

Rodden, Kerry. “How to choose the right UX metrics for your product.” Medium (GV Library). 2 December 2015. https://library.gv.com/how-to-choose-the-right-ux-metrics-for-your-product-5f46359ab5be.