🛠 The Product Person #47— 5 Mistakes To Avoid When Measuring Product Data

hi, Anthony here! In this newsletter, I curate insights on how to build great products. You’ll improve your product skills with every issue.

Let’s get to it.

(Btw, my PM Discord group has 50+ members now. Come join!)

(Insight from Pendo’s ProductCraft Blog)

When measuring data for your product, you’ll probably make these 5 mistakes:

• 📈 Not measuring real engagement

• 🙉 Ignoring qualitative feedback

• 🗣 Relying too much on your loudest customers

• 📊 Focusing too much on downstream metrics

• 🤔 Measuring too many things

Let me explain a bit further.

Why you might do this:

• There are tons of easy (read: lazy) metrics to measure (total users, raw page views, downloads).

Why you shouldn’t do this:

• Just because it’s easy to track doesn’t mean it’s actionable or tells you whether users are getting value from your product.

What you should do instead:

• Look less at “total metrics” and more at “frequency metrics”. (For example: how often do they use the product after signing up? How often do they use features X, Y, and Z?)

Why you might do this:

• Staring at dashboards of product usage data is a lot easier and cheaper than talking to customers.

Why you shouldn’t do this:

• That data often lacks important context — it lacks the “why” of certain behavior you’ll see on graphs and charts.

What you should do instead:

• Talk to users!

Why you might do this:

• It’s tempting to always reach out to them since they’ll happily provide feedback.

Why you shouldn’t do this:

• They often don’t represent the majority of users — For ex: they might be so vocal because they’re using your product for unintended edge cases (so they’ll need more support).

What you should do instead:

• Actively reach out to your less vocal users to get holistic data so you can make more informed decisions.

Why you’ll do this:

• Downstream metrics, like revenue, are easy to track.

Why you shouldn’t do this:

• When a customer churns, just seeing “-$X” isn’t an actionable metric for understanding why they left or how to fix the problem.

What you should do instead:

• Along with revenue, track more upstream metrics around user engagement within shorter time periods — For ex: Login frequency. If you see that a user logs in less and less over a 30 day period, you can reach out early to see why.

Why you’ll do this:

• With so much potential data to track, why not track them all?

Why you shouldn’t do this:

• “Analysis paralysis” kicks in.

• There’s no point in collecting data for data’s sake.

What you should do instead:

• Collect the most essential metrics first then collect more as you realize which metrics provide actionable insight and which don’t.

• Once a metric stops being helpful — stop collecting it.

📌 Announcement: If you’re a young PM looking for community with other young PMs, join my Discord group :)

Thanks for enjoying The Product Person. I’d love it if you shared it with a friend or two. If this was shared with you, you can subscribe here.

❤️ Hit the heart button as well, it helps a lot.

Have a great day,
Anthony

(P.S. Anything I can help you with? Just reply to this email, I’ll respond)

One last thing, how did you like this post? Vote below!