Ask HN: Prove the Impact of Your Work
Dear HN I am at a dilemma and seek the crowd wisdom. How can I justify the work I'm doing? (or rather: want to do).
I work at a B2B software company (~1500 R&D, and few thousands of people doing sales, support etc). We have one main product (accounts for the majority of sales) - a hardware appliance with two bundles of licenses - basic and premium. When the product is sold it comes with the premiumm for some time and then it depends on the customer decision whether to renew the premium bundle, pay for the basic one, or somewhere in between (pay for some of the premium features). The problem arises when I want to decide/suggest features to add to the product. I'm being asked how will it affect the sales of the product - and this question is terribly difficult.. sure I can measure the usage of the feature I'd like to add - but it might not affect the bottom line $$$ at all. I can also find examples of features that nobody uses but did improve sales because of their effect on the positioning of the product.
How do you measure effect of a feature which is part of a much larger product? How can I measure if it was significant in the customer decision to buy / renew? any examples? In an organization that size there should be a whole team of internal experts as well as independent, external customer and product research resources devoted to understanding customer needs and desires. Those needs should be researched and matched up to projected solutions within your product. Those high level solutions should be articulated in a feature roadmap. If appropriate product research has been conducted you will have explored the problem and potential features in depth with customer interviews, focus groups and broader survey customer research to gauge impact on customer retention, license upgrades and then extended that research to understand impact on acquiring new customers. As an engineer with a feature idea find a way to get it on the radar of that product research team so they can see about validating it with customers. Having been a professional researcher doing exactly that kind of validation at one point, and now doing a fair amount of development, I have come to believe that building a sandboxed prototype is better than just trying to describe the conceptualized feature. But be careful not to invest too much, time, money or emotional commitment in the prototype. Get it in front of customers for reaction as soon as possible. If you want more examples and stories feel free to reach out. Thanks!
You know what, I think I wasn't accurate enough with what I'm asking. In our org when I decide on a feature I know to say that customers need and want that feature. What I don't know is how to /measure/ it even after it is generally available.
For example:
Lots of customers say "yes I need X, if you build this that'd be great". So we go and build X. Release it. Customers use it. Now the question is - how do I know it helped increase sales? If sales didn't grow more than it usually does (by inertia) does that mean feature X did not contribute at all? You are correct that this is very difficult to untangle. But remember features that initially excite and please customers because they were new and provided even unexpected benefits, soon migrate into "more is better" kinds of features and then over time every competitor is offering the same feature and it is "table stakes." You really only expect sales upticks for features relatively early in that lifecycle. The rest just contribute to customer retention. You should assume that most businesses already have some kind of solution in place, either from a competitor or an internally developed a set of processes and supporting software. When you look for sales bumps you need to consider the length of the sales cycle - you might not see any switching for 2 or 3 quarters after it becomes generally known that the feature is available. I think your surest sign is in customer retention statistics. If normally you experience 10% non-renewals of license every quarter. I would look in the quarters right after it is introduced to see whether renewal rates improve. And, still your critics might say well it wasn't the new feature that your department created and introduced, it was the new marketing campaign and we revamped our process for renewing customers. And, you know they might be right. Or, the improvement is a subtle combination of all those factors. Unlike science, businesses, particularly B2B, rarely do controlled experiments. You can track inputs and outcomes over time and point to correlations but that is about the best you can do. Thanks!
I was thinking perhaps surveying customers after purchase/renewal to tick multiple selection boxes and say which features they liked which contributed to their decision. Still it can be a combination with the new marketing campaign but at least I will know that feature X is one of the reasons they decided for the product. I suspect that you are not the person who decides which feature will be implemented? So first I would try to understand how the management or a product owner normally decides which feature is next and which KPIs are used to make a decision. And beside that I would listen to the needs of my clients (I know that this sounds a bit too easy). Are they happy with the product? Do they wish something else? Is there a platform for enhancement requests etc. where you can match your idea or just talk to the sales-people as they normally know what clients miss in features. I am a middle manager, so have quite an influence but not ultimate decision. I do have to justify projects to my managers. The KPI is clear, we are trying to increase the sales. But the product is so large that it is extremely difficult to measure the contribution of a single feature.
Also, the client base is very large and as I mentioned thousands of sales persons so no matter who I ask I can't get clear answers. That's why I'm trying to find a measurement that somehow averages the answers.