In part 1, I talked about the difference between being correct (meets a spec) and good (meets a need).
So obviously it is better to be both correct and good but there are conflicts between them.
Go carefully
Long release cycles
Exhaustive testing
Culture of perfection
Don’t test at customer sites
Go fast
Fast iterations
Fast testing
Culture of shipping
Must test at customer sites
You can choose for yourself and your company where you want to be on each of these axes. And the right spot to be will change with the life of your product and company.
“Goodness” is something related to the product fit and can grow infinitely, while “correctness” is about how well implemented it is, and tops out at 1.0 (100% correct). I’m going to multiply them together to estimate the total value of a product.1 Using this model explains why startups can move so fast and big companies get slower and slower over time.
You can see that at the big company, they have been working for a long time and have built a lot of goodness and correctness. Adding new goodness is still possible, but each step is a smaller percentage. Even a very small drop in correctness has the chance to wipe out hundreds of good features’ worth of value. Big companies are paranoid about correctness, and rightly so. They have a ton of value to defend.
Startups, on the other hand, don’t have a ton of goodness or correctness. That means that a single new good feature is a much larger percentage improvement to the startup. In addition, the consequence of a drop in correctness is smaller.2
Being more correct encourages things like exhaustive testing, a culture of perfection, compliance review, process driven release, etc. All of those things help protect correctness at the cost of slowing down iteration, because the easiest way to make it hard for things to regress is to make it hard for things to change.
Engineers like correctness. We are people who deal with systems and rules and valuing correctness has been trained into us.
Engineers coming from mature projects to early/experimental ones often try to bring big company best practices. They’ve seen smart people succeeding by doing things a certain way and it feels right to copy them. And maybe they've seen a small drop in correctness cause a huge drop in a big company’s value. But if you are early in discovering the shape of your product, the last thing you want to do is carve it out of stone.
Lastly, if you are not correct enough it is embarrassing. Customers seeing bugs is embarrassing. Shipping code that isn’t quite right is embarrassing. It is very hard to look at your boss and say “Yeah, I shipped a bug to the customer, and I still think I made the right tradeoff between speed and correctness”.
Obviously it is best to get exactly the perfect balance between being correct and good, but what happens when you get it wrong?
If you aren’t correct enough then customers see bugs. If those bugs are bad enough they might fire you, but most of the time the consequence for a startup is embarrassment.
On the other hand if you aren’t good enough then you will find customers who order a small pilot, usually through their innovation team. They will use your robots occasionally, perhaps only when giving tours to VIPs. They won’t expand their deployment or will do so very slowly. Sales will be slow and expensive. You will always be ‘almost there’ in terms of scale. You will have a bunch of unused robots sitting around (that you paid for, couldn’t sell, and are now getting slowly outdated) and you can’t raise more venture capital because you don’t have enough traction and you run out of money and die.
This is how 99% of robotics startups die.
So, while you can die from lack of correctness,3 you are much more likely to die because your product isn’t good enough. That means you want to pick your sweet spot for ‘correct enough’ given your maturity and focus 1000% of the rest of your time and attention on getting good as fast as you can.
Build process and infrastructure to verify, as fast as possible, that you are at your minimum correctness bar.
Get great at release rollout and rollback so you reduce the consequences of shipping bugs.
Get robots into customer hands as fast as possible.
Remove tech debt that slows down iteration cycles.4
Do good product development: (guess → try & observe → iterate)
And if want to keep reading things like this I recommend Part 1 of Correct vs Good, or my article on Tech Debt.