Common wisdom is that enforcing 100% code coverage is a bad idea, but I am arguing you should do it anyway, especially if you are using python. You would be simply shipping better software with if you enforced a 100% coverage.
Code coverage as a metric is prevalent in software engineering. It is used by teams to measure how much of their code is covered. There are two main kinds - statement coverage and branch coverage, statement coverage measures how many statements are tested during a CI run. Branch coverage measures how many branches are covered during a CI run. Each conditional / loop / switch statement is a branch. You can read more about it here - https://en.wikipedia.org/wiki/Code_coverage
Firstly, let us get the criticisms of a full code coverage out of the way.
1. The most common criticism of requiring a 100% code coverage (and i am not even talking about 100% branch coverage) is that it will lead to bad quality test cases, or an excessive use of mocks. You would be forced a bunch of mocks, which are most certainly bad.
2. It also, does not tell whether the code is bug free or not. A test can only show the existence of a bug, and not the non-existence of the bugs.
3. Management gets a fake sense of security with 100% coverage – they are happy. But reality is fully covered code is still full of bugs.
So, 100% coverage is evil, but you should do it anyway. Here is why…
Any good developer would agree that some coverage is required. Tests help you avoid regression and increase confidence in your software. Most likely in these tests, people would just be covering the happy paths, which is what your customers hit most of the times, but the edge cases are the ones which cause your application to crash, and nobody likes a crashing software. Most of the dev shops would be happy with a 80-90% coverage, but who takes a call which lines to skip in testing? Once you start skipping one case, it is a slippery slope, you end up skipping more lines as the project grows.
1. Writing a test is hard for that case. Why is it hard when you can mock everything anyway? Having a bad test is better than having no tests at all. This is particularly true if you are using python. You have no idea what that line does if the line is not tested. Even if it does not assert anything, it guarantees that the line of code does not throw an exception, which is something.
2. You want to test only the happy paths. Sorry, then you will end up with an unstable path. If you love the craft of building good software, you care about edge cases more than the happy paths.
3. The code is already passing the threshold set in the CI. If that threshold is set to 0, would any developer bother writing any tests? Not writing the test is just being lazy.
4. You are suddenly concerned about the quality of the tests. That to achieve a 100% code coverage you end up writing tests which only test the implementation and not the behaviour. To that I ask you what about the quality of the software? You care so much about test quality, but not software quality? This is just lazy, as it requires you to work.
All of these are bad reasons to not write tests for the statement being missed.
Why python specifically requires a high coverage?
It is dynamically typed, there are no checks at compile times.
If you are working on a production python code, I am sure your sentry is replete with errors saying “<> object has no attribute xyz”.
100% coverage does not make a perfect software
All this is not to say 100% coverage is a silver bullet, it certainly does not guarantee a bug free software. It doesn’t make a perfect software. But it is a great place to start. What next after 100% statement coverage? Go for 100% branch coverage. A particularly famous example of a software using 100% branch coverage is SQLite. Everyone loves SQLite. Of course they do a lot more testing apart from the branch coverage, running millions of test cases per day, with various different mechanisms. You should read this - https://sqlite.org/testing.html for inspiration. What after 100% branch coverage? Go for 100% MC/DC coverage.
Ultimately, it is about the process of reaching to 100% coverage, you would find dead code, unnecessary logic which is never possible to reach. While testing one part of the code, you would find issues in another part of code.
A 100% coverage is also a mandatory requirement for various critical applications, where lives depend on the code being correct. Like an aircraft. Would you fly on an aircraft with only the happy paths tested? I think not. Then why subject your uses to a less tested software? You may argue that your CRUD app is not safety critical, but then my argument is that stability is a feature, which your project will benefit off. In a world where a lot of code is being spit out by LLMs, stability is a differentiator. How to make things stable? Write some tests, and ensure 100% coverage.
Convinced of 100% code coverage but worried where to start?
Ok now that you are convinced that 100% code coverage is required, but the reality is that your current codebase is not at 100%, in such a situation, you should ensure that the number of untested lines of code is reducing at every PR. Tools like coverage (for python), have an API which provides the number of untested lines of code. It can be checked in CI that all new lines must have been tested. Other languages also have libraries which give similar outputs. This would help you incrementally getting coverage.
This is better than keeping a fixed %ge as a coverage requirement. As an example, covering a 90% coverage, guarantees that the number of lines not tested increases as your project becomes bigger and successful. It is exactly then, that you need better coverage.
Your enemy in implementing the above would be existing tests which rely on random numbers, and hence the tested part of the code changes every time you run CI. Such tests are a roadblock in reaching a full coverage, and hence must be found and eliminated, to cover each scenario which is generated by using the random numbers.
Some teammates would be pained by asking 100% coverage, they would come up with reasons which you can counter using this post. Enforce this in CI, and create a well crafted software.