SQL Clone
SQLServerCentral is supported by Redgate
Log in  ::  Register  ::  Not logged in

Don't Use Code Coverage

By Steve Jones,

I got asked in a seminar recently how to perform code coverage for T-SQL. There aren't a lot of options, but there is a project from Ed Elliot called SQLCover that looks to examine how much of your code gets called from a test. There may be other methods, but my question back was why does code coverage matter? There were a few reasons given by various people, but ultimately the original questioner said that their boss wanted a report.

That's fine, and it's a valid reason to try and measure code coverage, but is that an effective use of developer time? I don't think so, especially as trying to meet some code coverage goal is fraught with all sorts of issues. Apart from various good points on Stack Overflow, no shortage of people have blogged about the problems of relying heavily on a code coverage metric. I won't repeat all the arguments, but I'll give a few of my own.

First, someone pointed out that you want to be sure all your code is tested. I disagree with that, especially in many  long dev cycle applications. There are all sorts of enhancements I've seen in many applications that never get used. While you should certainly check them for security issues, if the features aren't being used, remove them. Since most of you don't implement tests for security issues, I'm not sure there's value in telling you to write a unit test for logic or functionality.

The other issue with code coverage is that anyone other than a developer using this metric will put pressure on the developers to write more tests. Some developers may write good tests, but I wouldn't be surprised to have tests written that always pass and aren't checked for correctness. Tests like these are misleading and potentially create more bugs since anyone refactoring code might think their changes haven't caused an issue if the test passes. Bad tests, IMHO, are worse than no tests.

I do think that code coverage can be valuable. If a developer is looking to ensure that some part of their code is being checked, especially in long stored procedures, then code coverage can guide them to a place that might need  testing. Since most code needs more than one test, covering more than one case, code coverage doesn't help. I'd prefer a developer spent more time thinking about how to build better tests (or experimenting) than looking at coverage reports.

And if you need a code coverage report, try my technique for building one

Total article views: 58 | Views in the last 30 days: 1
Related Articles

You Aren't That Interesting

Fundamentally as I get older, I think more and more men are pigs. There is definitely something wr...


Test Coverage

We don't test software well enough. Obviously we don't write high enough quality software, but testi...


Holiday Coverage

It seems that IT people often get stuck working on the holidays, whether being on-call or even deali...


T-SQL Code Coverage in SSDT using the SSDT Dev Pack

Code Coverage What is code coverage? When you write some code and then test it, how sure are y...


Don't Use Code Coverage

Comments posted to this topic are about the item [B]Don't Use Code Coverage[/B] A real code coverage...