July 23, 2015 at 6:09 am
Shouldn't a better question be, can you write code perfectly on the first try 100% of the time?
If the answer is anything other than a definitely yes then yes you damn well better unit your code. Some of the worst developers I've worked with were people who thought the people doing QA testing were their personal servants and just submitted any code(or development work for that matter) that would compile without even doing a basic test of it themselves. Oddly enough these developers tend to be the ones who don't understand why not following a migration chain from dev -> test -> prod is a bad idea.....
July 23, 2015 at 6:13 am
ZZartin (7/23/2015)
Shouldn't a better question be, can you write code perfectly on the first try 100% of the time?If the answer is anything other than a definitely yes then yes you damn well better unit your code. Some of the worst developers I've worked with were people who thought the people doing QA testing were their personal servants and just submitted any code(or development work for that matter) that would compile without even doing a basic test of it themselves. Oddly enough these developers tend to be the ones who don't understand why not following a migration chain from dev -> test -> prod is a bad idea.....
A unit test is also code + a lot more, thus that logic defeats itself.
Don't get me wrong, there are plenty of developers that I rather would not see develop anything at all, but that does not validate unit testing as a way to always do things.
And I am all in favor of testing and doing so early on in the process.
But unit tests fail on many levels in what it is claimed to achieve. Developers that deliver broken code will not suddenly deliver working code or come to a fault free solution any faster.
More and/or more complex code does not lower the prolifically of errors.
It does however make development extra costly and in ways also inflexible!
July 23, 2015 at 6:29 am
peter-757102 (7/23/2015)
ZZartin (7/23/2015)
Shouldn't a better question be, can you write code perfectly on the first try 100% of the time?If the answer is anything other than a definitely yes then yes you damn well better unit your code. Some of the worst developers I've worked with were people who thought the people doing QA testing were their personal servants and just submitted any code(or development work for that matter) that would compile without even doing a basic test of it themselves. Oddly enough these developers tend to be the ones who don't understand why not following a migration chain from dev -> test -> prod is a bad idea.....
A unit test is also code + a lot more, thus that logic defeats itself.
Don't get me wrong, there are plenty of developers that I rather would not see develop anything at all, but that does not validate unit testing as a way to always do things.
And I am all in favor of testing and doing so early on in the process.
But unit tests fail on many levels in what it is claimed to achieve. Developers that deliver broken code will not suddenly deliver working code or come to a fault free solution any faster.
More and/or more complex code does not lower the prolifically of errors.
It does however make development extra costly and in ways also inflexible!
Unit testing to a developer doesn't mean doing a full regression test of the entire application, that is what QA testers are actually for. It does mean that if a developer does something they should at least do some rudimentary testing to validate that it works.
For example if someone modifies a stored procedure and adds some new flow control logic it's really not asking to much for them to take 10 minutes of time to run it with a few sets of input parameters and verify the results basically look right and are following the right paths.
Another part of this equation is not trying to make one giant component that does everything, yes that will naturally make any attempts at unit testing harder, of course it also makes any development in general harder....
July 23, 2015 at 6:55 am
ZZartin (7/23/2015)
peter-757102 (7/23/2015)
ZZartin (7/23/2015)
Shouldn't a better question be, can you write code perfectly on the first try 100% of the time?If the answer is anything other than a definitely yes then yes you damn well better unit your code. Some of the worst developers I've worked with were people who thought the people doing QA testing were their personal servants and just submitted any code(or development work for that matter) that would compile without even doing a basic test of it themselves. Oddly enough these developers tend to be the ones who don't understand why not following a migration chain from dev -> test -> prod is a bad idea.....
A unit test is also code + a lot more, thus that logic defeats itself.
Don't get me wrong, there are plenty of developers that I rather would not see develop anything at all, but that does not validate unit testing as a way to always do things.
And I am all in favor of testing and doing so early on in the process.
But unit tests fail on many levels in what it is claimed to achieve. Developers that deliver broken code will not suddenly deliver working code or come to a fault free solution any faster.
More and/or more complex code does not lower the prolifically of errors.
It does however make development extra costly and in ways also inflexible!
Unit testing to a developer doesn't mean doing a full regression test of the entire application, that is what QA testers are actually for. It does mean that if a developer does something they should at least do some rudimentary testing to validate that it works.
For example if someone modifies a stored procedure and adds some new flow control logic it's really not asking to much for them to take 10 minutes of time to run it with a few sets of input parameters and verify the results basically look right and are following the right paths.
Another part of this equation is not trying to make one giant component that does everything, yes that will naturally make any attempts at unit testing harder, of course it also makes any development in general harder....
Quite true, unit- and other forms of testing have been used here as if it were one and the same thing.
Running tests to see if the code works as intended is common sense, even part of coding itself and has always been since the first computers. There are many ways to do it, but the "modern" blanket pursuit to make everything unit testable is madness gone mad.
July 23, 2015 at 9:54 am
Running tests to see if the code works as intended is common sense, even part of coding itself and has always been since the first computers.
Indeed, so why not save those tests in case someone in the future needs to see what was intended? They could be automated tests that check single assumptions about the behavior of a single unit of work in your code. We could call it something catchy like a "unit test".
Peter, I don't mean to be snarky (well, maybe a little ;-)), but I feel like you're arguing for the sake of argument, or you don't completely grasp the concept. You also jump to pretty dismal conclusions about people who choose to use unit testing. This, after admitting that testing your code is common sense and part of coding itself.
July 23, 2015 at 10:24 am
as a DBA who specialises in helping test professionals I see unit tests as a valuable cost saving exercise for the business
if you look at the ISTQB data for cost of fixing a defect
http://istqbexamcertification.com/what-is-the-cost-of-defects-in-software-testing/
you can clearly see that unit testing has real value by identifying defects early. - couple that with the V model though and really you need to be writing tests at the same time as your requirements are defined. 🙂 that way your business analysts/product owners don't introduce defects too
MVDBA
July 23, 2015 at 11:14 am
cgeswein (7/23/2015)
Running tests to see if the code works as intended is common sense, even part of coding itself and has always been since the first computers.
Indeed, so why not save those tests in case someone in the future needs to see what was intended? They could be automated tests that check single assumptions about the behavior of a single unit of work in your code. We could call it something catchy like a "unit test".
Peter, I don't mean to be snarky (well, maybe a little ;-)), but I feel like you're arguing for the sake of argument, or you don't completely grasp the concept. You also jump to pretty dismal conclusions about people who choose to use unit testing. This, after admitting that testing your code is common sense and part of coding itself.
Testing and unit testing are very different things and in some ways deliver the opposite is my experience. Spending time doing conscious testing is more effective then spending time fleshing out formal unit tests. It is a skill one has to develop over time, hand in hand with programming itself and of course debugging (both active and passive).
From the people I met, those that are geared toward making unit tests the most, deliver the worst results. Mostly due to not understanding what they are making early on. But also due to not being assertive in how they work towards a solution.
The reasoning that it's a small step from testing to unit tests I cannot subscribe to.
Some factors that are needed to make the step are:
* Persisting and organizing tests
* Making tests useful to others later
* Making them part of a larger framework
* Change them to a form that can be automated
Each of these has its own list of requirements, which combined affects a lot. It adds complexity and limits what architecturally can work. Virtually everything must become that nail to hammer down. Let alone the efforts that go into it that can be used for testing instead.
Catching errors in small units of code really does not justify the effort in the vast majority of cases. I can make a simple function like "left", assume not part of an open library, and to cover all possible flows. It will need like 20+ tests for it to get reasonably tested and capture its intended behavior at a minimum.
No-one in their proper mind does that on a daily basis as part of their work. Most will do a few tests to satisfy the requirement of using unit tests and that is it. These tests are then incomplete, trivial, not descriptive and do not help with catching hard to detect or solve bugs. But need to be maintained and taken into consideration from the moment they exist.
For legitimate complex bodies of code, large or small, unit testing is relevant. But that is what exactly supporters always try to avoid to begin with. And breaking legitimate complex code down can make the code loose coherent meaning (making it harder to maintain and identify problems before they happen).
July 23, 2015 at 11:38 am
peter-757102 (7/23/2015) solution.
The reasoning that it's a small step from testing to unit tests I cannot subscribe to.
Some factors that are needed to make the step are:
* Persisting and organizing tests
* Making tests useful to others later
* Making them part of a larger framework
* Change them to a form that can be automated
How are any of those things needed to unit test code? Those might be part of the process your company has for official sign off on testing, but unit testing can be as simple as running a function a couple times and looking at the results.
Those sounds like using excessively long and usually unnecessary processes as an excuse to avoid doing something that in the vast majority of cases is very very helpful.
July 23, 2015 at 11:50 am
ZZartin (7/23/2015)
peter-757102 (7/23/2015) solution.
The reasoning that it's a small step from testing to unit tests I cannot subscribe to.
Some factors that are needed to make the step are:
* Persisting and organizing tests
* Making tests useful to others later
* Making them part of a larger framework
* Change them to a form that can be automated
How are any of those things needed to unit test code? Those might be part of the process your company has for official sign off on testing, but unit testing can be as simple as running a function a couple times and looking at the results.
Those sounds like using excessively long and usually unnecessary processes as an excuse to avoid doing something that in the vast majority of cases is very very helpful.
Read the text I responded to, those points are in fact what others understand as what unit tests represent. Actively testing a piece of written code, while always a unit, is not what people mean by unit testing.
It has to be repeatable, preferably automated and also work for others. There are products, plugins, libraries that help with all this. But the fundamental project complexity and flaws remain.
July 23, 2015 at 11:57 am
From the people I met, those that are geared toward making unit tests the most, deliver the worst results. Mostly due to not understanding what they are making early on. But also due to not being assertive in how they work towards a solution.
This seems like a more fundamental problem with the people you've met than with unit testing in general.
The reasoning that it's a small step from testing to unit tests I cannot subscribe to.
Some factors that are needed to make the step are:
* Persisting and organizing tests
* Making tests useful to others later
* Making them part of a larger framework
* Change them to a form that can be automated
Like others have stated, you should check out tSQLt when you have time. I'm not saying it's a trivial step, but probably a smaller step than you realize.
It adds complexity and limits what architecturally can work.
I agree that it adds complexity, but I'm not sure what you mean when you say it limits what can architecturally work. All the same things are possible with or without unit testing. I think you're grasping at straws here.
Catching errors in small units of code really does not justify the effort in the vast majority of cases. I can make a simple function like "left", assume not part of an open library, and to cover all possible flows. It will need like 20+ tests for it to get reasonably tested and capture its intended behavior at a minimum.
Maybe, maybe not. Only you (or your employer) can be the judge of what justifies your time and effort. That doesn't render unit testing completely useless simply because you don't feel like it's worth your time.
No-one in their proper mind does that on a daily basis as part of their work.
Easy now, that sounds a lot like an ad hominem attack.
Most will do a few tests to satisfy the requirement of using unit tests and that is it. These tests are then incomplete, trivial, not descriptive and do not help with catching hard to detect or solve bugs.
Again, this sounds like an issue with the people you've experienced and not an issue with unit testing.
For legitimate complex bodies of code, large or small, unit testing is relevant. But that is what exactly supporters always try to avoid to begin with. And breaking legitimate complex code down can make the code loose coherent meaning (making it harder to maintain and identify problems before they happen).
I think you're still confused about what unit testing is trying to accomplish and what it's supporters goals are. Testing units of complex code does not break down the code itself, nor does it make it less coherent or less legitimate for that matter. I have no idea what you've been exposed to that gave you this opinion, but I have a feeling it's those other people you met and not unit testing as a concept.
July 23, 2015 at 11:59 am
ZZartin (7/23/2015)
peter-757102 (7/23/2015) solution.
The reasoning that it's a small step from testing to unit tests I cannot subscribe to.
Some factors that are needed to make the step are:
* Persisting and organizing tests
* Making tests useful to others later
* Making them part of a larger framework
* Change them to a form that can be automated
How are any of those things needed to unit test code? Those might be part of the process your company has for official sign off on testing, but unit testing can be as simple as running a function a couple times and looking at the results.
Those sounds like using excessively long and usually unnecessary processes as an excuse to avoid doing something that in the vast majority of cases is very very helpful.
Well then we're just arguing semantic differences between debugging, testing, unit testing and validation testing 😀
All of which vary based on department, job description and relationship between who you're communicating with.
July 23, 2015 at 12:37 pm
Ed Elliott (7/23/2015)
Thank you!I guess what I was trying to get across is that it often seems quicker to go and just fix something but it always ends up taking longer in the end whereas an early trade off of doubling the time or more to do a fix is always worth it.
ed
Great article, Ed. And, I've found the comment above to be incredibly accurate. What I've personally found at the companies that I've worked at is that unit testing and a peer review process typically saves 8 times (or more as complexity of the code increases) the actual development time. This is particularly true in "utility code" that will be used in many areas regardless of complexity.
I've also found that, once a team has gotten into the habit of unit testing, it usually doesn't increase development time. In fact, we've found that it will, many times, actually save on development time because the processes that we've setup for unit testing have actually streamlined the development process.
--Jeff Moden
Change is inevitable... Change for the better is not.
July 23, 2015 at 1:32 pm
I agree that it adds complexity, but I'm not sure what you mean when you say it limits what can architecturally work. All the same things are possible with or without unit testing. I think you're grasping at straws here.
I was referring to C#, where for unit testing to be doable everything has to use interfaces and be non-static. When largely generating a data access layer with some basic stateless functional code, it can not be eliminated trough stubs in testable routines. Things generally get really messy when routines depend on other routines, especially those of the I/O kind.
Maybe, maybe not. Only you (or your employer) can be the judge of what justifies your time and effort. That doesn't render unit testing completely useless simply because you don't feel like it's worth your time.
I claim there are better ways to spend the same amount of time towards quality. The same time can be used to do actual testing by the programmer during coding, and even better during modeling and design.
Easy now, that sounds a lot like an ad hominem attack.
Not mend that way. It was rather obvious that such extreme unit testing benefits nothing but the very rarest of cases in common code. There are likely fields such as library construction and driver / OS development where it is justifiable. The amount of developers as a percentage where this applies to is likely less then 1% on the whole.
Again, this sounds like an issue with the people you've experienced and not an issue with unit testing.
See the comment above as to why I say it.
I think you're still confused about what unit testing is trying to accomplish and what it's supporters goals are. Testing units of complex code does not break down the code itself, nor does it make it less coherent or less legitimate for that matter. I have no idea what you've been exposed to that gave you this opinion, but I have a feeling it's those other people you met and not unit testing as a concept.
But breaking down code to create smaller units can have this effect.
And which testing being easier on smaller units, it is what people work towards, some to it's extreme.
Everything I read about the subject and my interactions with those with formal training on the matter state that unit tests have to test no other code then what is in the unit body itself. Calls to code outside this scope need to be simulated with instant results. I have a major argument against this as it is dangerous. It can and will hide errors when output assumptions of calls are wrongly simulated in unit tests. Not to mention it is far more time consuming to write and has very negative architectural impacts. It's the all green-lights in the tests, but the rocket falls out of the sky anyway kind of thing!
Closing
People following this threat might have gotten the wrong impression from my stance on the subject and think just because I am not strongly in favor of unit tests. I must be someone that make lots of coding errors. The fact is actually the opposite and the people I work and worked for, know me for it.
I do test more then thoroughly, but effectively, line by line if i have to. When I see suspect code from anyone, I test it then and there and leave a comment with my findings or fix it plus leaving a comment as to why the code was changed.
Dealing with many possible input/output scenarios is easier with frequent code walk-troughs during coding using a debugger and see what actually happens, then with formalized tests. With the first you can see directly anything that does not go as expected and that...is early! This leads to higher quality to begin with. My message to take home is that quality results from conscious effort and certain skills, not simply from automation. It can help, but it also has issues of its own that does not always make sense to engage in.
July 23, 2015 at 1:39 pm
Jeff Moden (7/23/2015)
Ed Elliott (7/23/2015)
Thank you!I guess what I was trying to get across is that it often seems quicker to go and just fix something but it always ends up taking longer in the end whereas an early trade off of doubling the time or more to do a fix is always worth it.
ed
Great article, Ed. And, I've found the comment above to be incredibly accurate. What I've personally found at the companies that I've worked at is that unit testing and a peer review process typically saves 8 times (or more as complexity of the code increases) the actual development time. This is particularly true in "utility code" that will be used in many areas regardless of complexity.
I've also found that, once a team has gotten into the habit of unit testing, it usually doesn't increase development time. In fact, we've found that it will, many times, actually save on development time because the processes that we've setup for unit testing have actually streamlined the development process.
Thanks Jeff I really appreciate your kind words.
I agree that it saves time and the benefits are more than just making sure you don't have bugs (i.e. being able to quickly make changes without unknowingly breaking something else)
July 23, 2015 at 2:32 pm
I was referring to C#, where for unit testing to be doable everything has to use interfaces and be non-static.
Oh, well I thought we were talking about T-SQL. You may be right in that regard. I don't know enough C# to make an intelligent statement.
Calls to code outside this scope need to be simulated with instant results. I have a major argument against this as it is dangerous. It can and will hide errors when output assumptions of calls are wrongly simulated in unit tests.
That is because the calls to code outside the scope should also have tests around them to ensure that they are giving predictable results. If you are certain that the code you are calling is giving predictable results, you can test your code with the predictable result you are supposed to be receiving. Bad code and bad tests will be a problem regardless of whether you use unit testing or not. Also, keep in mind that this isn't a replacement for integration testing, but another step in the process.
Not to mention it is far more time consuming to write and has very negative architectural impacts. It's the all green-lights in the tests, but the rocket falls out of the sky anyway kind of thing!
I won't argue with you there, it definitely is more time consuming up front. In regard to architectural impacts, I'll assume you're talking about C# again and defer to your judgement.
People following this threat might have gotten the wrong impression from my stance on the subject and think just because I am not strongly in favor of unit tests. I must be someone that make lots of coding errors. The fact is actually the opposite and the people I work and worked for, know me for it.
I've not made any negative assumptions about you. I think you have some misconceptions about unit testing, but I don't believe that makes you unintelligent or unfit to code simply because you have a differing opinion on this one matter. Hopefully you see this as a friendly discussion/argument over unit testing and not an attack to your abilities.
My message to take home is that quality results from conscious effort and certain skills, not simply from automation. It can help, but it also has issues of its own that does not always make sense to engage in.
I completely agree with you that quality results from conscious effort and skill...but then you can automate it. The exact same way you debugged one line in your code, you can save that for later to see if changes somebody made may have affected the code to perform in a way that you weren't expecting...across not just one but all stored procedures for instance. You're right, it's not perfect, but it's a good start and it can definitely help.
Viewing 15 posts - 16 through 30 (of 86 total)
You must be logged in to reply to this topic. Login to reply