August 24, 2014 at 8:04 pm
Comments posted to this topic are about the item Testing is Your Best Investment
August 25, 2014 at 3:43 am
Definitely agree.
Writing tests (and documentation) is also a "simplicity test". If a feature can't be tested in a simple way, it's probably a good moment to rethink and redesign the feature.
August 25, 2014 at 5:31 am
Amen to that.
I am making the switch from TDD to BDD (behaviour driven development) simply because the readability of the tests and output is vitally important in demonstrating software quality to non-technical stakeholders.
As someone with "Big Data" in their job title a large part of the battle is convincing people used to shrink wrapped software and big vendor SLAs that cutting edge stuff can be developed in a reliable manner. Using Behaviour Driven Development as a communication device as well as a quality metric is extremely important.
August 25, 2014 at 7:01 am
After two agonizing weeks of semi-productivity in SSIS on a huge project, I realized my actual job is testing vendors software. (I love the core SQL Server engine, but there's no way that much of the surrounding software (SSIS, SSRS) goes through the same functional testing and usability testing rigor.)
In my code, (mostly custom ETL and utilities), the functionality and users are limited, so tests exist, but they are not as numerous had they gone in end user applications. However my colleagues can use, read and maintain the code and tests. (If they can, then I can, even years later...)
As for using SSIS, the important test is can it be used by my coworkers for this project? Right now it is failing the usability and the functionality tests, so I guess we will be testing the alternatives...
August 25, 2014 at 7:39 am
I love this article. Too often, testing takes a back door and a lot of managers just don't understand how expensive troubleshooting and rework actually is... especially if the fault makes it out to the customer when they can least afford for a fault to appear.
--Jeff Moden
Change is inevitable... Change for the better is not.
August 25, 2014 at 8:25 am
If you want code working and out of the door quickly, there's only one way to do it: first spend enough effort understanding requirements and usability issues, if possible discussing user interfaces with people who will be users, and then make your development and deployment programme inculde thorough testing at every stage: unit testing (including stress testingn of units, checking handling of invalid input data and/or environmental data as well as function testing, throughput testing, and performance testing) is important; system testing must include both black-box testing (tests based on how the software is supposed to work) and white-box testing (tests based on what the software is supposed to do), it must include stress testing, it must include error injection to see if errors can be contained (and ideally recovered without too much impact on the user) and don't cause unlimited damage, it must include testing with workloads and data volumes that exceed requirements to be certain that there is smooth degradation rather than sudden total collapse, it must include testing with bizarre data, and tests must include response times (both for interactive operations and for batch operations); quality assurance testing must be white box testing thoroughly independent of testing that has gone before, it must include stress testing of all types and performance testing of all types as well as error management testing and functional testing, and any failure discovered in QA tests but not earlier should lead to a review of testing plans to try to ensure that that can never happen again. Both System tests and QA tests should have regard to usability and to views expressed by potential end users, so that acceptance trials won't throw up too many unexpected enhancement requests.
If all that is done, there won't be too many big expensive errors on customer sites. There will perhaps be some, as getting it right is difficult, so the software should always be designed to be easily mended as well as being thoroughly tested.
Unforunately it is usually impossible to convince management that doing the job properly form the start will deliver the goods sooner than attempting to do it the cheap way; many managers don't even learn from their failures, but find someone else to blame for the consequences of their decisions to ignore good development and testing practice.
Tom
August 25, 2014 at 2:09 pm
Unit testing is new to me. Where I worked it was the culture not to do it at all. However, since having left there I've been teaching myself new skills, and one of them is unit testing. I have come to really see it's benefits and will use it wherever and whenever I can.
Rod
August 25, 2014 at 5:27 pm
TomThomson (8/25/2014)
If you want code working and out of the door quickly, there's only one way to do it: first spend enough effort understanding requirements and usability issues, if possible discussing user interfaces with people who will be users, and then make your development and deployment programme inculde thorough testing at every stage: unit testing (including stress testingn of units, checking handling of invalid input data and/or environmental data as well as function testing, throughput testing, and performance testing) is important; system testing must include both black-box testing (tests based on how the software is supposed to work) and white-box testing (tests based on what the software is supposed to do), it must include stress testing, it must include error injection to see if errors can be contained (and ideally recovered without too much impact on the user) and don't cause unlimited damage, it must include testing with workloads and data volumes that exceed requirements to be certain that there is smooth degradation rather than sudden total collapse, it must include testing with bizarre data, and tests must include response times (both for interactive operations and for batch operations); quality assurance testing must be white box testing thoroughly independent of testing that has gone before, it must include stress testing of all types and performance testing of all types as well as error management testing and functional testing, and any failure discovered in QA tests but not earlier should lead to a review of testing plans to try to ensure that that can never happen again. Both System tests and QA tests should have regard to usability and to views expressed by potential end users, so that acceptance trials won't throw up too many unexpected enhancement requests.If all that is done, there won't be too many big expensive errors on customer sites. There will perhaps be some, as getting it right is difficult, so the software should always be designed to be easily mended as well as being thoroughly tested.
Unforunately it is usually impossible to convince management that doing the job properly form the start will deliver the goods sooner than attempting to do it the cheap way; many managers don't even learn from their failures, but find someone else to blame for the consequences of their decisions to ignore good development and testing practice.
+1000!
Heh... I remember the conversation we had in a similar vein... If you have to pick amongst doing it Right, Fast, or Cheap, make sure that you choose "Right" because, as you pointed out in that other conversation, there's a very good chance that "Fast" and "Cheap" will come along for the ride.
--Jeff Moden
Change is inevitable... Change for the better is not.
August 26, 2014 at 2:14 am
Jeff Moden (8/25/2014)
TomThomson (8/25/2014)
If you want code working and out of the door quickly, there's only one way to do it: first spend enough effort understanding requirements and usability issues, if possible discussing user interfaces with people who will be users, and then make your development and deployment programme inculde thorough testing at every stage: unit testing (including stress testingn of units, checking handling of invalid input data and/or environmental data as well as function testing, throughput testing, and performance testing) is important; system testing must include both black-box testing (tests based on how the software is supposed to work) and white-box testing (tests based on what the software is supposed to do), it must include stress testing, it must include error injection to see if errors can be contained (and ideally recovered without too much impact on the user) and don't cause unlimited damage, it must include testing with workloads and data volumes that exceed requirements to be certain that there is smooth degradation rather than sudden total collapse, it must include testing with bizarre data, and tests must include response times (both for interactive operations and for batch operations); quality assurance testing must be white box testing thoroughly independent of testing that has gone before, it must include stress testing of all types and performance testing of all types as well as error management testing and functional testing, and any failure discovered in QA tests but not earlier should lead to a review of testing plans to try to ensure that that can never happen again. Both System tests and QA tests should have regard to usability and to views expressed by potential end users, so that acceptance trials won't throw up too many unexpected enhancement requests.If all that is done, there won't be too many big expensive errors on customer sites. There will perhaps be some, as getting it right is difficult, so the software should always be designed to be easily mended as well as being thoroughly tested.
Unforunately it is usually impossible to convince management that doing the job properly form the start will deliver the goods sooner than attempting to do it the cheap way; many managers don't even learn from their failures, but find someone else to blame for the consequences of their decisions to ignore good development and testing practice.
+1000!
Heh... I remember the conversation we had in a similar vein... If you have to pick amongst doing it Right, Fast, or Cheap, make sure that you choose "Right" because, as you pointed out in that other conversation, there's a very good chance that "Fast" and "Cheap" will come along for the ride.
So, so true.
On top of that, by choosing "Fast" and "Cheap" one often has just selected "Delayed" and "Costly". Delayed because someone, somewhere decides that the early shipped software is unworkable, unreliable and/or unmaintainable and costly in terms or maintenance, reputation and/or loss of income.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
August 26, 2014 at 8:55 am
Gary Varga (8/26/2014)
Jeff Moden (8/25/2014)
TomThomson (8/25/2014)
If you want code working and out of the door quickly, there's only one way to do it: first spend enough effort understanding requirements and usability issues, if possible discussing user interfaces with people who will be users, and then make your development and deployment programme inculde thorough testing at every stage: unit testing (including stress testingn of units, checking handling of invalid input data and/or environmental data as well as function testing, throughput testing, and performance testing) is important; system testing must include both black-box testing (tests based on how the software is supposed to work) and white-box testing (tests based on what the software is supposed to do), it must include stress testing, it must include error injection to see if errors can be contained (and ideally recovered without too much impact on the user) and don't cause unlimited damage, it must include testing with workloads and data volumes that exceed requirements to be certain that there is smooth degradation rather than sudden total collapse, it must include testing with bizarre data, and tests must include response times (both for interactive operations and for batch operations); quality assurance testing must be white box testing thoroughly independent of testing that has gone before, it must include stress testing of all types and performance testing of all types as well as error management testing and functional testing, and any failure discovered in QA tests but not earlier should lead to a review of testing plans to try to ensure that that can never happen again. Both System tests and QA tests should have regard to usability and to views expressed by potential end users, so that acceptance trials won't throw up too many unexpected enhancement requests.If all that is done, there won't be too many big expensive errors on customer sites. There will perhaps be some, as getting it right is difficult, so the software should always be designed to be easily mended as well as being thoroughly tested.
Unforunately it is usually impossible to convince management that doing the job properly form the start will deliver the goods sooner than attempting to do it the cheap way; many managers don't even learn from their failures, but find someone else to blame for the consequences of their decisions to ignore good development and testing practice.
+1000!
Heh... I remember the conversation we had in a similar vein... If you have to pick amongst doing it Right, Fast, or Cheap, make sure that you choose "Right" because, as you pointed out in that other conversation, there's a very good chance that "Fast" and "Cheap" will come along for the ride.
So, so true.
On top of that, by choosing "Fast" and "Cheap" one often has just selected "Delayed" and "Costly". Delayed because someone, somewhere decides that the early shipped software is unworkable, unreliable and/or unmaintainable and costly in terms or maintenance, reputation and/or loss of income.
A previous AppDev manager I worked with used a reasonably colorful metaphor from his military background for this scenario. His quote was something to the effect of:
Either FMECA or BOHICA - your call either way.
In short - either plan for the failures you will encounter, or expect some unpleasantness in the delivery.
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
August 26, 2014 at 9:11 am
One thing that I think that gets overlooked is the knock on effect on morale. People get fed up with being blamed for what they predicted would occur and were ignored. They leave. Even if the body of that bright, intelligent, enthusiastic hire stays, eventually all the attributes that they were hired for gets left at the door when they trudge through it.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply