SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Test Coverage


Test Coverage

Author
Message
Revenant
Revenant
SSChampion
SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)

Group: General Forum Members
Points: 12679 Visits: 5010
However more and more testing is being pushed back onto developers to handle, with frameworks like NUnit and JUnit.

I would say that as testing tools are easier and easier to use, developers are expected to do more unit testing. I am expected to have a test class for each sproc, with number of methods generally depending on the number of parameters.

That, however, does not mean that QA will not do their own testing without looking at what I tested and how. (After a while, they do not have to look anyway because they know me and they know how I go about things.)
Matt Miller (4)
Matt Miller (4)
SSC-Dedicated
SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)SSC-Dedicated (30K reputation)

Group: General Forum Members
Points: 30217 Visits: 19009
Eric M Russell (11/12/2013)
Tom Bakerman (11/12/2013)
Jeff Moden (11/12/2013)
From the article:
These days most testing of software is automated.


Oddly enough, I believe that's part of the problem with delivered software, today. There's nothing like a human using the software as the ultimate test because that's the ultimate goal. I've also found that, except for certain types of load testing software, most automated software doesn't even consider performance and a lot of people don't set the test software up for "negative path" testing.


I agree. It takes something special to be a good QA person, and most developers (I include myself) don't have it. Also, while the *Unit test frameworks work well for unit tests, they are still unit test. I believe that QA groups are still needed for the larger integration efforts. I haven't read or heard anything about the test matrix for the new healthcare market place, but I can make some guesses about what it looked like prior to the opening, and how much larger it will be in a couple of months.

QA needs to be performed by a dedicated "QA engineer" using a formalized process and test plan. Simply asking the business analyst, a stake holder, or an intern to famliarize themselves with the requirements documents and then sit down and "poke around" with the application doesn't cut it.


No - "poking around" won't cut it. Test coverage of code and test coverage of scenarios require quantitative analysis to determine progress.

That said - a BA worth their salt should be able to take the requirements documentation, create a basic Ishikawa diagram of common failures, and document happy and sad paths to test. A QA engineer might be skilled in knowing how to CODE said tests into testing suite, but they would have to do the very same analysis to determine which tests to run if a BA was not involved.

----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
Jim P.
Jim P.
SSCommitted
SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)

Group: General Forum Members
Points: 1885 Visits: 2215
Revenant (11/12/2013)
I would say that as testing tools are easier and easier to use, developers are expected to do more unit testing. I am expected to have a test class for each sproc, with number of methods generally depending on the number of parameters.

That, however, does not mean that QA will not do their own testing without looking at what I tested and how. (After a while, they do not have to look anyway because they know me and they know how I go about things.)

The problem is that the DBA developer will work with the parameters given in the spec, using the old subset of the DB that they are allowed to use for testing.

They won't know that the end date is optional or what the limiting scopes will be. So then it goes to production and times out or fails. But they didn't do "sufficient testing" because they designed to the specs.



----------------
Jim P.

A little bit of this and a little byte of that can cause bloatware.
vliet
vliet
Mr or Mrs. 500
Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)

Group: General Forum Members
Points: 505 Visits: 773
Unlike testing applications, where failure is an common sign of some flaws in the code, testing database facilities like stored procedures, user defined functions, views and queries is much harder, because most of the time the result of a code flaw will not be a complete failure but just a little inconsistency in the results. For example, when using an INNER JOIN instead of a LEFT OUTER JOIN to list people with their skills, it might go easily unnoticed that you are missing someone without any registered skill, especially if nearly all of them do have registered skills. Even harder to track down are the small errors introduced in the values of SUMs or COUNTs when someone overlooked the sometimes unexpected behavior of NULLable columnsin JOIN and WHERE clauses.

As a long time developer and DBA, in my humble opinion testing a piece of code should not be the responsibility of the one that implements the feature. No matter how hard you try, you will always have some 'blind spots' and miss some edge cases that someone else will come up with. However, in most smaller organizations the DBA has no colleague to test and review his work, making it even harder to maintain the required code quality standards. I do agree that every part of the application should be tested both separately and in conjunction with other parts. But that is not always feasible without the contribution of testers with other skills then an average developer has in his suitcase.
Gary Varga
Gary Varga
One Orange Chip
One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)

Group: General Forum Members
Points: 27580 Visits: 6554
vliet (11/13/2013)
...For example, when using an INNER JOIN instead of a LEFT OUTER JOIN to list people with their skills, it might go easily unnoticed that you are missing someone without any registered skill, especially if nearly all of them do have registered skills. Even harder to track down are the small errors introduced in the values of SUMs or COUNTs when someone overlooked the sometimes unexpected behavior of NULLable columnsin JOIN and WHERE clauses...


Totally disagree. These are valid test cases that should be considered by the developer. These are similar types of scenarios to division by zero, null references, empty collections etc. They even have a term (or two): Edge Cases/Boundary Testing etc.

Gaz

-- Stop your grinnin' and drop your linen...they're everywhere!!!
Dave Poole
Dave Poole
SSCoach
SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)

Group: General Forum Members
Points: 17068 Visits: 3403
There is a very definite skill in writing a suite of useful tests. You can't approach it as a "check the box" exercise.

Automated testing allows a huge range of repeatable tests to be applied quickly. My first engagement with a professional tester was a humbling experience. I take pride in my work and do put a lot of effort into my code but the tester found gaps I simply hadn't thought of.

The thing is that over time the knowledge of what is required for testing is absorbed and results in a more rich set of automated tests.

I see UAT as separate to QA. UAT should be about the useability of a system rather than a "flush out the bugs".

LinkedIn Profile
www.simple-talk.com
Gary Varga
Gary Varga
One Orange Chip
One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)

Group: General Forum Members
Points: 27580 Visits: 6554
David.Poole (11/14/2013)
...I see UAT as separate to QA. UAT should be about the useability of a system rather than a "flush out the bugs".


I agree but feel that the reality is that other defects will be noted due to occurrences of such things as missed or misunderstood requirements.

Gaz

-- Stop your grinnin' and drop your linen...they're everywhere!!!
Dave Poole
Dave Poole
SSCoach
SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)

Group: General Forum Members
Points: 17068 Visits: 3403
Gary Varga (11/14/2013)

I agree but feel that the reality is that other defects will be noted due to occurrences of such things as missed or misunderstood requirements.


NSR = None Stated Requirements

LinkedIn Profile
www.simple-talk.com
Iwas Bornready
Iwas Bornready
SSC-Insane
SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)SSC-Insane (22K reputation)

Group: General Forum Members
Points: 22692 Visits: 885
Two areas we struggle with in testing. The first is that the data gets changed with testing and is difficult to get back to retest. Second is that there is no one better to test than the end user who doesn't know what is expected and seems to find all the right kinds of input to destroy the programmer's expectations. That doesn't happen for a unit test but for a more "releasable" condition of the product.
Dave Poole
Dave Poole
SSCoach
SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)SSCoach (17K reputation)

Group: General Forum Members
Points: 17068 Visits: 3403
There are so many dimensions to "realistic" test data not least that data has natural hot spots. It's difficult to come up with test data that truly simulate the production environment.

In terms of getting to a repeatable state we are looking to rebuild a baseline after a successful release rather than re-running an every increasing number of scripts with an ever increasing rebuild time.

The discipline we try to adopt is to look at what it is we are trying to achieve and to take a step back and ask what we need to be able to achieve it rather than to bash our heads against a brick wall with a particular solution. In effect OODA - Observe, Orient, Decide, Act.

LinkedIn Profile
www.simple-talk.com
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search