December 17, 2009 at 10:17 am
So as I am finishing up debuging a bit of code, wondering why I did not think of this
particular issue when testing. I started to wonder, is the process of identifing and creating
test situations, data to find and break code a learned skill, or more of a natural talent?
I work with medical claims data, which means a near limitless combination of data, very literally
almost anything can occur. In any given subsection of data there may or may not be one of those
'almost never' situations, or the 'should not happen' situations. The source system may allow the
users to do X in a certain way, but that does not mean it should have been. Leaving data which is
"Right", passing validations, yet it is actually "wrong", from analysis points of view.
What does the board think, is creating testing data a skill which can be learned, and taught, or is it more a natural frame of mind to think of the obscure and unlikely data which could break a process?
after all... it is not likely, but possible for a tribal surgeon to perform a heart transplant in a helicopter on an oil rig in international waters. It is possible, very unlikely, but possible. Such a situation could break any number of reports, analysis tools, programs etc. How far up the fruit tree
should one go in code testing?
Opinions would interest me alot.
December 17, 2009 at 10:49 pm
I think it is a nice combination of both. There is an inborn desire to break things, or take things apart to see how it might work. Teaching people to think outside the box can happen, and thus the skill can be learned.
Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events
December 18, 2009 at 8:11 am
I'm not sure if it's a skill, although it must partly be.
One old story. I wrote an app for managing user access to a database and turned it over to the QA team. I got back the report that a particular function was failing. I tested it, over & over, and couldn't find the problem. Then the QA guy showed me. If you add a user, delete a user, re-add the same user, then delete it again, and continue doing this, after 15 times, there was an error.
I ask you, is it realistic to add & drop the same user 15 times in a row? Of course not. However, if you do it, it caused an error. We went round & round on whether or not that was a valid test. I'm still convinced it was bogus. But, how do you determine? I don't know, I think it's a talent, not a skill, at least in some situations.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
December 18, 2009 at 8:33 am
Grant Fritchey (12/18/2009)
I'm not sure if it's a skill, although it must partly be.One old story. I wrote an app for managing user access to a database and turned it over to the QA team. I got back the report that a particular function was failing. I tested it, over & over, and couldn't find the problem. Then the QA guy showed me. If you add a user, delete a user, re-add the same user, then delete it again, and continue doing this, after 15 times, there was an error.
I ask you, is it realistic to add & drop the same user 15 times in a row? Of course not. However, if you do it, it caused an error. We went round & round on whether or not that was a valid test. I'm still convinced it was bogus. But, how do you determine? I don't know, I think it's a talent, not a skill, at least in some situations.
Curious, why did it error after 15 times?
December 18, 2009 at 8:35 am
Well, to give the classic answer, it depends.
I think that there is some level of skill involved. While you always could be one of the 12,000 monkeys getting lucky early on, the ability to perform some action, look through code and comprehend it, is a skill. It's learned, you've gone through training (self or led) to understand how code works, how the process flows, etc. A tribal doctor could perform a heart surgery, and likely would need some combination of talent and luck, but he would still be a doctor with some skill about working with the body.
The degree to which you success, find issues, observe the forest and the trees is a matter of talent. I've know programmers that were skilled, they could handle some code, but they didn't understand pointers. Or they couldn't wrap their head around how a correlated subquery works. They lack some level of talent in that area.
December 18, 2009 at 8:36 am
Grant Fritchey (12/18/2009)
I ask you, is it realistic to add & drop the same user 15 times in a row? Of course not. However, if you do it, it caused an error. We went round & round on whether or not that was a valid test. I'm still convinced it was bogus. But, how do you determine?
I'd argue it's a valid test, although an extreme edge case that wasn't necessarily worth fixing. Unless this was a leak of some sort that occurred under load (meaning effort and time). In that case it could potentially be an issue over time and would be worth fixing.
December 18, 2009 at 8:36 am
I have to agree, it's both. I can't really explain it, but sometimes you just know (gut feelings, intuition, whatever) and other times it is something learned.
December 18, 2009 at 8:56 am
Steve Jones - Editor (12/18/2009)
Grant Fritchey (12/18/2009)
I ask you, is it realistic to add & drop the same user 15 times in a row? Of course not. However, if you do it, it caused an error. We went round & round on whether or not that was a valid test. I'm still convinced it was bogus. But, how do you determine?I'd argue it's a valid test, although an extreme edge case that wasn't necessarily worth fixing. Unless this was a leak of some sort that occurred under load (meaning effort and time). In that case it could potentially be an issue over time and would be worth fixing.
No, it was weird. I never did figure out what caused it, but if you changed the user name every time, it never generated an error. Odd stuff.
But, I'm still down with, this is a ridiculous test and shows that it's not simply a skill (the tester did great, if totally misguided, work) but a talent because you have to make judgements, not simply measurements.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
December 18, 2009 at 9:01 am
This is all interesting so far! I would dig having a QA group here, I keep trying to sell the need, so far, no go.
Yeah, I would not think to add/delete the same information 15 times in a row, so there is definately a mindset involved.
Ofcourse, as with all things I do get better at finding the likely issues, it is the unlikely ones I need to envision more.
That is one of those depends issues I am sure, just how obscure and unlikely of a situation is feasible, and cost effective on time to find. With my original wild example for instance, it could be a one in ten million chance we would have data like that, does not seem worth the time to think up and test such excessive situations, compared to say, getting a claim for a hospital stay which has the admission date in 2020.
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply