SQL Clone
SQLServerCentral is supported by Redgate
Log in  ::  Register  ::  Not logged in

Testing, Testing, and More Testing

I've been out of pocket and off the forums for the last few weeks as we've had several major tasks where I work with respect to our Active Directory migration. This past weekend involved application migrations. Because of the nature of the migration, we provided test environments prior to "flipping the switch" this weekend. Some application team seem to have taken great advantage of those environments. Those applications came across with no real issues. Others, however, weren't so successful. It wasn't until we flipped the switch in the production environment that they realized parts of their application broke. This caused us to go back and ask a very simple question, "Did you test this (whatever it was that was broken) in the test environments?" The typical response we received was, "We didn't think to do so." That brought up the next question, "These weren't in your test plan?" And we got the expected response, "Unfortunately, no."

Now before anyone goes throwing shoes, fruit, or anything else at these application teams, the fact of the matter is I've seen this before in every other place I've worked. When it comes to testing a system or application, writing a thorough test plan isn't an easy task. It sounds easy enough to do but when you start to write one, especially for a complex application, the reality is that a comprehensive test plan is extremely difficult to create. Sure, the concepts are easy: find out what key business processes the application provides and ensure those work properly. But what is meant by work properly? And how many different cases do you have to test per business process? Do you test them all? Do you test just the major ones? What level is considered satisfactory?

Testing isn't easy. It's time consuming, it's resource draining, and it's very hard to define, at least in the software world, what is a successful test and what isn't. And the problem just isn't limited to the software world. For instance, how do you test a bridge's maximum weight without destroying it? How about an elevator? We see the little sign that says maximum capacity, but how do we know for sure that elevator can take that maximum weight level? Sure, there may be a yearly review by an inspector, but does that maximum capacity get looked at every single time? No, it does not.

With all that said, testing isn't something we scrap because it's difficult. It does require time and resources and any successful project should include time to develop and review the test plans as well as the time it takes to actually use them. Sometimes in the process of testing we determine we forgot things. This was the case when we were looking at our test and implementation plans for the start of the Active Directory migration, when we were tackling the underlying infrastructure. Before we did step 1 in production, we ran through our implementation and test plans in a stand-alone environment. We had a simple philosophy: if we had to make a change to either an implementation or a test plan, we'd run through it in the test environment from scratch again. Because of that design philosophy we had zero issues when we migrated over WINS, DNS, DHCP, and stoop up the Active Directory domains. We had the luxury of doing this because of the nature of our project: Active Directory. You don't want to get that wrong when it comes to infrastructure. Most projects don't have that sort of time.

However, I think an important principle to take from our initial testing is we were ready to modify our test plans as we discovered gaps. We didn't sit on them for a month or two until the implementation was done. We reviewed and changed them immediately while the gaps were still fresh in our minds. This ensured that even though we may not have gotten everything right in the initial test plans or subsequent revisions, we'd have that issue resolved for the next time.

Test plans... no one loves them but most folks understand the need for them.

K. Brian Kelley - Databases, Infrastructure, and Security

IT Security, MySQL, Perl, SQL Server, and Windows technologies.


No comments.

Leave a Comment

Please register or log in to leave a comment.