May 19, 2016 at 11:59 pm
Comments posted to this topic are about the item What's Your Test Plan?
May 20, 2016 at 1:33 am
It would really be nice to have some documentation that spans versions because I am now in SQL 2012 and wanted to upgrade to 2014 but decided against it when 2016's release is imminent. When our company decided to upgrade to 2012 from 2008 I did not have much time to read up on 2012 so I just did the best I could. The boss decided we must upgrade and I had no say. Maybe I missed it through my hasty reading of documentation but all of a sudden database mail was failing left right and centre and we made very good use of database mail. After many discussions with some experts I found a post on a forum where Microsoft said they are aware of the problem and we must wait for a service pack release. It is now four years and still database mail is working on and off.
Why do I say this? Well, good reading up of documentation before you upgrade and good documentation that will show the pros and the cons of upgrading. I am not so sure that the cons are equally well documented. :w00t::w00t::w00t::w00t::w00t:
Manie Verster
Developer
Johannesburg
South Africa
I am happy because I choose to be happy.
I just love my job!!!
May 20, 2016 at 3:34 am
Hello,
I am normally an Oracle guy, but even from that perspective I would have the same approach with SQL-Server. Yes, document everything you do when doing an upgrade. The documentation must be part of any change control system if you have one in place. You document not only for other people but also for yourself when you have to repeat the process over a period of many years.
I just recently finalized one upgrade for the Oracle Forms11 Middleware applications and I had both the installation and test process documented. For the latter I had to go to all my colleagues (project manager, programmer) to make a list of application functions to test. Some may find it stupid to list even the tiniest click and command, but when running an upgrade the idea is that you don't want to spend time going through your mind what you missed and what not. In case of any errors you will be able to pinpoint the exact part of the procedure you missed. Overall my test procedure is about 20-30 pages with screen-shots. Even if it had 100 pages it would be worth while, not that it has to be printed. I already had that kind of document partially ready because of our moves around disaster recovery procedures, in order for the system to work at the DR site we not only have to start the database and middle tier, but also test the functionality, especally with sending emails from an application where the SMTP relay settings have to be tested. The last thing to mention is that you will feel more secure knowing exactly how your application works and what are the relations between all the bits and pieces.
Unfortunatley SQL-Server upgrades at my place have not been handed over to me and some applications are third party. Some people in my department just do an upgrade and ask users to test. Rather a very unprofessional approach and I usually try not to comment on that kind of work. When asked for help I would usually first ask "Where was the process documented?".
May 20, 2016 at 6:47 am
When you have a lot of third party stuff (as we do) there's a real black box problem. It's hard to even know if you've tested adequately.
This is one reason why we are consistently several versions behind on SQL. We can't upgrade SQL until EVERY third party application is upgraded (with some of them that involves substantial license costs as well as extensive retraining and disruption costs). Each 3rd party app is owned by different groups with their own issues, priorities and schedules.
...
-- FORTRAN manual for Xerox Computers --
May 20, 2016 at 7:47 am
I am really following you here, Steve. I deem an instance/database/task "important" if smelly things slide my way if it doesn't work as expected.
I wish I had time to test these, but I don't; so in reality we don't test. I promise myself every time to do better than that - but reality and the bosses don't get the memo.
The good part: We have very little which is "important". And the truly important stuff is outsourced.
But even that, my test plan would be exactly like yours: Having a list of historic tickets, and check they still works after an upgrade.
Currently, we are upgrading from 2005 to 2014 in one go. It works nicely, mainly because I spent the better half of a month Googling and planning, and everything is done by scripting so I can go back and see exactly what I did.
It looks like I haven't missed anything crucial (knock-on-wood). But the important part: we have performed the upgrade on a new server (upgraded the OS while we were at it), so we have the luxury of running in parallel for as long as we need, thus enabling "a controlled (i.e. limited) user test", one important task at a time, and a quick fall-back, should anything be too odd to live with. Of course this is expensive in virtual resources and license, but still cheaper than hiring a test-person or engage a consultancy agency. Boss's choice. 😛
May 20, 2016 at 7:55 am
Hopefully you have a dev and test environment you can test this in first. That's where any major issues should shake out.
But the most important things are to check for any deprecated features that you may be using, that code will need to be rewritten, and any "weird" setting you had to tweak to get your application to work. New features usually aren't an important part of the migration process itself even if they were the motivation for the upgrade in the first place. Third party applications are indeed a huge wild card, the testing process for those could be as simple as does the application connect to retesting the entire thing.
Most of the issues I've seen come up from migrating have to do with that migration also including either the upgrade going to a new server or a new instance on the same server and weird settings that get affected by that on the sys admin side.
May 20, 2016 at 12:02 pm
hjp (5/20/2016)
Currently, we are upgrading from 2005 to 2014 in one go. It works nicely, mainly because I spent the better half of a month Googling and planning, and everything is done by scripting so I can go back and see exactly what I did.
If you want to write some of this up, maybe with a checklist, I'm sure others would love it.
May 20, 2016 at 6:20 pm
We recently hit a nasty upgrading some clients from 2008 to 2012, full text search! We have hundreds of clients (hosted or otherwise) some of whom use full test search for PDF documents, we're currently faced with maintaining them on 2008 linked servers only for full text search of PDF. Neither Microsoft nor Adobe (the PDF filter provider) have come up with a solution.
May 22, 2016 at 9:54 am
Our most important systems are running on Sybase. We do test thoroughly every time we are migrating to a new version. (We never upgrade our database servers, we're migrating the databases to a new server running the new version. That way it's much easier to go back to the old version if something isn't working as intended).
May 23, 2016 at 6:54 am
We're on 2008R2 and don't have plans yet for upgrading. But we will have to eventually.
January 17, 2017 at 1:20 am
Every application/system should have a regression test plan delivered as part of the first delivery. This should also form part of the DR plan. Otherwise how can you know that you have recovered from that disaster?
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
January 17, 2017 at 8:26 am
Most people just seem to assume if the server is up, things are working. This usually leads to constant fixes as still broken items are fixed on the fly. One reason we have poor quality in systems is that it might be worth paying the price when a disaster or deployment strikes more than building a comprehensive test plan up front.
I'd disagree, because lots of our dev tests are one-off hacks rather than formal test methods. This often means we rebuild tests constantly to check something, we don't have repeatable or reliable quality, nor can we apply these tests in places like DR. However, I'd also note that as engineers (DBAs and devs), we tend to try to be perfect, which means that our work on test plans can be far, far too complex and time consuming. We should be tackling test plans a test at a time.
January 17, 2017 at 9:49 am
Steve Jones - SSC Editor - Tuesday, January 17, 2017 8:26 AM... We should be tackling test plans a test at a time.
Yes, in development we create one test at a time. We just happen to create a whole bunch of them over the development leading to each release. 😉
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply