When we have a large scale disaster from a man made system, usually we have had multiple things go wrong. Chernobyl had multiple failures, so did the recent issues in Japan at their nuclear facility. It seems to me that in many cases when Mr. Murphy has struck in my career, he has done so multiple times.
Recently I was studying some high availability (HA) scenarios. In a mirroring situation the implication of a long mirroring queue was shown dramatically as a the primary server failed and the company experienced data loss in a failover to the secondary server. The data was eventually recovered by bringing up the primary server, extracting the missing data, and then manually merging it back into the system.
Incidents like that are never fun to deal with, but are they common? It seems that the old adage about "when it rains it pours" has happened more than a few times in my career, but for this Friday I wanted to see if that has been the case for the rest of you.
How many of you have had multiple failures at the same time?
A RAID drive fails and shortly thereafter a network card dies. You find corruption and while working on that, the server hardware crashes. Have any of you had two or more failures occur on the same system at almost the same time?
Let us know this Friday, along with any lessons learned. Have you changed the way that you prepare for disasters or plan for issues in your career? I'm sure there are many people that would love to learn those lessons from a post rather than experiencing them first hand.
The Voice of the DBA Podcasts
The podcast feeds are available at sqlservercentral.mevio.com. You can also follow Steve Jones on Twitter:
Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com. They have a great version of Message in a Bottle if you want to check it out.
I really appreciate and value feedback on the podcasts. Let us know what you like, don't like, or even send in ideas for the show. If you'd like to comment, post something here. The boss will be sure to read it.