Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 123»»»

Another Disaster (Almost) Expand / Collapse
Author
Message
Posted Friday, December 20, 2002 12:00 AM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: Moderators
Last Login: Yesterday @ 1:53 PM
Points: 6,705, Visits: 1,684
Comments posted to this topic are about the content posted at http://www.sqlservercentral.com/columnists/awarren/anotherdisasteralmost.asp>http://www.sqlservercentral.com/columnists/awarren/anotherdisasteralmost.asp

Andy
SQLAndy - My Blog!
Connect with me on LinkedIn
Follow me on Twitter
Post #8947
Posted Monday, January 13, 2003 4:10 PM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Thursday, February 03, 2005 3:17 PM
Points: 504, Visits: 1
All I can say is OUCH!

I'm sitting here thinking that I sure better go back over my disaster recovery plan again and make another dry run to make sure I haven't missed anything! This is the type of wake up call we all don't need to have happen but must be prepared for. Excellent job keeping your users up Andy!

Gary Johnson
DBA
Sr. DB Engineer





Gary Johnson
Microsoft Natural Language Group
DBA, Sr. DB Engineer

This posting is provided "AS IS" with no warranties, and confers no rights. The opinions expressed in this post are my own and may not reflect that of my employer.
Post #50150
Posted Monday, January 13, 2003 4:25 PM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: Moderators
Last Login: Yesterday @ 1:53 PM
Points: 6,705, Visits: 1,684
Thanks. We actually sat down today with our vendor to figure out what clustering will cost..another OUCH! I'll probably write up some info on that as well, amazing how quickly the costs add up.

Andy
http://www.sqlservercentral.com/columnists/awarren/




Andy
SQLAndy - My Blog!
Connect with me on LinkedIn
Follow me on Twitter
Post #50151
Posted Tuesday, January 14, 2003 3:18 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: 2 days ago @ 9:43 AM
Points: 1,098, Visits: 6,419
Andy,

Great article. I think this topic is fantastic. Probably one of the best area's on this site, even tho there are only three entries. It's a good insight to other DBA's Disaster Recovery (DR) and it's always interesting to see what extra work you have to do on the fly when the problem falls out of the DR plan scope.

Fortunately (touch wood) we've not experienced anything like that.....but who know's what the future holds...especially as were moving to a new production box....

I guess you can test and test your DR plan, but computers, as wonderfull as they are, are always full of surprises!

Don't wish to curse all the rest of you out there, but any further disasters (or near disasters) would make interesting reading....


Clive Strong
sqlsrvr_dba@hotmail.com
Post #50152
Posted Tuesday, January 14, 2003 6:55 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, September 22, 2008 7:19 AM
Points: 34, Visits: 8
While the content of the article was useful, I must offer some well intended criticisim. I am not sure if english is your second language, or if you intended the article to read in the style of a personal journal. The article would have been much easier to read if you had used proper puctuation, spelling or even close to the correct grammar. I would expect higher quality writting on a site such as this one.




Post #50153
Posted Tuesday, January 14, 2003 7:01 AM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: Moderators
Last Login: Yesterday @ 1:53 PM
Points: 6,705, Visits: 1,684
Nope, English is my first and only language. Well, I speak geek a little. We're pretty informal here, though we do try to get the spelling correct. We (Steve, Brian, myself) find most technical content hard to read, so we try for a less intense approach. In this case I did write in journal fashion because I wanted to try to present as best I could what happened/how I felt, not really try to clean it up.

We've got a book project under way to compile info from the site and the majority of readers did want us to correct typo's and minor mistakes, so we'll be reviewing old content and putting more time into proofing new ones. In the interim, if you see something wrong, please do point it out and we'll try to fix it.

Andy
http://www.sqlservercentral.com/columnists/awarren/




Andy
SQLAndy - My Blog!
Connect with me on LinkedIn
Follow me on Twitter
Post #50154
Posted Tuesday, January 14, 2003 8:03 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, June 20, 2003 12:00 AM
Points: 1, Visits: 1
I had some bad memories while reading your article. Things can get ugly in a big way when dealing with SQL Server...Can you believe that SQL Server 7 was supposed to be able to run itself without needing DBA's (right)? SQL 2000 should probably fly the space shuttle or something. I have found that when SQL breaks now...it's not a small issue.

At the United Network for Organ Sharing, we had SQL 2000 running on clustered servers with the databases on a SAN. All of a sudden (I'm not kidding), a very important (1 megabyte) database file disappears off the SAN and now our 240 GB database is suspect. Now nobody in the country can match organs.

Having the cluster didn't help, since the problem was on the SAN. We pointed the app to our Hotsite, which was kept up to date via Log Shipping. It took several hours the next day to get everything back to normal, but lives were saved, thanks to Log Shipping.




Post #50155
Posted Tuesday, January 14, 2003 1:56 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Today @ 6:29 PM
Points: 32,834, Visits: 14,975
Glad I'm not Andy or the last guy

I've been pretty lucky with very few disasters, and hopefully that will continue.

Steve Jones
sjones@sqlservercentral.com
http://www.sqlservercentral.com/columnists/sjones
www.dkranch.net







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #50156
Posted Wednesday, January 15, 2003 9:49 AM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Wednesday, December 17, 2003 12:00 AM
Points: 13, Visits: 1
Reads like a story. We know we need to restore or re-snapshot when such a thing happens. What would have added value to this article is, "what originally caused the problems" and "how to identify the root of the problem" and "how to avoid such hardware failure (Checks one should do on a regular basis)"


Post #50157
Posted Wednesday, January 15, 2003 3:59 PM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: Moderators
Last Login: Yesterday @ 1:53 PM
Points: 6,705, Visits: 1,684
As I mentioned above, it IS written in journal style. As for your questions, you tell me? How could I know that the container would drop? What could I do to prevent it next time? Easy to say cluster everything, keep a warm standby, etc, but sometimes companies truly cannot afford it (or you can't convince them to afford it).

What I was hoping to share was how something gets handled when regardless of whether you tried to prevent it, could have prevented it, etc, things go bad and you have to make decisions. If you can learn something, think of something, not do something from reading my sad story, then I've done some good.


Andy
http://www.sqlservercentral.com/columnists/awarren/




Andy
SQLAndy - My Blog!
Connect with me on LinkedIn
Follow me on Twitter
Post #50158
« Prev Topic | Next Topic »

Add to briefcase 123»»»

Permissions Expand / Collapse