Crash and Burn

SQLServerCentral is supported by Red Gate Software Ltd.

Popular Topics

Home

Members

Calendar

Who's On

Home » SQLServerCentral.com » Editorials » Crash and Burn

11 posts, Page 1 of 2

1 2 »»

Crash and Burn

Rate Topic

Display Mode

Topic Options

Author

Message

Steve Jones - SSC Editor

Posted Monday, December 28, 2009 8:22 PM

SSC-Dedicated

Group: Administrators
Last Login: Today @ 5:09 AM
Points: 31,526, Visits: 13,864

Comments posted to this topic are about the item Crash and Burn

Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help

Post #839662

Jeff Moden

Posted Monday, December 28, 2009 11:51 PM

SSC-Dedicated

Group: General Forum Members
Last Login: Yesterday @ 9:01 PM
Points: 33,111, Visits: 27,037

Heh... I absolutely agree... all the backups in the world mean squat unless you can restore them. The DBA's a couple of jobs ago went through that. Took 40 people ten days to almost get all the data back. It was a huge mess.

I noticed that Jeff lost the data on the hosting provider... the more I hear about things like this, the more I'm inclined to say keep the cloud... there's too much weather there.
Wink

--Jeff Moden

"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers on T-SQL questions, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/

Post #839700

bwillsie-842793

Posted Tuesday, December 29, 2009 6:52 AM

SSC-Enthusiastic

Group: General Forum Members
Last Login: Wednesday, May 29, 2013 1:03 PM
Points: 107, Visits: 288

Excellent point on checking ability to restore.

I started as IS manager at a company some years back and asked what they had for backup and restore procedures.

The VP of Sales (also their "Techie") promptly informed that they knew they had good backups because they hadn't gotten any error messages on their backup runs for over 6 months.

This was particularly odd to me when I noticed the dust on the backup tape that was in the drive. It looked like it had been in the machine 6 months itself.

A check in the backup logs indicated that last successful run was also over 6 months in the past, due to a dead drive.

We quickly replaced the drive, did backups, and tested restores.

I still haven't figured out how they managed to be so lucky that they made it 6 months without someone wanting at least a minor file restored.

Post #839817

OCTom

Posted Tuesday, December 29, 2009 8:13 AM

SSCrazy

Group: General Forum Members
Last Login: Today @ 6:06 AM
Points: 2,029, Visits: 2,868

Trust no one but you with your stuff. You are utlimately responsible.

Post #839853

bwillsie-842793

Posted Tuesday, December 29, 2009 8:21 AM

SSC-Enthusiastic

Group: General Forum Members
Last Login: Wednesday, May 29, 2013 1:03 PM
Points: 107, Visits: 288

skjoldtc (12/29/2009)

Trust no one but you with your stuff.

When it comes to my data, I find it hard to even trust myself.

Trust, but verify...

Post #839860

Jon Russell

Posted Tuesday, December 29, 2009 12:51 PM

Old Hand

Group: General Forum Members
Last Login: Thursday, June 06, 2013 5:46 PM
Points: 326, Visits: 901

Agreed. Also, practice makes perfect. I perform restores on a regular basis. Not just for rehearsal, but also to refine DR recovery procedures and to confirm that a copy of the latest backup is not corrupt.

Post #840021

bradmcgehee@hotmail.com

Posted Tuesday, December 29, 2009 1:55 PM

SSC-Enthusiastic

Group: General Forum Members
Last Login: Tuesday, May 14, 2013 1:08 PM
Points: 175, Visits: 719

In two of the presentations I regularly give, I emphasize the importance of maintaining backups that are restorable, including the need for storing backups offsite and regularly checking to see if backups can be restored. Often, when I say this during presentations, many members of the audience roll their eyes, as if saying to me, "yeah, duh, that's obvious." The reason I include such "obvious" advice in these presentations is because some DBAs don't put this obvious advice into action, and I want to provide a little reminder to them. So, if you attend one of my presentations in the future, and you hear this advice from me, please stand up and shout, "great advice man", so that those DBAs that roll their eyes will know that I'm not the only one who believes in the importance of DBA basics.

Brad M. McGehee
Microsoft SQL Server MVP
Director of DBA Education, Red Gate Software
www.bradmcgehee.com

Post #840051

Daniel Hallam

Posted Tuesday, December 29, 2009 1:59 PM

SSC Rookie

Group: General Forum Members
Last Login: Tuesday, June 11, 2013 5:56 PM
Points: 35, Visits: 386

Last year we had a full test of our disaster recovery plan (always comforting to be part of a company that has one). Mostly went well except that the disk segment replication to our DR site ended up with unsync'd filegroups for our primary db server so we couldn't do the restore, meaning a complete failure in the end.

If it was a real disaster, we would have had to resort to tape backups taking our DR recovery from 3 hours to over 48 hours, and having some data loss.

Always good to make sure a restore works.

Post #840054

markjholmes

Posted Tuesday, December 29, 2009 2:29 PM

Valued Member

Group: General Forum Members
Last Login: Wednesday, June 12, 2013 10:47 AM
Points: 65, Visits: 95

I've encountered the rude awakening a few times myself. Once, at a company that had no DBA, just "the SQL boy," a manager issued a DROP TABLE statement by accident on a database in bulk-logged recovery mode. The transaction logs had never been backed up, and were 300GB on 20GB of data. We soon realized that in order to restore last week's backups (because they took so long they were only done on weekends), the 100GB free was insufficient, and needed some creative space management and several days of data transfers on a severely overloaded and badly designed system. They ended up regenerating the missing data over a three week period because it was faster. They didn't lose ALL their customers...

Post #840077

Grant Fritchey

Posted Wednesday, December 30, 2009 5:28 AM

SSChampion

Group: General Forum Members
Last Login: Yesterday @ 9:49 AM
Points: 13,436, Visits: 25,281

Excellent, excellent, excellent post. Nothing is more important than verifying backups, except, verifying that you know how to run a restore. You're absolutely right when you say that backups are no good unless you can restore them, but it goes beyond actually validating that the backup files themselves are valid and accessible. You need to know that you, and any other DBA's in the organization, can actually run a restore, knows how to read the file header, can do a point in time recovery, etc. Practice restoring databases not only validates that the backups are good, but that you're good as well.

----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software

Post #840263

« Prev Topic | Next Topic »

11 posts, Page 1 of 2

1 2 »»

Permissions