SQLServerCentral Article

From the soapbox: Does anyone know what disaster recovery is?


Disclaimer:  From

the soapbox does not necessarily represent the views of SQLServerCentral.com,

Microsoft, or the author himself.

So you have full backups,

differential backups and transaction log backups enabled and functioning for

your production databases.  You have

documented procedures for restoring databases, and you perform quarterly tests

of the process.  Is this disaster

recovery for SQL Server?


Disaster recovery is NOT the

equivalent of a backup and restore plan; it encompasses much more than that. 

To be sure, this is an integral part of a DR plan, but it is still only a

piece of that plan.  Here are some

common beliefs I've encountered; in fact, I've held some of them myself at one

point or another in my career:

"Our server staff backs

up all of the databases with product X -- I don't need to worry about it".  I really hope there's no full-time DBA that would say this. 

I've run into this with “pseudo”-DBAs -- developers or systems staff

that have the responsibility thrust upon them. 

While a centralized backup methodology in a large environment makes life

much, much easier, you need to be certain that whoever manages that backup

solution understands the specifics of how to properly backup a SQL Server

database.  More than once I’ve

seen a DBA-less company use an open-file backup solution on a database; it’s

the easiest way to get introduced to dealing with a “suspect” database. 

Oh, it's so much fun.

"As long as my databases

are backed up, my responsibility as a DBA ends". 

At one point earlier in my career, I actually believed this. At the time

I was working with a company where IT was organized in "silos" -- all

of the DBAs were in one group, the systems people in another, etc. 

This made cross team interaction difficult, both from a communication and

logistical standpoint to a cultural one.  Each

group had very clearly defined areas of responsibility, and we all clung to them

to ensure that we didn't get blamed for an issue that beyond our control. 

I now fervently believe that it is irresponsible for a DBA to simply stop

with the backups of databases.  It

doesn't do much good to have a solid database backup plan if there's no plan or

process for recovering the server itself.

"As a DBA, I know the best

way to set up a backup scheme for my databases". 

Of course, the DBA knows the best technical way to set up backups.  The scheme, however, should always be dictated by

business units.  There are three

questions that always need to be asked of any business unit:

1) What is an acceptable amount

of data loss

2) What is an acceptable amount

of downtime

3) How long do backups need to

be retained

Initially, the answers will

almost always be 1) "None", 2) "None", 3)

"Forever".  When that

happens, simply explain the costs of co-location, data archival services, SAN

implementation, clustering, etc.  At

that point you'll get a reasonable answer. Your legal department should also be

involved, because there may be legal ramifications of data loss unknown to the

business unit.  From there you can

build a technical backup scheme -- how often you backup up, do you need to use

differentials, how long do you retain transaction logs, etc. 

This plan and it’s ramifications should then be reviewed with the

business unit, legal, etc.  Having them sign off on it is also an advisable thing. 🙂

"Disaster recovery is the

responsibility of IT". While IT plays an integral role in the development

of the plan, it is ultimately the responsibility of the business itself. 

Executives and management need to understand the impact of each disaster

scenario, as well as what's required to get up and running again. 

Business units must have some sort of contingency to continue operating

without access to an application. Facilities staff must be ready to correct

power, HVAC and environment problems (such as water leaking into the server room

-- has anyone else had the joy of dealing with a soggy server?). 

Again, I believe it is irresponsible for a DBA not to raise awareness of

the company's entire responsibility towards disaster recovery.

"I'm just a DBA -- I can't

make a company-wide disaster recovery plan happen". 

Ah, the sweet smell of despair -- I've been there myself. 

It is true that you can't make a company-wide DR plan, but you can

make it happen.  Talk to your

manager about concerns you have.  Talk

to business units and their managers.  Pound

on the CEO's door as if you're storming a castle. As long as you are tactful and

professional in your approach to driving change (avoid wandering around the

halls beating a drum and shouting "We're doomed!!" – take it from

experience, it doesn’t go over too well), chances are someone will eventually

listen, and you're efforts will be appreciated. 

“I have a full backup scheme

in place – I can recover to any point in time. 

I have redundant hardware, off-site backups, transaction logging, etc. 

We could lose this entire facility in an earthquake, and I can still

restore the database to another server up to the latest committed transaction.

The database can be recovered in the event of any disaster.” 

Any disaster?  Are you sure? 

What about the data entry clerk who’s been inconsistently entering

incorrect data? Sure, you can recover to a point in time, but when is that point

in time? And how disruptive will it be to the use of the application to attempt

this type of restore?  User error is the most frustrating and difficult disaster

scenario to deal with.  When the CEO

is standing in front of you demanding that you “fix it”, you still have to

deal with it, whoever is responsible.

“Our company understands the

true needs of a disaster recovery process – we just don’t have the time or

resources to undertake a project of this size.” 

Creating a DR plan doesn’t have to be a major project – in fact,

it’s less likely to be successful or useful if it is undertaken as a single,

all-encompassing project.  Take things in small steps and create the DR plan as you go. 

You could start by simply writing down the names and phone numbers of

anyone that needs to be contact, from internal staff to outside vendors (power,

plumbing, HVAC, etc).  Document restore procedures. 

Store backup information off-site.  The

point is, take it slow. Disaster recovery planning should be viewed as an

ongoing process, not a single task to be completed.

If any of these scenarios sound

familiar, you could be heading for trouble. 

At least that’s what the view looks like from up on the soapbox.

Am I wrong? 

Am I right?  Am I just plain

crazy?  Just because the rant is

over, it doesn’t mean the debate is resolved.


5 (1)

You rated this post out of 5. Change rating




5 (1)

You rated this post out of 5. Change rating