There was a large hurricane in the US a short while back. It was a devastating storm for many people, and my heart goes out to those that suffered or are still suffering. There are lots of lessons to be learned in many areas, but a few surprising ones for those people that run technology infrastructures. A number of data centers were shut down because of physical flooding, but others were shut down after electrical substations failed and they were unable to run their generators.
When I evaluated data centers a decade ago, I was always shown the number of UPSes on site, and the high capacity diesel generators with large fuel tanks that were available just in case of extended outages. Salespeople would bring out their contracts that showed suppliers would commit to refilling their fuel tanks, providing for every contingency.
Except a lack of diesel. In Denver we wouldn't have the need for a staircase bucket brigade, but we might have the need for a roadside chain gang carrying containers in a blizzard. You cannot plan for every contingency because there are many factors out of your control. When a disaster gets large enough, it doesn't matter what you have contracted for. There will be outside influences, like the lack of elevators or the inability of trucks to physically reach your location.
One of the Red Gate customers was in New York and almost lost their data center after the storm. They asked our technical team to help them prepare scripts to restore data backups for their clients in the event they had to send the full, diff, and log backups (along with application code) to the customers. It was an last-ditch effort to allow their customers to continue to run their service. Fortunately they never had to use any of the scripts, but it did help the company realize they need to build more options for business continuity in the event of future disasters.
I hope none of you ever experiences anything like Hurricane Sandy. You can't completely prepare, but you can practice your recovery skills on a regular basis and be prepared to respond when disaster strikes.
Steve Jones
The Voice of the DBA Podcasts
We publish three versions of the podcast each day for you to enjoy.
- Watch the Windows Media Podcast - 22.4MB WMV
- Watch the iPod Video Podcast - 21.4MB MP4
- Listen to the MP3 Audio Podcast - 4.3MB MP3
The podcast feeds are available at sqlservercentral.mevio.com. Comments are definitely appreciated and wanted, and you can get feeds from there. Overall RSS Feed: 
or now on iTunes! 
Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.
You can also follow Steve Jones on Twitter:
