Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Grace Under Pressure


Grace Under Pressure

Author
Message
Steve Jones
Steve Jones
SSC Guru
SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)

Group: Administrators
Points: 51361 Visits: 19004
Comments posted to this topic are about the item Grace Under Pressure

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Sigerson
Sigerson
SSChasing Mays
SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)SSChasing Mays (625 reputation)

Group: General Forum Members
Points: 625 Visits: 1232
[deleted]

Sigerson

"No pressure, no diamonds." - Thomas Carlyle
TravisDBA
TravisDBA
SSCommitted
SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)

Group: General Forum Members
Points: 1702 Visits: 3069
I agree most definitely with practice, practice,practice. However, with most Disaster Recovery scenarios I have had experience with in the last 20 years, the actual event never goes as planned in our dry runs, and they most definitely are different everytime around. Someone once told me "Don't panic!". Well... that is definitely easier said than done when you are in the middle of the lake up to your rear-end in alligators. I think that should be restated as "Don't do anything rash when you first feel that pang of panic." As far as managers raising their voice at you, I have a real good comeback for that and have used it in the past too. "Sir, do you have anything to contribute besides browbeating everyone here? If not, please leave the server room and let the experts think so we can analyze and address this issue for you as quick as possible, thank you.":-D

"Technology is a weird thing. It brings you great gifts with one hand, and it stabs you in the back with the other. ...:-D"
Miles Neale
Miles Neale
SSCrazy
SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)

Group: General Forum Members
Points: 2854 Visits: 1694
Steve, Honestly thank you for this. Very good piece.

I remember two disasters in the IT field over the years. One year back in the 70's the mainframe computer room caught fire and the emergency shutdown happened. This was a complete shutdown with a hard down and Halon filling the room. Two people were almost caught in the room when the emergency glass door closed to smother the fire. In the weeks that followed there were meetings, planning, and more meetings to replan and restart the center. Vendors were there 24x7 and people worked hard just to get the system back online. When management types would mouth off and start complaining they were taken into more meeting to work it out behind closed doors and the technicians were not there. The techs were insolated from the politics and screaming while they worked the problem and got the site operational as soon as they could. That worked.

The second disaster we when due to a single compromise in the engineering of the building. One water pipe entered or crossed the area where the main computer wiring traversed the building. In essence there was an unlimited supply of water traveling across the mainline channel which was lined with cement. Needless to say one day that water line broke and the water traveled down the channel directly back into the computer and it dumped out in the ceiling just above the racks of servers. Vendors and techs were on site within two hours and the entire site was back and operational in a few days. But again it was a very controlled project with those who had questions not given access to the techs.

In these cases cool heads and calm words bring out the desired result. Yelling and the blaming since it is counter productive needs to be removed from the area where productivity needs to optimized. This is a good case where if there is someone who really needs to yell should be instructed to go yell at the wall. :-)

M.

Not all gray hairs are Dinosaurs!
Peter Schott
Peter Schott
Ten Centuries
Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)

Group: General Forum Members
Points: 1271 Visits: 1911
I remember a situation like that in the past - had a server issue that was being worked on by our tech guy and saw him being watched over by three managers, doing nothing except watching. (At least they weren't trying to be helpful at that point.) I ducked in, asked if I could do anything helpful, then expressed that I'd leave him to do the job at hand then. Never understood what they thought they'd get out of standing over his shoulder when they likely didn't even know what he was doing in the first place.

Shortly after that we had a better plan in place to test out a recovery plan. All backup tapes were sent there with a small number of servers that we needed to completely restore and get everything back online. We sent our main IT guy up to the center with pretty much nothing and he verified that he could restore everything to a functional state within the week. It was a bit painful, but it did ensure that we knew we could recover if something went critically wrong.



TravisDBA
TravisDBA
SSCommitted
SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)SSCommitted (1.7K reputation)

Group: General Forum Members
Points: 1702 Visits: 3069
Peter Schott (12/10/2012)
Never understood what they thought they'd get out of standing over his shoulder when they likely didn't even know what he was doing in the first place.



Peter, it's called micro-management, and many managers still actively practice it. Its a basic mistrust that your people need to be constantly supervised, even when they don't. It's a mind-set, and not an easily broken one either.:-D

"Technology is a weird thing. It brings you great gifts with one hand, and it stabs you in the back with the other. ...:-D"
tabinsc
tabinsc
SSC Eights!
SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)SSC Eights! (816 reputation)

Group: General Forum Members
Points: 816 Visits: 917
I've had managers who would come in and "supervise" even the smallest issues. I've also worked on projects where we would waste 1-2 hours a day in status update meetings.

Steve, I see you have very good taste in music... and album covers.

Tony
------------------------------------
Are you suggesting coconuts migrate?
Steve Jones
Steve Jones
SSC Guru
SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)

Group: Administrators
Points: 51361 Visits: 19004
tabinsc (12/10/2012)
I've had managers who would come in and "supervise" even the smallest issues. I've also worked on projects where we would waste 1-2 hours a day in status update meetings.

Steve, I see you have very good taste in music... and album covers.


Thanks! Been a Rush fan since the early 80s.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Barry Wright-268269
Barry Wright-268269
Grasshopper
Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)Grasshopper (18 reputation)

Group: General Forum Members
Points: 18 Visits: 189
It is interesting that you are calling this Disaster Recovery Week. In my company, we strenuously avoid the term "disaster recovery". Instead, we use the term "business continuity"; we are not going to recover from a disaster, we are going to continue business. We have a colo site with SAN across multiple sites, replication, etc. We practice our BCP (Business Continuity Plan) regularly with the entire company running from the colo or "mirror site" for a full 24 hours.
Steve Jones
Steve Jones
SSC Guru
SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)SSC Guru (51K reputation)

Group: Administrators
Points: 51361 Visits: 19004
I see this as two different items.

DR is recovering the systems back to a working state, or using new systems.

BC is the continuation of operation for your business. BC includes DR, but encompasses much more.

It's a valid point, that BC is really the goal.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search