Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Grace Under Pressure Expand / Collapse
Author
Message
Posted Sunday, December 9, 2012 3:43 AM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Yesterday @ 3:56 PM
Points: 33,182, Visits: 15,320
Comments posted to this topic are about the item Grace Under Pressure






Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #1394391
Posted Monday, December 10, 2012 7:06 AM


SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: 2 days ago @ 6:57 AM
Points: 262, Visits: 808
[deleted]

Sigerson

"No pressure, no diamonds." - Thomas Carlyle
Post #1394602
Posted Monday, December 10, 2012 10:22 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Thursday, March 6, 2014 1:05 PM
Points: 1,334, Visits: 3,068
I agree most definitely with practice, practice,practice. However, with most Disaster Recovery scenarios I have had experience with in the last 20 years, the actual event never goes as planned in our dry runs, and they most definitely are different everytime around. Someone once told me "Don't panic!". Well... that is definitely easier said than done when you are in the middle of the lake up to your rear-end in alligators. I think that should be restated as "Don't do anything rash when you first feel that pang of panic." As far as managers raising their voice at you, I have a real good comeback for that and have used it in the past too. "Sir, do you have anything to contribute besides browbeating everyone here? If not, please leave the server room and let the experts think so we can analyze and address this issue for you as quick as possible, thank you."

"Technology is a weird thing. It brings you great gifts with one hand, and it stabs you in the back with the other. ..."
Post #1394705
Posted Monday, December 10, 2012 11:46 AM
SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Yesterday @ 3:25 PM
Points: 2,294, Visits: 1,345
Steve, Honestly thank you for this. Very good piece.

I remember two disasters in the IT field over the years. One year back in the 70's the mainframe computer room caught fire and the emergency shutdown happened. This was a complete shutdown with a hard down and Halon filling the room. Two people were almost caught in the room when the emergency glass door closed to smother the fire. In the weeks that followed there were meetings, planning, and more meetings to replan and restart the center. Vendors were there 24x7 and people worked hard just to get the system back online. When management types would mouth off and start complaining they were taken into more meeting to work it out behind closed doors and the technicians were not there. The techs were insolated from the politics and screaming while they worked the problem and got the site operational as soon as they could. That worked.

The second disaster we when due to a single compromise in the engineering of the building. One water pipe entered or crossed the area where the main computer wiring traversed the building. In essence there was an unlimited supply of water traveling across the mainline channel which was lined with cement. Needless to say one day that water line broke and the water traveled down the channel directly back into the computer and it dumped out in the ceiling just above the racks of servers. Vendors and techs were on site within two hours and the entire site was back and operational in a few days. But again it was a very controlled project with those who had questions not given access to the techs.

In these cases cool heads and calm words bring out the desired result. Yelling and the blaming since it is counter productive needs to be removed from the area where productivity needs to optimized. This is a good case where if there is someone who really needs to yell should be instructed to go yell at the wall.

M.




Not all gray hairs are Dinosaurs!
Post #1394726
Posted Monday, December 10, 2012 12:31 PM
SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Yesterday @ 9:46 AM
Points: 840, Visits: 1,276
I remember a situation like that in the past - had a server issue that was being worked on by our tech guy and saw him being watched over by three managers, doing nothing except watching. (At least they weren't trying to be helpful at that point.) I ducked in, asked if I could do anything helpful, then expressed that I'd leave him to do the job at hand then. Never understood what they thought they'd get out of standing over his shoulder when they likely didn't even know what he was doing in the first place.

Shortly after that we had a better plan in place to test out a recovery plan. All backup tapes were sent there with a small number of servers that we needed to completely restore and get everything back online. We sent our main IT guy up to the center with pretty much nothing and he verified that he could restore everything to a functional state within the week. It was a bit painful, but it did ensure that we knew we could recover if something went critically wrong.



Post #1394755
Posted Monday, December 10, 2012 12:53 PM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Thursday, March 6, 2014 1:05 PM
Points: 1,334, Visits: 3,068
Peter Schott (12/10/2012)
Never understood what they thought they'd get out of standing over his shoulder when they likely didn't even know what he was doing in the first place.



Peter, it's called micro-management, and many managers still actively practice it. Its a basic mistrust that your people need to be constantly supervised, even when they don't. It's a mind-set, and not an easily broken one either.


"Technology is a weird thing. It brings you great gifts with one hand, and it stabs you in the back with the other. ..."
Post #1394761
Posted Monday, December 10, 2012 2:19 PM


Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Yesterday @ 8:32 AM
Points: 561, Visits: 731
I've had managers who would come in and "supervise" even the smallest issues. I've also worked on projects where we would waste 1-2 hours a day in status update meetings.

Steve, I see you have very good taste in music... and album covers.


Tony
------------------------------------
Are you suggesting coconuts migrate?
Post #1394783
Posted Monday, December 10, 2012 2:41 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Yesterday @ 3:56 PM
Points: 33,182, Visits: 15,320
tabinsc (12/10/2012)
I've had managers who would come in and "supervise" even the smallest issues. I've also worked on projects where we would waste 1-2 hours a day in status update meetings.

Steve, I see you have very good taste in music... and album covers.


Thanks! Been a Rush fan since the early 80s.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #1394788
Posted Wednesday, December 12, 2012 7:23 AM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Thursday, June 19, 2014 4:17 PM
Points: 10, Visits: 134
It is interesting that you are calling this Disaster Recovery Week. In my company, we strenuously avoid the term "disaster recovery". Instead, we use the term "business continuity"; we are not going to recover from a disaster, we are going to continue business. We have a colo site with SAN across multiple sites, replication, etc. We practice our BCP (Business Continuity Plan) regularly with the entire company running from the colo or "mirror site" for a full 24 hours.
Post #1395662
Posted Wednesday, December 12, 2012 11:11 AM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Yesterday @ 3:56 PM
Points: 33,182, Visits: 15,320
I see this as two different items.

DR is recovering the systems back to a working state, or using new systems.

BC is the continuation of operation for your business. BC includes DR, but encompasses much more.

It's a valid point, that BC is really the goal.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #1395818
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse