Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Worst Day as a DBA Story Finalists


Worst Day as a DBA Story Finalists

Author
Message
Press Release
Press Release
Valued Member
Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)Valued Member (74 reputation)

Group: General Forum Members
Points: 74 Visits: 0
Comments posted to this topic are about the item Worst Day as a DBA Story Finalists
Arcanas
Arcanas
Old Hand
Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)

Group: General Forum Members
Points: 380 Visits: 350
As much as I love to tell my story about renaming everyone Jong, wow. How can any story ever written compete with the Karate Kid story? :-D
logitestus
logitestus
SSC-Addicted
SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)SSC-Addicted (452 reputation)

Group: General Forum Members
Points: 452 Visits: 511
The company I was working for at the time was fairly large and after the incident I had plenty of opportunities to re-tell the story. It happened as I said it did but I glossed over the fact that I spent the night in jail (not fun) and narrowly escaped prison/probation (the judge sympathized with me - the security guard had several citations against him for physically confronting other people). The only reason why I wasn't bankrupted was due to my company agreeing with me that I was performing my job (up to the assault) and supplied the lawyer.

I use the story now as a funny anecdote in don't-get-caught-like-this sorts of training sessions.
David Moutray
David Moutray
SSC Veteran
SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)SSC Veteran (276 reputation)

Group: General Forum Members
Points: 276 Visits: 895
Well, so far, "Karate Kid" is the runaway favorite. Is this due to some sub-conscious accumulation of violent frustration on the part of DBA's? Hmmm.

My personal favorite was the Early Morning Drive Disappearance. Frankly, that seems like a bigger nightmare to me.
Arcanas
Arcanas
Old Hand
Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)Old Hand (380 reputation)

Group: General Forum Members
Points: 380 Visits: 350
The karate kid story is one of my favourite stories of all time. It made me laugh. I can just imagine the sinking feeling as you realize you jusk took out the data center... Laugh

The only other great story I have to share is the day we lost our ERP. At the time, the company I was working for was running without an IT manager. My position was in IS, on the development side, so I had little to no pull with that team. The IT team was running by committee, and they had one loud mouth who tended to railroad the team into bad ideas.

On this given Sunday, our building maintenance team needed to take the power in the server room offline for an hour to do some upgrades to the system. The only system impacted was our old SAN, which our ERP databases just happened to be running on (it was the only system left to migrate). The maintenance guys, being good guys and fairly smart to boot, offered to build us a power cord to keep the system online during the outage, at significant expense (220V cable, over 75 feet long, not cheap!). Our mighty hero, though, figured no. We can just take the SAN offline during the outage. And he made this call despite the IT guys saying it was a bad idea, and the maintenance guys saying it was a bad idea.

Sunday rolls around, and the power upgrade goes well. Maintenance cruises through it, and has everything back online ahead of schedule. But then the SAN is brought back online. And you can guess how this ends. I get a call after hour two of the SAN outage to let me know that it was gone. The SAN lost 24 of the 26 disks, and the vendor has no idea how to get them back. Apparently, the firmware on it has a bug that caused it to lose the entire configuration, and there is no fix. Enter the IS team, who conferred for half an hour to come up with a plan to do an emergency migration of our ERP system to a new server. Luckily, we had a new system in place, but had not started testing any migration plan to it yet.

What started off as a horrible day had a good ending. We were able to get our entire ERP back online within 6 hours, with zero data loss. And our mighty IT anti-hero? It was revealed that he was bilking the company for OT that he didn't work, and was fired about a month later. The lesson here is, when even the non-IT staff think it is a bad idea, it's time to reevaluate your thinking!
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search