I figure it's not a bad idea to chime in here. My current DBA (I'm playing little ol' developer at the moment) is beating his head on the walls creating a proper DR system. I don't respect restrictive DBA's often, but he has my respect, it makes sense. With that in mind...

It's been a political nightmare. The gear is not the issue. The setup is not the issue. The volume of wtf outside of our domain is the issue.

In some ways I want to go back to the 1990s... when you could piss on the sysadmins and they ran so you could get your DB setup properly. One LUN goes sideways now and you've got four parties screaming 'Not ME!' until you can nail one to the floor with a sledgehammer.

And then they cry.

I want to go back to when we OWNED our damned work. I want to go back to the time when three people stood up at a meeting and said "Whoops, that's mine, sorry.".

You ask about the decision of when to fail. I have an issue with that, but not with the question. My issue is "can we fail?!". Too often, the answer is "No. We die."

I realize this seems like a bit of a soapbox, but this is five companies deep where I've seen this. Seriously, own up.

I own my failures. They should too. DR is not a one system show. I can't DR anything that doesn't have proper backup from all parties. Currently DR is like pissing uphill and upwind for anywhere that doesn't have dedicated staff dedicated to their components. There's no way for things to end well.