Your Recovery Time

  • Comments posted to this topic are about the item Your Recovery Time

  • (Disclaimer: we don't actually have a formal DBA position; most of the data systems we have were built by contractors and have been maintained by people with little to no DBA experience.  I've been with the company for less than a year, in which time my boss has expressed a fervent desire to get out of reactionary mode and into a more proactive state.  Unfortunately it sometimes takes a serious disaster to convince managers of the importance of preparation.)

    We found out the hard way last year, that most of the legacy systems the company has been using for 10+ years didn't even have backups.  So, some of those machines will never be rebuilt.  But even for the systems that had backups, we had little to no idea of how to go about rebuilding systems that we didn't have good documentation on.  So our emergency mode lasted for almost 2 months, and 3 months later, we're finally working on outlining and implementing backup and DR practices that will keep us from such a nightmare happening again.  Luckily our system is just the place we aggregate data from multiple different sources, so business didn't quite grind to a halt during our recovery.  We just had a lot of very frustrated analysts.

  • Most enterprise database environments have a hub and spoke or snowflake architecture. There is a central transactional database server, and then there are data marts and other line of business databases that feed from or integrate with it. It's better when the spokes can still be fully or mostly operational even while the central hub experiences unplanned down time. It depends, but if some predictable latency is tolerable, then periodically replicated or merged local tables can insure better overall availability when compared to and architecture that is tightly integrated with linked server connections. If you're leveraging micro services, then they should feed from data marts or replicas.

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • Hi,
    In early days of ERP boom, in one of the companies where I worked, ended with an Informix DB running on HP-HA disk array with RAID 5 enabled. We done that because of the ERP vendor recomendations. The system working well even in the case of one damaged disk It worked as expected: while one drive down, the information was still available. Of course we were satisfied with the investment on the solution but as soon as we replaced the hot swap damaged disk, and the information started sync, somebody asked this question: what happen if we lost a second drive?

    I´m telling this story due to, the Steve´s comment about, even with the primary node was restored, we don´t end the disaster mode, until we replaced all the array disks with new ones (one each time), and reconfigured our db to RAID 1.

    It took us about 1 month to officially ended the disaster mode operation.

  • I work for a large state agency, with a DBA staff. As far as I know, this is covered, at least for all of the databases we have.

    I make that distinction because there's one large division, which does have a SQL Server database for many of their functions. But one of my colleagues who works there tells me that there's lots of other people around the division who have MS Access databases where they store their data. These rogue players would rather die, then put the database into the SQL Server system they use, although I believe it could do what they want. They're used to one particular interface that someone back in the past, wrote for them. Anyway, I seriously doubt that whoever wrote these MS Access apps ever gave a thought to disaster recovery.

    Kindest Regards, Rod Connect with me on LinkedIn.

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply