February 16, 2019 at 1:00 pm
Comments posted to this topic are about the item You Need Offline Backups
February 16, 2019 at 1:34 pm
Steve, backup strategy is one area where defense in depth and breadth is relatively inexpensive insurance which more than pays for itself whenever it's actually needed.
Also worth remembering is that not every attack (or disruption) starts with a neon sign. A few years ago, I found myself involved in a recovery effort where a critical system was sabotaged by redefining (and then encrypting) a single stored procedure to include a series of progressively worsening deletions and corruptions, intended to be space out over several weeks. By the time the attack was even recognized as an attack, there weren't any backups which didn't contain corrupted or incomplete data. Even if the fail-over system had been properly prepared, it would have merely contained very recent snapshot of the corrupted data.
Coming from a project where I had compressed daily backups covering the previous month, then one per week preserved for at least a year, and (if I remember correctly) at least one backup per month going back a few years, I'll let you fill in however many blanks on what I had to say about the folks who had been in charge when the logic bombs were first set.
February 19, 2019 at 9:10 am
I completely agree. While a hardware disaster might merely need 2-3 full backups, a subtle bug or malicious attack across a short time might need some from weeks ago. I still think that having at least a weekly and at least 2-3 monthly backups would be good ideas in today's world.
February 20, 2019 at 8:33 am
2/17/19 "An older offsite backup server was unearthed and revived. The last backup is in August of 2016"
From https://www.vfemail.net/incident.php
I guess they didn't use an offsite tape solution? Does anyone know?
Feel sad this happened but we should all learn from this, I think.
February 20, 2019 at 8:47 am
Wow, crazy. They might not use offsite. Lots of companies have moved away from that. I think most assume that user errors are discovered quickly or failures wouldn't go back more than a week for a restore.
February 20, 2019 at 9:02 am
Yes. We use 'A VENDOR' to backup to disk and tape. With tapes moved / stored offsite on a regular/weekly basis.
The 'VENDOR' have a solution to convert our tapes to a Cloud storage solution, which at first I though, great, moving away from magnetic media etc.
BUT now, I'm not so sure!
February 20, 2019 at 9:11 am
Steve Jones - SSC Editor - Wednesday, February 20, 2019 8:47 AMWow, crazy. They might not use offsite. Lots of companies have moved away from that. I think most assume that user errors are discovered quickly or failures wouldn't go back more than a week for a restore.
For what it's worth, their FAQ claims that they backup all user data nightly to an off-site location.
https://www.vfemail.net/faq.php
What is your backup strategy / data retention policy?
VFEmail feels it's important to provide a long-term, stable, environment for our users. In that effort, we perform nightly backups to an offsite host from all on-site and off-site mail storage locations. This backup runs at 12am CST (-0600) and contains all user data.
3rd party storage of user data is generally not wanted by privacy-conscious users. If you fall into that category, you will want to use POP3 and download your mail daily. Our backup is on a daily/weekly rotation, initiated by a snapshot. If you do recieve mail between your last POP and the snapshot at 12am, it will exist on backup for a week - unless it's on Saturday night, then it's a year. You should set your POP program to download every 5-10 minutes in order to avoid having your mail caught on backup.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
February 20, 2019 at 2:36 pm
If that's true, then how could they lose more than a day or two of data? Offline shouldn't be accessible remotely. A human would have to kill tapes.
February 20, 2019 at 2:58 pm
Steve Jones - SSC Editor - Wednesday, February 20, 2019 2:36 PMIf that's true, then how could they lose more than a day or two of data? Offline shouldn't be accessible remotely. A human would have to kill tapes.
It seems they need to update their backup process... or their FAQ.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
February 21, 2019 at 9:21 pm
You should just post a link to this forum on posts to your blog that cover the same content as your editorials.
The conversation here is much more robust.
412-977-3526 call/text
February 21, 2019 at 9:23 pm
Eric M Russell - Wednesday, February 20, 2019 2:58 PMSteve Jones - SSC Editor - Wednesday, February 20, 2019 2:36 PMIf that's true, then how could they lose more than a day or two of data? Offline shouldn't be accessible remotely. A human would have to kill tapes.It seems they need to update their backup process... or their FAQ.
They recycle tapes 1-6 and store tape 7 which goes into a 1-52 rotation?
412-977-3526 call/text
February 22, 2019 at 5:27 am
Steve Jones - SSC Editor - Wednesday, February 20, 2019 2:36 PMIf that's true, then how could they lose more than a day or two of data? Offline shouldn't be accessible remotely. A human would have to kill tapes.
Why do you think offline backups shouldn't be remotely accessible, Steve? As long as sensible protocols are followed to preclude premature destruction, why not enable immediate retrieval of any backed-up data?
The only time I haven't had immediate remote access to my off-site back-ups was when I was backing up in-theater logistical data to a CONUS site -- and that limitation was driven by a combination of security zoning and constrained bandwidth. If I needed to fetch a a full back-up from the off-site archive, it was usually faster to pull it down on a laptop in my living quarters (my half of a custom-built double-wide down by the lake).
February 23, 2019 at 2:33 pm
If there is an attack, especially an insider attack, you want an air gap between some of your backups. While it's nice to have immediate access, I would also be concerned about a virus/worm/insider removing backups first, or disabling them in some way, then attacking the main system.
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply