Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Digital Storage Expand / Collapse
Author
Message
Posted Saturday, January 12, 2008 2:33 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Yesterday @ 11:24 AM
Points: 32,781, Visits: 14,942
Comments posted to this topic are about the item Digital Storage






Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #442154
Posted Monday, January 14, 2008 12:32 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Thursday, October 04, 2012 9:05 PM
Points: 382, Visits: 196
There is an ISO standard for the long term archival of documents. It's called PDF/A (I assume the 'A' stands for archival, but I'm not sure.

See http://en.wikipedia.org/wiki/PDF/A for more info
Post #442319
Posted Monday, January 14, 2008 3:29 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Friday, April 04, 2014 7:01 AM
Points: 239, Visits: 368
Several years ago (10?), when I was on a local township board, I attended a session put on by the State of Michigan Archives discussing long-term archival of documents.

At the time, digital storage required CDs of a certain quality, with all data to be re-read and re-stored on new CDs at no more than 5-year intervals.

But (and this is what I found most interesting) all documents ALSO required stored in some HUMAN-READABLE format. The archivists understood that the ability to decipher the digital format may become lost over time. Once lost, the digital format is no good. But the human-readable format would still be legible and useful.
Post #442354
Posted Monday, January 14, 2008 5:33 AM
SSC Journeyman

SSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC Journeyman

Group: General Forum Members
Last Login: Thursday, May 24, 2012 2:06 PM
Points: 97, Visits: 356
Steve,

How ironic that your editorial is on the same day as the
Worth a Second Look - Running a Query Using a Text File for Input is in the featured articles. I've worked for a large regional bank and as a consultant for a major Aluminum manufacturer. Both are required to keep financial, tax and production information for an exceptional number of years to guard against litigation. (Digital storage manufacturers benefitted from the tobacco industries problems too.) We utilized the pdf/a for reporting purposes but trusted them only a little. For data that you really truly need to recover, no matter what digital technology is in play export to a delimited file. Provided the world doesn't completely lose all power sources and reenter the dark ages we should as data professionals always be able to import the data again. (As an alternative energy geek myself I don't see that happening.)

So that's how we got the raw data out of the database.

We then stored this exported data on CD and then DVD media because of it's stable long term capabilities. AND the data was rolled to archive tape. But the archive tape was required to be restored and the consistency checked every 5 years on data with a high risk footprint. The restored export files were then archived again in the newer archival solution. This way we were covered for degradation in media as well as expiring technology. (The business process owners were responsible for assessing the risk footprint.)
Post #442407
Posted Monday, January 14, 2008 7:37 AM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Yesterday @ 11:24 AM
Points: 32,781, Visits: 14,942
Bob, that's an interesting solution, but it has the problem of you constantly having to read and store back more information on a regular basis. That could end up being a full-time job over time. Maybe even a job for a few people.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #442443
Posted Monday, January 14, 2008 10:18 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Thursday, March 20, 2014 9:41 AM
Points: 353, Visits: 423
I've thought about this over the years for any number of reasons. The "best" idea I came up with was to use "printed digital information". Some years back I read a (very small) article in Scientific American where Xerox had come up with this idea to implant the digital information of a graph in the graph itself, based on a printed pattern. Essentially something like / = 0 and \ = 1.

If you take that to a DPI level high enough you start to look at holding very large amounts of data; no where near digital storage (I think), but still respectable. If you keep the resolution "low enough" so that a low tech approach could be used to read it (magnifying glass or rough microscope) you add "backwards compatibility" if needed. Going in the direction of "higher tech", a scanner / ETL program could interpret the "data sheet" and produce the output automatically.

With the quality of printers / ink increasing as they have (I have an Epson Stylus 4500 with "archival" ink rated for 90+ years) you could in theory print the data archive on (photo?) paper, then store the paper. The volume of raw data would increase compared to written text. The potential for extracting the data again if desired is present (scanner, etc), and it has the possibility to be "backwards compatible" to lower tech retrieval (if needed).

I kept wanting to add to the density of storage by a couple of ways. The idea of a "denser" arrangement of pixels was one (the / and \ in a 2 by 2 pixel array uses half the available pixels). I kept thinking that using all 4 would be better (a nibble for all us old timers out there). If you add to that the idea of using color in the encoding, you should be able to go with some multiple of 4 bits quite nicely.

There are other aspects to this idea as well; adding instruction in the beginning to help decode (if needed), noting the standards used to encode data; define MP3, mpeg4, etc.


Beer's Law: Absolutum obsoletum
"if it works it's out-of-date"
Post #442543
Posted Monday, January 14, 2008 10:33 AM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Wednesday, October 31, 2012 4:55 PM
Points: 11, Visits: 38
The problem with archiving data is not just that you archive the application that consumes it but that you have the required hardware that it runs on. I hate to think of how many digital tapes there are stored about the world where there is no longer the appropriate hardware to read them.

Paul Richmond
IS Project Leader
L.D. McFarland Co
Post #442553
Posted Monday, January 14, 2008 3:56 PM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Monday, October 20, 2008 4:01 PM
Points: 147, Visits: 53
After reading many of Steve Jones' editorials I can see that Steve himself is suffering from the most basic of digital malaise viz relying on his spell checker. In todays editorial Steve has spelled struggling and later incorrectly. No big deal really but this is a document that is being distributed to 600,000 plus people around the world.

My old teacher once told me - "accuracy in spelling reflects accuracy in thought" - and the main issue with complaining about spell checkers is that the complainer himself has to be without sin so that the first stone can be cast.

If there are any spilling mistaeks in this post please let me kno.

Regards

Tim Brimelow

PS the spell checker missed the incorrect spelling of spelling above (I typed it as spilling which is a legitimate word)
Post #442745
Posted Monday, January 14, 2008 4:09 PM
Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Wednesday, April 16, 2014 1:11 PM
Points: 3,081, Visits: 11,230
Any one have a way to read the following?
1. 800 BPI 9 track reel-to-reel tapes formatted to hold punch card images.
2. 8 inch floppy disks formatted for a Honeywell DPS-6 mini-computer
3. Cassette tapes formated for a VIC-20
4. DLT-1 backup tapes for a VAX 3100
5. Paper tapes for an Altair

Post #442752
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse