A New World of Data

  • Comments posted to this topic are about the item A New World of Data

  • As I recently pointed out, even though mass storage is pretty inexpensive, there is also the consideration of time and effort spent with backups and validation and such that goes along with the whole thing.  I think that data architects need to include in their efforts ways to summarize and archive data even offline such that it can be retrieved effectively IF AND WHEN needed to get to the detail.

    Massive storage of detail may be of value in the case of audits and such requirements, but is hardly of much value as it ages and can be more meaningful for many needs if well summarized.

    Offline storage of detail can be so much more cost-effective if it is properly designed and managed.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • I think it's funny... way back when a 1GB disk was the "ultimate", people filled them up.  Now we have easy access to inexpensive disks that measure in TeraBytes and, yet, people still manage to fill them up.   In both instances, they're usually filled up with junk that no one will ever use a year from now but no one will delete it even 5 years from now.

    Same old problems for the last 20 years... the only two things that have changed are the sizes of the disk and the ratio of absolute junk data that people keep compared to what is really needed.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Maybe it's time for a new version of DVD-RW about the size of the old LP records, and a drive the size of a turntable.  You could get lots of those DVD-RW's in a 5-drawer lateral file cabinet and it could be indexed and automated to mount and dismount.  My old LP turntable would hold ten at a time decades ago and then I had a Sony CDP-CX400 rotary CD player that held 400 at a time.

    • This reply was modified 4 years, 3 months ago by  skeleton567.
    • This reply was modified 4 years, 3 months ago by  skeleton567.
    • This reply was modified 4 years, 3 months ago by  skeleton567.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • skeleton567 wrote:

    Maybe it's time for a new version of DVD-RW about the size of the old LP records, and a drive the size of a turntable.  You could get lots of those DVD-RW's in a 5-drawer lateral file cabinet and it could be indexed and automated to mount and dismount.  My old LP turntable would hold ten at a time decades ago and then I had a Sony CDP-CX400 rotary CD player that held 400 at a time.

    Oh my... too funny that you brought that up.  I was working for a DOD company back in the 80's and early 90s and their document library was stored on a large array of what they referred to as "optical storage" and the plastic disks they used were actually the same size as an LP, as you've described.  I don't recall how much they could hold but I don't believe that it came anything close to what a DVD can.

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Jeff Moden wrote:

    skeleton567 wrote:

    Maybe it's time for a new version of DVD-RW about the size of the old LP records, and a drive the size of a turntable.  You could get lots of those DVD-RW's in a 5-drawer lateral file cabinet and it could be indexed and automated to mount and dismount.  My old LP turntable would hold ten at a time decades ago and then I had a Sony CDP-CX400 rotary CD player that held 400 at a time.

    Oh my... too funny that you brought that up.  I was working for a DOD company back in the 80's and early 90s and their document library was stored on a large array of what they referred to as "optical storage" and the plastic disks they used were actually the same size as an LP, as you've described.  I don't recall how much they could hold but I don't believe that it came anything close to what a DVD can.

    I never worked with them myself, but recall the "optical storage" days too.  Assuming those LP-sized disks had the same density as a DVD-RW, they would be some pretty cheap bulk storage.

    I've never tried it, but can you put a SQL Server db on a DVD-RW and then detach and store it for when you want the data?  Waiting for slow retrieval and processing might be an alternative for seldom needed data.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • Just looking back at this old article and responses, I will throw out that my home network has now grown to 34TB of storage on two 4-drive NAS devices and two docking stations for SATA and USB drives besides five computers, and keeps expanding as I digitize over 100 years of family photos from B&W, to 35mm slides, thousands of digital camera pictures, and a music library of 73,000 tracks.

    Gone are the days of what we considered 'permanent' offline storage.  Now we have to plan on at least double our needed capacity just to keep safe single backup.   Just yesterday the machine with my 1TB of digital music failed to boot up due to what seems to be a failed power switch.  Fortunately, I have two digital backup copies, but even those are still on mechanical drives.

    One question is should I absorb the expense of repairing a six-year-old machine, if parts are available, for the music library or just buy a new one.

    And this all reminds me of the days (1960's-1970's) of the old 9" reels of magnetic tape on the 6-foot high/3-foot wide tape drives when we mounted tapes on two drives and did tape-to-tape copies merging in new historical data from very limited disk storage on a single-task computer.  The issue then was how much time could you justify for the data merge versus having to design around data being stored in multiple off-line locations.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply