SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Zettabytes and Beyond


Zettabytes and Beyond

Author
Message
Miles Neale
Miles Neale
SSCertifiable
SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)

Group: General Forum Members
Points: 5262 Visits: 1695
Nadrek (3/21/2013)
Point of order: in 1983, 30 years ago, we were using 1.2MB 5.25" DS floppies, and 720KB 3.5" DS floppies.

1.44MB floppies didn't show up until 1986, so they're only 27 years old.

8" floppies, now, that's where it was at!


Nadrek - I agree completely. In the very early 70's I first saw a floppy disk. An 8 inch disk with no jacket just the disk itself was produced and put in the back of a highly modified 029 Keypunch machine as a prototype for key-to-disk entry. It was as if the world went crazy with the idea of unlimited storage at your finger tips. After a few months on the floor, the prototype had to be modified because people would fold the disk up and put it in the pocket, or staple it to the paper datasheets they entered in on the disk. Also it was so dirty that data was often lost.

Prior to this we were booting computers using decks of cards, with the new innovation of a floppy we were able to move to the "new age" of computing. only a short 40+ years back it was considered cutting-edge. Now sitting in the dusty corner of the Smithsonian people wonder what that stuff was used for. Once "Big Data" it is now old rusty techno-garbage.

History will repeat itself. I wonder what it sill be like in another 40 years. :-)

M.

Not all gray hairs are Dinosaurs!
Revenant
Revenant
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16587 Visits: 5049
"How much data is there in the world?"

A lot. In the Friday or Saturday evening in North America Xbox BI is getting 1TB of logs every 15-25 seconds. That does not include the data that the front end servers are streaming or downloading to the consoles.

That forces an entirely new way of thinking not only about BI but about data processing in general.

And that is only one system.
Eric M Russell
Eric M Russell
SSC-Forever
SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)SSC-Forever (43K reputation)

Group: General Forum Members
Points: 43597 Visits: 12043
Steve Jones - SSC Editor (3/21/2013)
In a practical sense, it doesn't matter if it's cat videos or anything else. We still need to manage the data, which means work for us in the data business. Businesses are finding more ways to capture data, arguably lots of which isn't useful or important, but we must still deal with it.

I know, and I appreciate all the opportunity it provides. I was just commenting that most today's databases have a low signal-to-noise ratio. Somtimes it seems we spend too much time thinking about how to manage and store data without considering how best to architect and optimze data.


"The universe is complicated and for the most part beyond your control, but your life is only as complicated as you choose it to be."
SQLRNNR
SQLRNNR
SSC Guru
SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)SSC Guru (100K reputation)

Group: General Forum Members
Points: 100254 Visits: 18616
Eric M Russell (3/21/2013)
Eric M Russell (3/21/2013)
majorbloodnock (3/21/2013)
What I'd like to know is how much the sum of human knowledge has grown in the same time.
. . .

Good point.

How percentage of total data consists of the following?
denormalized table design, improper data types, markup tags, duplicate data, replicated data, indexes (both used and not used), unfilled pages, table fragmentation, temp space, transaction log space, etc.

Even after eliminating all of the above, a huge percentage of actual data that's left consists of digital junkfood like email spam, cat videos, and pornography. There are even some companies that accumulate TB of data recording things like web page visits and mouse clicks.

I guess it could eventually becomes usable information for someone, but the total size of our collective databases are hardly a benchmark for measuring the ever expanding body of human knowledge.

I recall more than a few nights spent in front of the TV with Seinfeld and M*A*S*H, a shoebox full single sided 5.25 diskettes, notching a little hole in the corner of each one, and re-formatting them my new double density drive.


Wow, I had almost forgotten about doing things like that. I still have boxes full of diskettes of various sizes/density.



Jason AKA CirqueDeSQLeil
I have given a name to my pain...
MCM SQL Server, MVP


SQL RNNR

Posting Performance Based Questions - Gail Shaw

Jim P.
Jim P.
SSCrazy
SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)

Group: General Forum Members
Points: 2641 Visits: 2215
Just look at flash drives. I remember being given a couple 512MB drives buy a tech salesperson who obviously didn't know the costs. I looked it up, and at the time to get them with a logo cost about $20 a piece unless you were buying thousands. Then it dropped to about $15 a piece.

I recently purchased a 32GB drive in the form of a heart for a friend. It was $15.

But the flip side of all this story -- what will be left behind a hundred, let alone a thousand years from now? We can still decode the data left on the pyramids at Giza. Will anyone be able to still read the data left on your hard drives? Ermm



----------------
Jim P.

A little bit of this and a little byte of that can cause bloatware.
Miles Neale
Miles Neale
SSCertifiable
SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)

Group: General Forum Members
Points: 5262 Visits: 1695
Jim P. (3/23/2013)
But the flip side of all this story -- what will be left behind a hundred, let alone a thousand years from now? We can still decode the data left on the pyramids at Giza. Will anyone be able to still read the data left on your hard drives? Ermm


Thanks Jim, you made me think of Spock and Bones when they use to comment on how things were done in the 21st century. Some will look at today's data and laugh and others cry. One because they are amazed we were able to accomplish what we did with the simple tools we have now, and other will cry because of the mistakes we made and the diversions we took.

But they will look. Unless the Apes seal it all off from us and put it in the forbidden zone. :-)

Not all gray hairs are Dinosaurs!
Revenant
Revenant
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16587 Visits: 5049
Jim P. (3/23/2013)
. . . But the flip side of all this story -- what will be left behind a hundred, let alone a thousand years from now? We can still decode the data left on the pyramids at Giza. Will anyone be able to still read the data left on your hard drives? Ermm

Will anyone care what we left on our hard drives? Does anyone care what is punched in the cards that were processed 50 years ago and which are, in some cases, still in storage?
Jim P.
Jim P.
SSCrazy
SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)SSCrazy (2.6K reputation)

Group: General Forum Members
Points: 2641 Visits: 2215
Revenant (3/25/2013)
Will anyone care what we left on our hard drives? Does anyone care what is punched in the cards that were processed 50 years ago and which are, in some cases, still in storage?


What about the Diary of Anne Frank? What about the writings of MLK? What about the writings of the shyster politicians? What about the rest of books, movies, of popular culture?

A lot of it is waste of space, but it might not hurt to see history so that it isn't repeated.



----------------
Jim P.

A little bit of this and a little byte of that can cause bloatware.
Miles Neale
Miles Neale
SSCertifiable
SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)SSCertifiable (5.3K reputation)

Group: General Forum Members
Points: 5262 Visits: 1695
Revenant (3/25/2013)
[quote]Jim P. (3/23/2013)
Will anyone care what we left on our hard drives? Does anyone care what is punched in the cards that were processed 50 years ago and which are, in some cases, still in storage?



Yes, you bet. Historians love this kind of stuff. Just look at the excitement every time a new document is found from Bach, or Newton, or Picasso. I am not saying the some day the "Dead Sea Harddrives" will be something that changes history, but there are those who are interested in almost anything.

And not to tee up those who bad mouth the government funding odd or irregular projects, but if the US government will fund research to see how frisbees fly they will one day pay for someone to research old spinning disk technology or how Holllerith card readers really worked. And the feds will probably fund it to the tune of a hundred and fifty million over 20 years and not bat an eye at it.

Not all gray hairs are Dinosaurs!
jfogel
jfogel
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1409 Visits: 1183
Good point but the writings of Frank or King were one offs and of value from the start. Important information will migrate to new storage methods so it is accessible. I think it will be a rare occurance to find a masterwork on a 5 1/4.

Cheers
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search