Maybe you never thought you had a big database to manage or work on. I know my largest was in the 700GB range and I thought that was plenty big, preferring to work on smaller systems. I've seen people that worked on TB sized database and lately in talking with a few people on the SQL Server Customer Advisory Team (CAT), they're talking about 100s of terabytes and the issues that they deal with.
So many very large databases are warehouses of some sort. The Winter Corporation has a Top Ten program that tries to survey companies and find out who has the largest database. The latest results I've seen are for 2005, so I expect that things have grown since then. In that survey, the Land Registry had a 23TB OLTP database (DB2) and Yahoo had a 100TB data warehouse (Oracle).
There are some large databases listed here, but I'm sure some classified agencies (NSA, CIA, etc. as well as overseas counterparts) might have more data that they keep. However I saw an article recently that talked about the Library of Congress in the US and their problems with trying to archive a tremendous amount of data about the US and what happens to this country.
This article talks about having 500TB of data, though not all of that is necessarily database space. I've seen articles talk about smaller databases in the lower TB range for all this information.
Working in a library or archival environment brings all kinds of challenges, but perhaps none as overwhelming as the data growth that they experience. With so many types of media and information being produced as digital media, the explosion of data is completely overwhelming.
At least until you turn to the idea of deciding what to keep and what to discard. That's not a job that I would want.
Steve Jones
PS: If you're thinking of attending the PASS Summit 2008 in November, and you should, be sure that you use the "SSC" registration code when you register. We'll have a great opening night party you can attend and you get $100 off your registration cost. Until June 30, you save an additional $800 off the full price with early registration.
The Voice of the DBA Podcasts
The podcast feeds are now available at sqlservercentral.podshow.com to get better bandwidth and maybe a little more exposure :). Comments are definitely appreciated and wanted, and you can get feeds from there.
or now on iTunes!
- Windows Media Podcast - 25.6MB WMV
- iPod Video Podcast - 22MB MP4
- MP3 Audio Podcast - 4.5MB
Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.
I really appreciate and value feedback on the podcasts. Let us know what you like, don't like, or even send in ideas for the show. If you'd like to comment, post something here. The boss will be sure to read it.