It's not for me, but who out there wants to work on the largest database? The NSA is collecting phone records
of calls made and brags that "It's the largest database ever assembled in the world."
If they're really recording all phone calls, then it has to be billions or rows. Heck, it's probably tens or millions or more a day! After all there are more than 200 million people in the US and with cell phones, home businesses, and prepaid calling cards, there's probably a good percentage of those people making a call a day.
Can you imagine having 10 or 20 billion rows of data that you want to index? How about testing a query that joins this table with a list of the people, a much smaller 200 million record table, and you forget the where clause? Or what about a genius manager asking for a list of people that didn't make a phone call between two dates? Probably have time for some nice coffee breaks there.
I've worked with smaller databases, 10s of GB or less, for most of my career and even on older hardware, things ran relatively fast. That includes backups, restores, broken queries, etc. Even cross joins of my largest tables might be annoying, but would run within minutes or an hour or two. It's also made me less concerned about some of the scale issues that the VLDB folks face, like maintenance windows, online operations, or even table recovery. In most of my jobs, if someone deleted a table, like me forgetting a WHERE clause, I could just restore the latest backup on another machine and then move the data across. Often I could even do this on the same server!!!
But that's not possible in a VLDB environment. Or even a large database. I had a 600GB or so database at JD Edwards and we didn't have space to restore that database onto another server to recover data. It would require a complete restore of the entire system. Fortunately it was a warehouse and could be reloaded without data loss if we ran into that type of situation.
VLDBs present some complex challenges to the simplest things that we take for granted. I can imagine that it's fun to work out solutions to some of the problems, like reindexing, defragmenting, and other tedious, mundane chores we take for granted on smaller databases. At least until something's broken.
I know it's not fun to try and fix something in that type of environment when you boss or his boss is watching over your shoulder.
Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com