Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
Log in  ::  Register  ::  Not logged in

Problems with Big Data

By Steve Jones,

Big Data is constantly in the news. We've been asked at SQLserverCentral to try and develop some articles, perhaps even a stairway to explain what Big Data is and how we might use it. I'm still trying to grasp the concepts myself, and unlike the amorphous cloud, I'm still looking for some good examples of what Big Data really is.

When I ran across this piece warning that Big Data isn't the final solution to all our questions in the world, I wasn't surprised. The piece notes that Google Flu hasn't been very accurate in its predictions of outbreaks. At first glance, this gives lots of credence to the idea that the good, solid data analysis and mining techniques we've used for years are just as good as any new Big Data fad.

However as I read more about the piece, it's not that big data and the analysis of large quantities of information is flawed, it's that a solid hypothesis matters. Researchers need to be willing to evolve their algorithms as they learn more about a problem. Probably they should also assume their algorithms are not correct until they've proven their ability to predict actual trends for some period of time.

We'll constantly be searching for ways to better interpret information and make better decisions. No new technology or product is going to magically solve our problems. Good solid understanding of the problem domain will continue to matter as much as the data itself.

Total article views: 197 | Views in the last 30 days: 1
Related Articles


It's been half a year since SQL Server 2005 was released and it's shown itself to be a fairly so...


Fundamentals of Storage Systems, Understanding Reliability and Performance of Solid State Storage

Solid state storage has come on strong in the last year. With that explosion of new products it can ...


Fundamentals of Storage Systems, Solid State Storage Basics

Solid state storage is the new kid on the block. We see new press releases every day about just how....


Solid State Replacements - Database Weekly (August, 18, 2008)

New SSD (Solid State Device) hard drives are infiltrating enterprise servers more and more, but are ...


Solid State drisk

We are planing to upgrade the hardware hosting our SQL server. Does someone have any information of...


Join the most active online SQL Server Community

SQL knowledge, delivered daily, free:

Email address:  

You make SSC a better place

As a member of SQLServerCentral, you get free access to loads of fresh content: thousands of articles and SQL scripts, a library of free eBooks, a weekly database news roundup, a great Q & A platform… And it’s our huge, buzzing community of SQL Server Professionals that makes it such a success.

Join us!

Steve Jones

Already a member? Jump in:

Email address:   Password:   Remember me: Forgotten your password?
Steve Jones