SQL Clone
SQLServerCentral is supported by Redgate
Log in  ::  Register  ::  Not logged in

Big Data: Cash Cow or Horsemeat?

By Dave Convery,

Last year, we all heard that we were going to have be ready for ‘Big Data’; that every business worth its salt would have a quant on its staff and an ever-growing Hadoop database spitting out the secrets of the universe into its sweat-slicked hands. Only that never really seemed to happen. Somewhere between the excitement of “We’re going to get all the data, and we’re going to ask it things, and it’s going to be brilliant” the reality of implementation caught up.

This article seems to suggest why – that like everything else worth doing, the whole ‘Big Data’ idea is incredibly hard to implement well. It’s not something that slots naturally into existing business practices, no matter how great you are – it needs specialists, just like any other large function. As another Wired article points out, just because you can gather and read this data, it doesn’t mean you’re doing something useful with it, or even interpreting it well. In the two companies in the article, Valve succeed at least in part because they hired specialists who really know how to look at the data, how to eliminate false positives, and crucially, what’s worth looking at and what isn’t. It’s not like Zynga couldn’t afford to hire good people, but they base their business on rapidly iterating their product based on the data they gather. If you ask the wrong questions or make false assumptions, you’re going to run into trouble quickly, and put time and effort into fruitless enterprises.

The truth is, if you want to find small patterns in big numbers, you need a vast amount of data, and you need someone smart enough to deal with it. You probably have smart people who work with a lot of data in your organization, but this is the sort of thing that’s going to make a maths PhD a very high-paying qualification to have among the companies that get it right.

You can improve a well-run and sustainable business with this sort of data analysis, but you can’t keep throwing things at the wall and hope you get enough data to turn them into something great.

Total article views: 144 | Views in the last 30 days: 1
Related Articles

Is the old stuff worth learning?

Often we are encouraged to learn the latest things, but sometimes the old stuff is worth knowing too...


Worth Reading – Something I learned while unemployed by Rod Falanga

SQLServerCentral recently published Something I learned while unemployed. Written by Rod Falanga, it...


Is the MVP Worth It?

The MVP award is coveted by many people and there are constant questions about how to get it. Is it ...


N Things Worth Knowing About SELECT

SELECT is this kind of Swiss Army Knife SELECT is our bedrock, our foundation, our now-and-forever ...


Standards Are a Good Thing

This week we have another article from Andy that discusses some changes he made at work in conjuncti...

big data