SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Big Data


Big Data

Author
Message
Henrico Bekker
Henrico Bekker
SSChampion
SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)SSChampion (13K reputation)

Group: General Forum Members
Points: 13577 Visits: 5336
Comments posted to this topic are about the item Big Data

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
This thing is addressing problems that dont exist. Its solution-ism at its worst. We are dumbing down machines that are inherently superior. - Gilfoyle
sherifffruitfly
sherifffruitfly
Mr or Mrs. 500
Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)Mr or Mrs. 500 (570 reputation)

Group: General Forum Members
Points: 570 Visits: 427
It'll go the route every other new shiny thing in tech goes, I would imagine:

1) There'll be a proliferation of home-rolled cobbled together codes to perform big data gathering, storage, access, and analysis.

2) Some bright CS/Math Ph.D. student and/or prof will prove some basic theorems along these lines regarding efficiency, integrity, and the like.

3) The programming world will coalesce around that foundational work.

4) Frameworks will begin to appear based that work, resulting in standardization of the topic.

4a) There will be non-stop religious wars of the form "zomg dotnet suxxors for this! clearly *nix is the only reasonable platform to do this on!".

5) People will say it's no big deal, and start looking for the next new shiny thing.

6) Throughout the entire process, there will be a group that nay-sayers the entire investigation.
Eric M Russell
Eric M Russell
SSC Guru
SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)SSC Guru (53K reputation)

Group: General Forum Members
Points: 53505 Visits: 12402
In the past most data orginated in a database because it was deemed important enough by users that they entered it manually (patient charts, student records, inventory, etc.).

Today, the bulk of "big data" is stuff like web click stream analysis, phone call records, highway traffic monitoring, serialized application objects, etc. Most of it (% of total volume) is derivative, duplicated, and mundane. We have to slog through gigabytes and terabytes of digital.. detritus.. to produce little nuggets of relevent business intelligence.


"The universe is complicated and for the most part beyond your control, but your life is only as complicated as you choose it to be."
Revenant
Revenant
SSCoach
SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)

Group: General Forum Members
Points: 19425 Visits: 5073
For better or worse, big data is my daily bread. We are staging over 150 million rows a day, which I think is "big." I have been warned that it will be 250 million a day before the end of this year.

Big data is not the future, it is the present.
Dave Poole
Dave Poole
One Orange Chip
One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)One Orange Chip (29K reputation)

Group: General Forum Members
Points: 29519 Visits: 3601
What I've found so far is that Big Data is a nice vague term so loved by IT vendors rather like "Cloud".

There is Big Data as in vast amount of data (either in number of records or in storage volume) and the challenges that go with that.
There is Big Data as in a big technical challenge to mine because it is not in a form that traditional RDBMS were designed to handle.

The latter required "Big Brains" to figure out how to get meaning from the verbal diarrhea and literary dysentry from social media sites.

then there is the combination of the two.

Some of the web analytics and click-stream data can be huge in terms of records but not necessarily as big as you think it would be in terms of TB.

As ever one of the hardest challenges is to get some meaningful requirements out of customers for the big data. They kind of know what they want but can't really articulate it.

If you have big data in the cloud then what sort of queries are you going to run against it? If it is analyze a humungous amount of data and bring back a simple and small answerset then you are in with a chance.
If it is bring back a large data set then that is going to sting a bit.

To make this work well or indeed at all database guys and storage/infrastructure guys are going to have to work so closely together people are going to think it is possible for Siamese twins to be born in different families!

LinkedIn Profile
www.simple-talk.com
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum







































































































































































SQLServerCentral


Search