SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Data Decisions or Instinct?


Data Decisions or Instinct?

Author
Message
Chris Cradock
Chris Cradock
Valued Member
Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)Valued Member (73 reputation)

Group: General Forum Members
Points: 73 Visits: 86
Steve, the way you describe it I think you're confusing ideas with evidence.

Data is never going to tell you what to do directly. For that you'd need an idea.
Once you have the idea the next part is collecting the evidence to tell you its a good one.
If you have several 'competing' ideas the evidence can then tell you which one is the best of the bunch.

Without the story line the collected data is just that: collected data. (e.g.if you had a database of the location of every atom in the Universe it still wouldn't tell you a whole lot about the Universe).

Chris.
Eirikur Eiriksson
Eirikur Eiriksson
SSChampion
SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)

Group: General Forum Members
Points: 14928 Visits: 18591
From a slightly humours point of view, often there is quite a resemblance to Chinese Whispers in Data Collection, Processing and Analysis processes, which in turn makes one at least question any decisions or findings based on the "data".
Cool
RP1966
RP1966
SSC-Enthusiastic
SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)SSC-Enthusiastic (128 reputation)

Group: General Forum Members
Points: 128 Visits: 886
Chris Cradock (7/24/2014)

Data is never going to tell you what to do directly. For that you'd need an idea.
Once you have the idea the next part is collecting the evidence to tell you its a good one.
If you have several 'competing' ideas the evidence can then tell you which one is the best of the bunch.



The problem with ideas is that they are going to impose a narrative on the data. If you are not careful you can impose a false narrative by collecting the wrong data, after data collection you also have the same problem in that a false narrative can be imposed by selective interpretation of the data.

It can be the difference between:

'Can we get data to prove X?' and 'What is this data telling us?'

You said: 'Once you have the idea the next part is collecting the evidence to tell you its a good one.'

Often you are better to look for data that will prove your idea is a bad one - try to challenge your hypothesis rather than support it.
Steve Jones
Steve Jones
SSC Guru
SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)

Group: Administrators
Points: 61883 Visits: 19099
OCTom (7/24/2014)
Steve,

Does this really apply to data professionals? I have been a programmer and data provider since 1982 and I have never been asked to interpret the data. It was always someone else's job to do that. I provide the data, the reports, and the Excel spreadsheets. I think that it is dangerous for the same person who is providing the data to be making decisions based on it. I just have to make certain that the data is correct and has not been corrupted, and, that it is reported correctly.


Maybe some, maybe not others. I've often worked with business people to identify patterns and understand data. Sometimes it's a back and forth as I find business people I've worked with often haven't understood how to interpret large sets of data, or how averages, counts, distribution, and other statistical measures impact things. On the other hand, I don't always understand the importance or impact of a particular piece of data, and I can work with them to better understand how to structure things.

As an example, years ago (far too many, like 18), we were trying to determine how to plan for inventory of a few products. Demand can vary, but the salesman suspected some broad patterns. However it wasn't quite apparent from the data. I would look at patterns, go talk with them, get feedback on what they saw based on my patterns, and eventually we could make a decision on how to predict a portion of our demand (70% or so). Then the ordering people would need to apply their knowledge for the rest.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Steve Jones
Steve Jones
SSC Guru
SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)

Group: Administrators
Points: 61883 Visits: 19099
SQLRNNR (7/24/2014)

While stories are great, I think the story should complement the data. People should be able to use their judgement and make informed decisions and many times that may involve relying on the gut feeling.


Yes, though the gut needs some data support. Not 100%, but no 10% either. I'd hope for a preponderance of support, at least 60-70% supported by data.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Miles Neale
Miles Neale
Hall of Fame
Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)

Group: General Forum Members
Points: 3026 Visits: 1694
OCTom (7/24/2014)
Steve,

Does this really apply to data professionals? I have been a programmer and data provider since 1982 and I have never been asked to interpret the data. It was always someone else's job to do that. I provide the data, the reports, and the Excel spreadsheets. I think that it is dangerous for the same person who is providing the data to be making decisions based on it.


Tom, I agree that no decision should be made by the one who gathers and supplies the data without the work of others. However, in the decision process the one gathering the data is often crucial to the evaluation of that data. Any filtering, aggregation, normalization, and other valid or required manipulation of the data to become information needs to be explained to those pouring over it.

I have also been in the business for some time, and I have gathered both data and decision packages. I have often been involved in the decisions or at least in the discussions that led to the decisions. That should not be rare, but common.

But I have never been nor will I ever be the only one reviewing the data and making the decision. After decades of being in this and like business I have, like you, considerable knowledge of the processes, practices, validity, and quality of the data. When we were a rookies making a report, we followed certain instructions and did as we were told. Now after the years of experience I would hope that we are exercising the knowledge we gained concerning the business that our contribution is far greater, and our opinion considered. That is what we do.

M.

Not all gray hairs are Dinosaurs!
Miles Neale
Miles Neale
Hall of Fame
Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)Hall of Fame (3K reputation)

Group: General Forum Members
Points: 3026 Visits: 1694
Steve,

In reading both your post and the reference article, I find them both true and accurate. There are those who will look at data as the hallowed scrolls that should never be questioned. I remember years back someone saying "the, data, the whole data, and nothing but the data!" In addition, I think that was followed by "So Help Me Codd." However, that was long ago and about modeling.

However, it is not really just the data. Trends within the data, trends in the metadata concerning use and usability tend to help us think more of the process and what we are doing. Further, there is a knowing that something is not right, or trending in a different direction and things could be made better. This is not always a wild dream as some things are, but it often is some specific out of the box thinking that might just revolutionize what we are looking at.

We have to have the statisticians and gurus to maintain the status quo and advance it as far as it can be. Nevertheless, we must also have those who have the gut feel that it can be made better. Some of the time, you know in your knower that things need to changes, even if the data has not told the story yet. Your knower and experience can forewarn us that change is necessary to better meet the future that is headed your way.


M.

Not all gray hairs are Dinosaurs!
Steve Jones
Steve Jones
SSC Guru
SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)SSC Guru (61K reputation)

Group: Administrators
Points: 61883 Visits: 19099
Miles Neale (7/24/2014)
Steve,

In reading both your post and the reference article, I find them both true and accurate.


Thanks


We have to have the statisticians and gurus to maintain the status quo and advance it as far as it can be. Nevertheless, we must also have those who have the gut feel that it can be made better.


Agree. We need some collaboration between multiple disciplines and groups to make better decisions.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
lshanahan
lshanahan
SSChasing Mays
SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)SSChasing Mays (616 reputation)

Group: General Forum Members
Points: 616 Visits: 436
It should also be remembered that the methodology in gathering and analysing the data is as important as the data itself (Google Flu Trends anybody?).

____________
Just my $0.02 from over here in the cheap seats of the peanut gallery - please adjust for inflation and/or your local currency.
David.Poole
David.Poole
SSCertifiable
SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)

Group: General Forum Members
Points: 7476 Visits: 3282
RP1966 (7/24/2014)
In relation to this question I would recommend everyone read at least the first section of 'Thinking, Fast and Slow' by Daniel Kahneman.


+1

I see people using data to strengthen the case for something they believe in and not using data to as the source of those beliefs.

LinkedIn Profile

Newbie on www.simple-talk.com
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search