July 13, 2015 at 8:02 pm
Comments posted to this topic are about the item Data or Experience
July 14, 2015 at 1:35 am
I've been hunting for the podcast from Netflix where they stated that they learnt the hard way to trust the data. They found that many of their intuited decisions were just plain wrong.
I work with a web user experience manager who has loads of examples where the customer result data contradicted the view of the designers of web pages and of the people making the decisions.
Also worth reading Daniel Kahneman "Thinking fast and slow". It's a real eye opener.
I think the important point is that if you want a system where decisions are driven by data then you have to treat data with respect. Data quality has to become everyone's responsibility and that message has to come down from the highest levels of an organisation. It's a cultural thing not a technological thing.
July 14, 2015 at 5:40 am
I always remind people that data doesn't necessarily provide answers, it gives direction. Even then normal human biases come into play and can affect interpretation. This is to say nothing about selection and confirmation bias, correlative vs causative effects, confounding variables and so forth.
Look up Tyler Vigen's website on Spurious Correlations for an hysterical but thought-provoking take on the above.
My signature project at work is a solution that captures production labor data primarily for billing purposes. Recently there's been an interest in deriving certain metrics from it. I have to constantly remind supervisors that just because a given production rate is low or high (or has increased/decreased) doesn't necessarily mean anything in and of itself. At best it is a flag to indicate maybe there is something to investigate further.
You can gather as much or as little data as you want but the meaning of that data is still determined by humen beings, with all the foibles and fallibilities thereunto pertaining.
____________
Just my $0.02 from over here in the cheap seats of the peanut gallery - please adjust for inflation and/or your local currency.
July 14, 2015 at 6:21 am
Data will find correlation with experience among other characteristics of the data. It's not really a choice between the two. There is no right or wrong. It's just that we rely on experience to be a key driver to success because it normally does have a strong correlation with being successful.
You have 10 years experience with SQL Server. You should be able to do everything I want with SQL Server. That's not always the case though as some of the veterans here have pointed out.
In my line of work, we find predictive analytics to be very helpful in pointing us in a direction, but not helpful in giving us the answer out the box. If we could predict the lotto numbers, that would be epic.
July 14, 2015 at 6:46 am
Excellent editorial! Focus on Sample size, data over time, and variance were well phrased.
These concepts are frequently lost in the demand to have a simple answer to a nuanced problem. A common problem with the business focus on a "Metric" or single number. I'm sure the stories are end less...
July 14, 2015 at 7:06 am
Correlation does not ensure causation.
Trend analysis is like a compass, it can help you go in the right direction, it does not save you when the road turns.
You still have to be watching what goes on around you.
The real estate bubble of the naughts was caused by investors using trend analysis and not actually inspecting the properties or loan applications.
They saw values go up all around it for several years and assumed the values would follow that trend for at least another quarter. The Chinese mathematician that pushed this process, took his speaking fees and went back to China when the bubble burst.
A lot of people made a lot of money, then EVERYBODY lost a lot of value.
Computers were supposed to level out the peaks and valleys. Instead the bosses still direct how tall the peaks should be, the computers just make the rush to the valleys faster.
July 14, 2015 at 7:18 am
lshanahan (7/14/2015)
I always remind people that data doesn't necessarily provide answers, it gives direction.
I fully agree.
July 14, 2015 at 7:22 am
One could can say that they work and make decisions purely from the data, but someone at some point in time had to put their interpretation as to what that data means and the kind of action one should take with it. That's where it gets all messed up. We don't disagree on the what the data is. But I say the data means this, you say it means that.
July 14, 2015 at 7:24 am
Great to see someone finally point this out. Now if only we can get people to listen. I predict that only 3% of the 11% of managers who listen to 1% of their senior employees will read at least 2% of this article though.
Dave
July 14, 2015 at 9:12 am
djackson 22568 (7/14/2015)
Great to see someone finally point this out. Now if only we can get people to listen. I predict that only 3% of the 11% of managers who listen to 1% of their senior employees will read at least 2% of this article though.
If only.... 😀
February 2, 2017 at 5:07 am
David.Poole - Tuesday, July 14, 2015 1:35 AM...They found that many of their intuited decisions were just plain wrong...
I think that this has been proven statistically. And it is counter-intuitive. Which surely makes it ironic.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply