• Steve Jones - SSC Editor (7/10/2012)


    jay-h (7/10/2012)


    SQLRNNR (7/10/2012)


    If you can show that there is data to support a decision, it is more likely to be well received than one based on the experience, instinct, or hunches of any person.

    I agree with that statement. Even though experience should have significant import, it's the numbers that matter - or so it seems.

    But experience can tell you when to trust the numbers, and how much weight to give them.

    As long as you look at the numbers. Too many people ignore them

    At least partially because they are too often better off ignored.

    Weather forecasts for example. Studies have been done that show that flipping a coin is more accurate.

    Decades of weather forecasts were analyzed for the simple measure, "If the forecast said x-percent chance of rain tomorrow, how often did it actually rain the next day?" For example, if it said 10% chance of rain, and it rained 15% of the time on the following days, then it was incorrect by 50%. If it said 50% rain, and it rained 45% of the days, then it was off by 10%. And so on. This was then measured against flipping a coin, "heads it'll rain tomorrow, tails it won't". The coin-tosses had better overall accuracy.

    Huge, well-funded industries depend on weather forecasts. Aggriculture, shipping, energy, to name a few. The R&D dollars are there. Vast amounts of data are gathered by local weather stations, satelites, and so on. And the accuracy is still lower than a random guess when taken over any meaninful period of time.

    Hurricane-season "predictions" are modified repeatedly throughout the year, and still end up wrong by the end of the year more often than not.

    Moving out of the weather, take a look at the US CBO. The non-partisan Congressional Budget Office. Math and finance wizards with more PhDs than you can shake a stick at. They have all the access to vast amounts of information. And, so far as I can tell, they've never been right yet about anything. They crunch huge amounts of data, they use the most widely accepted and practiced algorithms, and they're consistently off by huge margins, even when they aren't being made to load the numbers a certain way for some political agenda.

    On the other hand, insurance actuary tables are amazingly accurate, but only on large systems of data. X-people will likely die of Y cause in Z time period, given large populations with known smoking, eating, etc., habits. (Smoking and income-level being the two biggest factors, if I remember correctly. Gender might be right up there, I'm not sure.) Can't predict the age-of-death of any given individual with any accuracy, but that's not what they're for.

    Horse racing is another one where real odds can actually be predicted. And, of course, poker and blackjack give the odds to the house or they wouldn't be played in Vegas.

    So, you have to know whether you're dealing with something simple like risk factors associated with smoking, or winning poker hands, or with anything that involves complex systems, like weather, or systems that are very poorly studied and often intentionally muddied, like money+politics.

    But in many of these systems, ignoring the "data" is quite likely the best option. After all, if a coin toss is more accurate, then the data is beyond being simply flawed, and is actually likely to increase your chances of error.

    Keep in mind, though, that all of what I just wrote is written by a serious data-junkie (me), and may be skewed by my prejudices on the subject. (On the other hand, I've accurately predicted every US presidential election since I started noticing them in the '80s. So maybe I'm slightly more accurate than a coin-toss. Who knows?)

    - Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
    Property of The Thread

    "Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon