Too Good at Data Analysis

  • Slightly dated (May 2008), but it's a good look at some of the issues:

    http://www.thislife.org/Radio_Episode.aspx?episode=355

    Don't forget we've been having issues for months. It came to a head a few weeks ago.

    One thing about 24 minutes in, they used "old" data to make loans, meaning data from loans made to people that proved their income. Over the last few years they had "stated" loans, where you stated your income, you didn't prove it.

  • There were also NINJA loans: No Income, No Job, No Assets (collateral).

    Really now, who in their right mind could think these would ever be repaid, regardless of what the models predicted?

  • Steve Jones - Editor (10/14/2008)


    Short selling isn't stealing, and actually short sellers can help ensure the market doesn't grow too wildly. (http://blogmaverick.com/2008/09/30/the-stock-market-is-not-a-barometer/)

    Sorry, Steve, Just stating it isn't doesn't make it so. As I said, as far as I can see, short sellers 'borrow' the shares they use from 'long' investors and, in some case, make a profit without telling the share owners. The profit comes from the lost value of the shares owned, so effectively the short sellers have taken money from the share owner. The fact that they took a risk on it is irrelevant. This is still theft if the share owner wasn't aware of it. Of course, if they notify the shareholder and pay them a fee for the use of the shares that's different (the share owner is paying the short seller to take the risk), but that doesn't always happen.

    The problem is that we were allowing people to borrow on stock they didn't own to sell short.

    As I understand it, short sellers NEVER own the shares they sell short. If you sell stock you own and buy it back later that's just normal trading.

    The models being used were supposed to be proven by the market, which is just crazy. If you want to do that, then they have to be "test" models, not models on which you can base your business.

    Wish I could find a reference to the New Scientist article; it pointed to the collapse of one financial insititution a few years back which apparently proved that the most common economic models were wrong, but unfortunately no-one could devise better ones so they've continued to be used. People knew they didn't work well but didn't have anything better to base their decisions on.

    Derek

  • The whole thing boils down to the stock market just being insane. The problem with the model, more data or no, is that trading is based on the perception of how a stock is going to perform, which is not in any way guaranteed by past performance. Rumors/political stances/employee changes/strikes/wars are all unpredictable and affect stocks wildly.

    Hell, I heard on NPR this morning that the Japanese index had its highest one-day gain ever, because European and Asian governments pledged to infuse trillions of dollars into their banks. Not that they did it, just pledged it.

    I'm selling my stock at half the value. The guy next to me assumes I know something I don't, and starts selling his low just to get out of the way of a fall. The fire catches, and suddenly the stock price is down, with no basis on reality.

    Lovely system.

    Almost forgot to add my own $0.02 on statistics - statistically speaking, every American has one ovary and one testicle.

    ---------------------------------------------------------
    How best to post your question[/url]
    How to post performance problems[/url]
    Tally Table:What it is and how it replaces a loop[/url]

    "stewsterl 80804 (10/16/2009)I guess when you stop and try to understand the solution provided you not only learn, but save yourself some headaches when you need to make any slight changes."

  • I think we'll just disagree. I have never seen short selling as a problem. People sell you something, in essence a promise to get you the stock if you need it. You buy the promise that it will be there. The brokerage houses are really lending out the stock.

    Perhaps it's splitting hairs, and maybe it's a bad idea to have short sells just open for any length of time. Essentially they're a call option, just without a date.

  • jcrawf02 (10/14/2008)


    The whole thing boils down to the stock market just being insane. The problem with the model, more data or no, is that trading is based on the perception of how a stock is going to perform, which is not in any way guaranteed by past performance. ....

    Every market from real estate, used cars, stocks, grain futures, etc is based on perception, which in the short run is quite volatile. That in itself is not necessarily a problem.

    There does, however seem to be a tendency to confuse the stock market with the economy... two completely different things.

    ...

    -- FORTRAN manual for Xerox Computers --

  • The entire meltdown, crash, implosion or whatever you want to call it came about and could have been avoided but for 1 simple human principal; the 'Me First' principal. Where ones actions are always in accordance with what will benefit 'Me' regardless of how doing so affects others.

    There are of course some innocent victims like those who are in foreclosure because they lost their job and not because the bought more then they could afford. The majority however are all accomplice's to what happened from the Politician's to the Banks (and bankers) to even the consumer who purchased a mortgage they could not hope to pay.

    There's nothing bad about Capitalism or the free market when those involved behave like decent human beings. If everyone in our society moved some of their self-focus towards others and how to do for others, the world would be a far better place. I hate to sound like a peace hippy from the 70's but just imagine how much better off we would all be if instead of focusing inwards we all turned our attention outwards towards others. That way instead of each of us having 1 person looking out for us we'd have millions.

    The Sub-Prime Mortgage Participant Breakdown:

    The Politicians: For them the knowledge of whether this was a good idea (to force banks to give mortgage to high risk persons) is irrelevant because most are concerned only with power and how to keep what they got (which means getting re-elected thru vote buying actions) and how to acquire more.

    "No politician is too proud to take from the few and give to the many in order to secure their position as champion of the many".

    The Banks[ers]: For them it's not how to prevent the government from enforcing bad business practices that is the priority but how to make the most money possible out of the current situation regardless of the long term impact on others or on society or the world in general.

    "For the banker, the need to satisfy the pockets of shareholders is as strong and irresistible as the street junkies need for the next fix. Their only difference is that one is from the streets and is always searching for the next high while the other works for the street (Wall street) striving for that next stock price high."

    The Consumer: This group is the saddest of all because of all those involved they are the only ones who will actually have to pay for their sins. The politician and the banker have put in place safe guards along with a little "you scratch mine and I'll scratch yours" to ensure they come out as unscathed as possible. Even so, every person who took out a loan for something they could not afford, have only themselves to blame. Even if the bank lied to sell you own the loan, unless they did something like alter documents you signed it's hard to point the finger at anyone but yourself.

    If you work at the local grocery as a check out clerk and one day the bank says to you "Hey! Remember that $75,000 mortgage we turned you down for? Well how would you like one for twice that at a teensy-weensy higher rate with less money down and we'll even throw in a lower rate for the first X years so you have time to save up?", you have only yourself to blame for where you ended up.

    "For the average, everyday citizen, the allure of having that which is normally reserved for the privileged is so strong that it can cause the brain to shutdown while sending the wallet into over drive. After all, what if this once in a lifetime opportunity never comes around again?"

    Kindest Regards,

    Just say No to Facebook!
  • Let's not forget one of the biggest factors the various models forget to deal with...the power the model itself wields on the underlying system it was designed to monitor. By allowing lots and lots of people access to using that info, you end up in some sick observer's paradox. The models designed to predict problems end up causing the problem.

    IBM's revenue is off by a penny, which triggers an alert on someone's system, which tells EVERYONE, and next thing you know, IBM is in the tank, just because everyone acted on the same info.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • They knew the models weren't perfect...

    ... but the morons did it anyway. That bloody well sums it up and that's why "Business Intelligence" is an oxymoron. 😉 There's no intelligence on the business side of the data. :hehe:

    Worst case of "time to market" I've seen yet! 😛

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • Matt Miller (10/14/2008)


    IBM's revenue is off by a penny, which triggers an alert on someone's system, which tells EVERYONE, and next thing you know, IBM is in the tank, just because everyone acted on the same info.

    This was first shown as a problem in the late 80's when brokerages first set up fully automated computer systems which could monitor the market and issue buy/sell instructions based on some parameters. The human element (and associate delay) was effectively taken out of the equation so minor fluctuations got magnified as everyone's model told them all to do the same thing!

    A minor drop... the computers all say sell... nobody stops to think... the price hits the floor.

    And all the dealers who dumped stock early congratulate themselves on getting out before it crashed, ignoring the fact that by acting as a group they caused it!

    Derek

  • Derek Dongray (10/15/2008)


    Matt Miller (10/14/2008)


    IBM's revenue is off by a penny, which triggers an alert on someone's system, which tells EVERYONE, and next thing you know, IBM is in the tank, just because everyone acted on the same info.

    This was first shown as a problem in the late 80's when brokerages first set up fully automated computer systems which could monitor the market and issue buy/sell instructions based on some parameters. The human element (and associate delay) was effectively taken out of the equation so minor fluctuations got magnified as everyone's model told them all to do the same thing!

    A minor drop... the computers all say sell... nobody stops to think... the price hits the floor.

    And all the dealers who dumped stock early congratulate themselves on getting out before it crashed, ignoring the fact that by acting as a group they caused it!

    Perfectly stated...

    --Jeff Moden


    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.


    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • There are actually limits to that automated selling. Various tripwires will actually stop selling to prevent that from cascading across all stocks.

    I'm not sure there was a lot of forcing loans. There have been numerous interviews from lots of brokers that were making upwards of $50k a month, that they were always searching out new loans and offering them to people because of the profit. Even when there were soldiers and others that qualified for other loans.

    And consumerism is also to blame here, IMHO. Lots of people took easy money, without doing their due diligence, or not watching their own finances.

    It's a mess all around, and technology is in there. A little too much trust in the technology. Or in the people building it. As Jeff said, it really makes the whole BI game look like an oxymoron.

  • Matt Miller (10/14/2008)


    Let's not forget one of the biggest factors the various models forget to deal with...the power the model itself wields on the underlying system it was designed to monitor. By allowing lots and lots of people access to using that info, you end up in some sick observer's paradox. The models designed to predict problems end up causing the problem.

    ...

    You make a very critical point, and getting back to one of Steve's points, it looks like some times you can have too much (accurate) information. Many systems (especially markets) have the potential to go pathological unless there is the frictional component of incomplete and late knowledge.

    To take it to an extreme, insurance would not be possible if you could actually predict when you'd have an accident, or get sick.

    ...

    -- FORTRAN manual for Xerox Computers --

  • Steve Jones - Editor (10/15/2008)


    There are actually limits to that automated selling. Various tripwires will actually stop selling to prevent that from cascading across all stocks.

    My reading of the history was that the tripwires were introduced after the event.

    Another case where the business experts didn't think it through until after they'd tried it on the 'live' system (in this case the real stock market!) :w00t:

    Derek

  • jcrawf02 (10/14/2008)


    Almost forgot to add my own $0.02 on statistics - statistically speaking, every American has one ovary and one testicle.

    So THAT's why I keep getting email ads sent to me to enlarge anatomy which I don't have!

Viewing 15 posts - 16 through 30 (of 43 total)

You must be logged in to reply to this topic. Login to reply