Dig Out the Root Cause

  • Spot on Steph!!!

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

  • I don't have a lot of good stories in my career about companies' dedication to performing root cause analysis and problem resolution.

    So many companies and CIO's I've worked for had a "good enough" is good enough attitude that permeated their I.T. culture. Nobody even cared about best practices but me. If root cause solutions were implemented, it was usually because I persisted at them, even in the face of other priorities assigned to me by management.

    If things seemed like smooth sailing and I talked about the rocks barely under the surface, I was viewed as negative and a worrier.

    Then, when the corporate ships hits those rocks, root cause analysis and solutions were rarely considered.

    Most companies I've worked for had a:

    > "all hands on deck" approach to any problem until the fire was (temporarily) put out,

    > "fighting fires from the top" mentality.

    The first rule of fire fighting is to fight fires from the bottom, never the top. It's much more effective and resource efficient.

  • Gail Wanabee (4/28/2015)


    ...

    So many companies and CIO's I've worked for had a "good enough" is good enough attitude that permeated their I.T. culture. Nobody even cared about best practices but me. If root cause solutions were implemented, it was usually because I persisted at them, even in the face of other priorities assigned to me by management.

    If things seemed like smooth sailing and I talked about the rocks barely under the surface, I was viewed as negative and a worrier.

    ...

    The first rule of fire fighting is to fight fires from the bottom, never the top. It's much more effective and resource efficient.

    +100 for sure.

    I'm fortunate that my current upper management understands the value of root cause analysis and wants us to do MORE of it. 😀


    Here there be dragons...,

    Steph Brown

  • Gary Varga (4/28/2015)


    I am certain that as an industry that we reduce our productivity over the longer term due to ignoring root cause analysis or, sometimes more frustrating, ignoring the results of root cause analysis.

    Yeah, I'm sure this is true everywhere, but it's sad. There's never time to do a root cause and if there is, there's never time to act on it. :crazy:

  • As Eric Russell observes, root cause analysis is often a political issue.  In an organization where mistakes are punished, no RCA will be productive.  Everyone will be too busy avoiding discussion and claiming that their part works.  Similarly if IT has a lower status than (for example) Sales, pointing out that bad data came from Sales may be politically difficult.  The best approach in a siloed organization is finding a way to show a rival silo that improving their data (or changing a process somehow) helps them.

    When there are no other teams involved, it's easier.  However, it should still be done on a quiet day when people aren't feeling that they should be doing something more important.

    • This reply was modified 4 years, 9 months ago by  larry.blake.
  • Unfortunately, "good enough" is often good enough, even if it does result in a bit of downtime.

    This attitude was always the most disappointing aspect of my years in IT.  It was especially rampant in the last large shop I worked in.  On the other hand, downtime was taboo.  The so-called Project Managers were too timid to allow updates to fix issues that hindered getting things done and produced invalid data.  If it ran, didn't matter if it was valid.

     

    My inclination was always to first fix the problem,  then, if it was possible, fix the bad data.  I always considered bad data to be worse than no data at all.  If data was not 'fixable',  at least you could inform users that it was unreliable.

     

    • This reply was modified 4 years, 9 months ago by  skeleton567.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • It is indeed true that some companies are so organised that chasing root causes is a good way to get side-lined or fired.  However, in other companies it sometimes happens that the technical staff are 100% behind chasing root causes because they are fed up with taking the blame for repeat problems they had been forbidden to fix.  That can lead to some quite amusing side-effects of attempts to punish someone for doing the right thing - the designated victim is promoted to a higher pay-grade and replaced as manager of what caused the dispute by someone equally sure to want to do things right, but with the right company political background, not just a manager with a developer background. But of course the one particular issue that triggered this now has to be done wrong, not right, and the idiot who decided to make trouble gets the blame for the extra expenses entailed by doing it wrong instead of right once the Finance Director looks at the damage.

    Tom

Viewing 7 posts - 16 through 21 (of 21 total)

You must be logged in to reply to this topic. Login to reply