Cascading Human Error

  • Revenant (5/3/2011)


    Eric M Russell (5/3/2011)


    Steve Jones - SSC Editor (5/3/2011)


    Eric M Russell (5/3/2011)


    It is possible for a database application (not using AI but just basic SQL ranking, fuzzy indexes, and reference tables) to return the correct result for the following questions:

    "Who is the president of England?"

    "List all customers with the last name Edwerds"

    There was a feature called English Query that was supposed to do this, but it didn't work well.

    I never used it, but my impression was that English Query was just a tool that parsed engligh phrases into a valid SQL select statement.

    Knowing that "president" ~= "prime minister" or soundex("Edwerds") = soundex("Edwards") requires cross-reference tables and function based indexes. Google's search engine does this routinely. It won't assume to know exactly what you're talking about but will instead rank what it considers possible matches. For example, if the user enters "Mary Edwerds" AND 36052, it will return "Mary Edwerds" first (if any), followed by "Mary Edwards" in 36052, followed by "Mary Edwards" not in 36052.

    I am not sure about Google, but based on history of the previous searches, Bing would assume and offer 'president of [The Bank of] England'.

    There are so many questions asked on the web by so many people that even right answers for commonly phrased wrong questions are indexed. When I did a search for "who is the president of england", ranked at the top was a link to the same question on a website called Ask.com, and someone had correctly pointed out that England has a prime minister not a president.

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • Eric M Russell (5/3/2011)


    . . . <big snip> . . .

    There are so many questions asked on the web by so many people that even right answers for commonly phrased wrong questions are indexed. When I did a search for "who is the president of england", ranked at the top was a link to the same question on a website called Ask.com, and someone had correctly pointed out that England has a prime minister not a president.

    Was this on Bing or on Google?

  • Revenant (5/3/2011)


    Eric M Russell (5/3/2011)


    . . . <big snip> . . .

    There are so many questions asked on the web by so many people that even right answers for commonly phrased wrong questions are indexed. When I did a search for "who is the president of england", ranked at the top was a link to the same question on a website called Ask.com, and someone had correctly pointed out that England has a prime minister not a president.

    Was this on Bing or on Google?

    When I type "who is the president of england" on Google, I get the following as the first search result:

    Answers.com - Who is the president of EnglandEngland question: Who is the president of England? England does not have a president, but rather a prime minister. England's Prime minister is David ...

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • While we are on the subject of cloud outages and human error, here's vmware's recent woes.

    April 26, 2011

    One of the action items from the previous day’s partial outage was to develop a full operational playbook for early detection, prevention, and restoration should our systems fail to properly handle any sort of intermittent loss of connectivity to storage. At 8am this effort was kicked off with explicit instructions to develop the playbook with a formal review by our operations and engineering team scheduled for noon. This was to be a paper only, hands off the keyboards exercise until the playbook was reviewed.

    Unfortunately, at 10:15am PDT, one of the operations engineers developing the playbook touched the keyboard. This resulted in a full outage of the network infrastructure sitting in front of Cloud Foundry. This took out all load balancers, routers, and firewalls; caused a partial outage of portions of our internal DNS infrastructure; and resulted in a complete external loss of connectivity to Cloud Foundry.

    http://support.cloudfoundry.com/entries/20067876-analysis-of-april-25-and-26-2011-downtime

  • dallas-1069889 (5/3/2011)


    While we are on the subject of cloud outages and human error, here's vmware's recent woes.

    April 26, 2011

    One of the action items from the previous day’s partial outage was to develop a full operational playbook for early detection, prevention, and restoration should our systems fail to properly handle any sort of intermittent loss of connectivity to storage. At 8am this effort was kicked off with explicit instructions to develop the playbook with a formal review by our operations and engineering team scheduled for noon. This was to be a paper only, hands off the keyboards exercise until the playbook was reviewed.

    Unfortunately, at 10:15am PDT, one of the operations engineers developing the playbook touched the keyboard. This resulted in a full outage of the network infrastructure sitting in front of Cloud Foundry. This took out all load balancers, routers, and firewalls; caused a partial outage of portions of our internal DNS infrastructure; and resulted in a complete external loss of connectivity to Cloud Foundry.

    http://support.cloudfoundry.com/entries/20067876-analysis-of-april-25-and-26-2011-downtime

    If they only had a dog...grrrrrrrr 😉

  • Unfortunately, at 10:15am PDT, one of the operations engineers developing the playbook touched the keyboard. This resulted in a full outage of the network infrastructure sitting in front of Cloud Foundry.

    When they say the engineer "touched the keyboard", I assume what they mean is that he entered some series of commands that brought down the network.

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • And we'll always have those errors. Cables pulled out of servers, something dropped, something spilled.

  • The public perception of computer systems is that they are mindless robots who interpret commands from operators literally and then follow through with them relentlessly, even when the operator realizes he made a mistake. There is actually a lot of truth to that perception.

    Reminds me of a rhyme from one of my college computer science professors:

    I truly hate this damn machine

    I really wish they'd sell it

    It never does quite what I want

    But only what I tell it.

Viewing 8 posts - 16 through 22 (of 22 total)

You must be logged in to reply to this topic. Login to reply