• I once worked for a very large beverage company that also owned its own packaging (think bottles and cans) subsidiaries.   Everything actually worked very well and very fast.   Almost all of the major systems including warehousing of the product was performed by robotic systems that relied heavily on the databases to record data as it occurred. 

    Plant managers wanted to have access to the status of the plant at all times, and to facilitate that, they were given MS Excel and trained to load data into Excel from the same SQL databases that so diligently monitored production.  It was fairly simple for the managers to establish some charts and what-not, and then set the auto-refresh to every minute.  Everyone was happy.  Even the DBA's, because we had no idea that this sanctioned access to the database had been allowed and encouraged. This was at the start of a matrixed service support model at the company.  We were a small team of DBA's that covered a lot of ground but were treated like outsiders for the most part by the subsidiaries that used our services. We were not the masters of our domain when it came to the databases.  Instead we were shepherds and caretakers.

    Around this time, database performance suddenly went into the toilet, and this, in turn affected automated systems that were programmed to wait on database commits. Phone calls were made, DBA's were blamed, same old story.   After some quick investigation (this was using SQL 2000 and some SQL 2005), we determined that the connections from the plant manager's machine was causing considerable locking and blocking and effectively bringing the server to its knees.  As soon as we pointed this out, we were shouted down, called stupid, and told to keep looking, because this excel spreadsheet could not possibly have been the culprit.  And the managers liked having the almost real-time data.   This was the start of a long month or so of watching performance around the clock, noting the time when the excel queries hit the system (the manager's desktop only ran during business hours), and then killing the  queries to clear blocking and allow cans to be produced and tracked by the computer systems.   Occasionally we would go out to lunch,only to be paged because of database performance affecting plant output.  We would mention the excel connections.  Shouting would occur, our management would be called,  and we would be told to just make it all work.   

    This was all actually due to the way that the Microsoft JET engine copied data from SQL tables into Excel and also Access.   The JET engine transferred all of the data in the tables to the the requesting computer and then performed the queries at the destination.  If the destination computer did not have enough memory to handle these tables, then it would start to page data.   We had to show the documentation from Microsoft to management again and again and eventually, they brought in some real programmers to pull the data out of the databases  with .NET.     It was a long summer.

    So whenever I hear that Excel is connecting directly to a SQL database, I get a little tic that twitches in one eye for a while.