I work for a software consumer, as opposed to a development house. We are not happy with the quality of deployments in the health care industry. Let's skip software quality for now and stick to the point.
Let me start with saying that we have some vendors that do an exceptional job. Some of our vendors do remote installs and upgrades, some come on site, and the best of them tend to do very well with their product. Some of them even have some understanding of third party tools that their application requires. Kudos to those and other companies that take the time to do the best they can.
We also have vendors that absolutely need to find another way to do things. In some cases, everything about their process is flawed. In other cases, what is usually done very well, fails in cases where things are slightly out of norm. The industry has an issue with this, and it needs to be fixed. Below are some examples of things I have seen.
A vendor is doing an upgrade on one of our systems. The old database is SQL 2000. The new database is SQL 2008. They provided instructions for SQL 2005. The requirements they provided for the servers were wrong, and 75% of them had to be rebuilt. They uninstalled SQL without permission and later restored a test database over future production, with the expectation that it will be used going forward as a test database. Guess we won't ever need a production database...
Another vendor typically does excellent work. Previous upgrades have gone very smooth, with any issues that came up being within what I feel are acceptable limits. This last upgrade was one of the worst I have ever experienced. Things that were outside their control probably led to most of the issues, for example the birth of a baby a week before the upgrade, but the lack of processes to cover this type of issue is unacceptable. People get sick, and replacements need to be able to jump in and hit the ground running. Documentation of the customer's system isn't hard to create, and can go a long way to eliminating issues when replacement staff must take over. Some of the other issues were simply due to mistakes being made, but I don't know that they would have occurred had there not been a change in team members so close to go live.
Part of the issue here is that companies are always trying to do more with less. Increasing the profit margin by reducing staff can only go so far before this leads to costs on the other side that end up in a loss of revenue and reputation. Unwillingness to hire people with the knowledge required, and unwillingness to train those that are employed, is leading to an environment where mistakes occur with regularity, and in some cases are extremely costly.
Fixing these issues is larger than a single developer, but we can point out where weaknesses exist, and we can improve our own processes in the hope of reducing their impact on the overall environment.