There have always been those that seek to defraud, deceive or mislead their customers. In order to do this, someone inside an organization has always been willing to pressure employees to compromise their ethics and morals. Construction companies may use substandard materials or ignore standards for construction. Financial companies falsify performance records of products. Medical professionals may order unnecessary tests to charge more. Perhaps one of the common areas many people feel cheated is with used vehicles, where there are numerous stories of deceit from companies and individuals. I've had my own experiences with cars to support this.

It may be no surprise to you this is also happening in the digital world, which seem fundamentally more disturbing to me. After all, software is constantly changing in a way that many other industries don't. We can fundamentally rewrite the rules under which systems work by deploying new code, something many of us do on a regular basis. What's more, we have rules and regulations that fundamentally prevent us from disclosing how the systems work, limitations that don't allow for anyone to easily audit or evaluate what the programming might be doing.

This is different than a car, where we can examine the components and test them, or have a third party perform this for us. It's different than health, where we can solicit second opinions. Even construction has inspections from independent groups that can verify some specification is met. However, none of that exists for software. In fact, our industry resists remotely attempting to implement any sort of ethics by limiting access to source code.

There's an article that lists some of the issues developers have encountered. It also talks about teaching ethics as a way to reduce the impact, which is a good idea. But without any sort of accountability or transparency into the actual code, I'm not sure ethics will help much. There are always people that need a job, and may be willing to write code that continues to provide a paycheck because they aren't sure they can get another job. I know most of us would like to think we wouldn't succumb to pressure, but it can be hard to be sure what you'd do until you are confronted with a particular situation.

I do think this is a problem that will grow, and become more pervasive in the digital world, precisely because no one really knows what software is doing under the covers, and it's hard to even determine what might be happening if you could access the source code. After all, can you be sure that code you get is actually what is running? You can, but not easily.

What I'd like to see is some sort of framework that allows us to specify the behaviors of our software, along with the data movement and handling so that end users would have an understanding of how the software should work. Maybe some automated way of producing a "contract" based on the code. Then we could resolve disputes without ever needing to examine the source.