I ran across an interesting piece in Wired recently that talks about algorithms and the power they hold in the modern world. I don't think it's a great piece, and it starts to mix code, data, algorithms and other aspects of modern software in a way that seems disjointed to someone that works on software. I think there is come confusion about how data and software differ.

I don't deny that there are potential negative impacts of the ways in which we process data. Certainly we have bugs in software that cause issues, but we also end up with implementations of specifications that may not quite match up with the expectations of our clients. Even in simple applications, if I'm not constantly interacting with a client in a detailed way, I find that the software can operate in a subtlety different manner than others expect. Of course, there are certainly people that deliberately program systems to intentionally help, or hurt, others, but I think we have just as many issues from small communication mistakes. That's one reason why DevOps, CI/CD, and frequent releases are valuable. We get feedback quicker on the implementaiton of the design.

Does it make sense to disclose algorithms? Make source code available for examination? I definitely think there are intellectual property issues to consider, but when there are legal challenges or arbitration, I would expect that the actual code of a system can be examined. We definitely need to ensure that code examined in these cases is not part of the public record. However there needs to be open examination in such a way that the groups involved, and perhaps even other interested parties, have confidence that the algorithm performs as expected.

This becomes easier in databases, since our code is often easily accessible, but more complex in applications. Many of us realize that subtle changes in application code can cause issues for data, which is part of the reason we like declared referential integrity in our databases. That way we're not affected if an application makes a mistake in enforcing our data integrity rules.