I was talking with someone recently who was building a dynamic process that generated its own code at runtime to process ETL files based on a set of rules. It was an impressive system, and reminded me of how BIML can dynamically produce code and build packages.

However then I started to wonder. If you produce code at runtime that then runs, how well is that code tested? I suspect most people would test the code generation process. Is this producing the code I expect, but how many would then write additional tests for the generated code as it executes? I suppose people might also just test the generated code, but if that's the case, are they really testing the generation process extensively?

I'm not sure there's an easy answer, and not sure if there's actually a strong testing framework that exists to help here. As BIML becomes more popular and even ETL frameworks are used more and more in our software processes, I hope that we develop new testing methods to better ensure we are building quality code. I know some SSIS test frameworks exist, and there has been some writing on the topic, but I wonder how many SSIS developers bother to perform anything more than cursory tests? My suspicion is very, very few.

Ultimately I think we as an industry need to learn to build better software by improving our techniques, samples, skills, and testing.