• I had to work on ETL system that read from a bunch of sources. After the data was imported in the staging table, I ran a bunch of set based validations and marked the records invalid and also kept track of of reasons why they were bad. After the validations were done, i imported only the good records and send an email with list of bad records and the reasons.

    Some of my set based validations involved calls to CLR function which made it slower but the granularity of the error report made it well worth it and it was 10-15 faster than the ETL process it replaced.

    Let me also add that this is not a run every 15 minutes kind of application but process data whenever the clients upload data.