• We can all jump through flaming hoops to accomplish our ETL. However, there are limits on what we can or should accept as input. If the data format can't be parsed with consistent business rules that can be coded in T-SQL or C# or VB.Net, then we really can't process the file. Columns have to either be fixed-length or have a well defined delimiter. Same with row termination. It needs to be something that we can parse with the tools at hand.

    We have powerful tools, bcp, bulk insert, SSIS and others, but they can't guess at what the data is supposed to be. They have to be told what the data is supposed to be, and then when it isn't we must handle the exceptions.

    IMHO, It is acceptable to go back to the people who own the source data and say to them that a data format is unacceptable for specific reasons. Otherwise we need to hire a roomful of 10-key data entry clerks. Remember those days?

    Brandon_Forest@sbcglobal.net