p.s. Almost without exception, I load files to a staging table first so that I can validate data, check for duplicates, etc, yada, yada. With that in mind, I also usually just load all the columns in the file so that I don't have to mess with anything if someone suddenly wakes up and wants to use previously excluded fields. It also makes it so I usually don't need format files but, when I do, I can also write a little ditty that reads the first line as a whole row, does some analysis as to what the delimiters are, and then generate a BCP format file and the command to do the import just by running a proc with the file path provided.
Sounds good. My tables will be holding files for further validation. But some of them have over 200 fields, and I only need about 50 of them, so didnt want to manually enter all the fields into the table design.
My files actually do have a header (I had stripped them off for the test). Is there any way I can use the header field names to generate column names in a table easily ? (I would be happy with all varchar field types)
Yes. Just use a string splitter to do so once you've loaded the header row. Just hope that the header row has precisely the same delimiters as the body of the file.