I have been trying to use PS to load text files to SQL using the script from article "Import tricky CSV's". The script is very good
but the speed seems to be the issue. Sometimes you have to give up speed for accuracy and field parsing since data is not always
clean which that script addresses nicely.
Issues I have found with script processing ETL
I like the idea of having ETL in a script, but many of the examples I have seen speed always seem to be an issue. I have files
that can range from 50 records to millions, and row size from 10 fields to 200. I have used SSIS, but sometimes not flexible when you have to change
the criteria or add\remove fields.
would like to hear\see other solutions people are using to accomplish file loading to SQL