August 2, 2012 at 8:19 am
Scenario
Used SSIS bulk insert to loop through a series of flat files and load them into a database.
The package returned success on all files loaded into the database.
Ran a count of rows per file loaded. A C# program created by a co-worker was created to verify the count in the actual raw file to the database count. The program starts the count off @ -1 to discount the file header. When comparing the results in the database to the raw files. Some of the counts in the database where 1 row less then the raw file. About 10% of the results were this way. The filecount of the raw files checks for nulls in the file.
I do not have access to the log for the job because I did not put in an increment every hour into the log creation. I do not have access to the data files because of the sensitivity of the data. I cannot think of an explanation at this juncture. Any suggestions.
Marshall
August 4, 2012 at 10:00 pm
Perhaps header row is also inserted as data row into tables.As that C# program begins from -1 there'll be 1 row difference.
August 5, 2012 at 1:34 am
Jonathan Marshall (8/2/2012)
ScenarioThe filecount of the raw files checks for nulls in the file.
Marshall
I don't think counting a null would be an issue , It depends how are you counting a row.As null counting is ignored by sql server on a column basis. The c# program may have been counting based on row breakers , probably the new line, may be one of your row is not inserted .If you are having ignore on failure marked, then , i think the C# code may have been adding something based on the raw file data structure.
~ demonfox
___________________________________________________________________
Wondering what I would do next , when I am done with this one :ermm:
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply