I can get these results, can you explain what is wrong with them? I am trying to follow the conditions in your code, and this is what I get so...

What does this 'overnight refresh' process actually do?

Just to rule out a NULL issue, try importing a test file containing 5 or 10 rows, with no NULLS in those columns.

It looks like a NULL handling issue then ...

"I'm scraping data ..."

What do you mean by this?

Your query is checking for the existence of columns in tables, and has nothing to do with data.

Is TempparsedHeaderCSVData a table? Maybe...

To see the data interactively (and to do lots of other types of debugging), you need to run the package in Visual Studio.

You will be saving yourself a lot of...

No matter how closely I look at those two columns, with the data from row 1, I cannot see any issue whatsoever. You can prove that to yourself by overwriting...

• This reply was modified 6 years, 4 months ago by Phil Parkin.

Can you show us a couple of rows of actual data, indicating which are the new columns and which are those which import correctly?

Can you clarify what this means, please?

Seems like you are trying to bend Git to work in ways it was not conceived for. Maybe it's not the right choice in your organisation.

Just so I understand what...

DateCreated? DateModified? DateOfBirth? DateOfNextScheduledDentistVisit? ....

If it contains the date the row was created, you can do something like this

SELECT TOP (1000) 
col1, col2, ...
FROM tbl
ORDER BY <DateColumn>...

Why is the 'first table' being truncated? Best solution is to keep the data, or move it to an archive table, rather than deleting it.

If you delete it and you...

Firstly, I think it would be slightly more accurate to say that you are moving to using Git as your VCS, with BitBucket as your cloud-based code-management solution.

Is the dev...

Borrowing from the script posted earlier, you can do this by adding an extra code block near the end: