sturner (2/22/2013)
Abu Dina (2/22/2013)
By the way, the ID column appears to contain a mixture of integers and GUIDs. Should I create a new Identity column on the big table and use that as my ID? Would that help?yes... and I would do the same on the destination table ... and make it a clustered index because you want to make sure all the inserts go at the end of the table.
Also, if you had an identity ID on the big table you could change the "WHERE NOT EXISTS (SELECT 1 FROM dbo.Loading_B2C_keys_ as b WHERE a.ID = b.ID)" in your inserts to "WHERE ID > @NextChunkID" instead. You would increment @NextChunkID by 1000000 (or whatever your top(N) is ) in each batch loop.
heh.. I saw your reply come in as I was about to post my previous one!
The new identity column seems to have done the trick. Although I'm still using the exists. Will try your method to see if I can get better performance.
---------------------------------------------------------
It takes a minimal capacity for rational thought to see that the corporate 'free press' is a structurally irrational and biased, and extremely violent, system of elite propaganda.
David Edwards - Media lens[/url]
Society has varying and conflicting interests; what is called objectivity is the disguise of one of these interests - that of neutrality. But neutrality is a fiction in an unneutral world. There are victims, there are executioners, and there are bystanders... and the 'objectivity' of the bystander calls for inaction while other heads fall.
Howard Zinn