September 14, 2008 at 1:46 pm
Hi All,
Can we load a single destination table from different sources removing the TABLE LOCK check option...
Does it really work...
Can we load million records (from 8 sources) like this parallely...
Any help on this?
Thanks
Thanks [/font]
September 15, 2008 at 5:58 am
Yes and no.
You can have several OLEDB destination components (or even SQL Destination components) loading into the same table at the same time. However, they may not be able to really load in parallel depending on the situation. Even with the table lock option turned off, SQL may decide to escalate locks to a table lock or they may get conflicting page locks. You are also likely to find that the disk sub-system cannot handle multiple writes to the same disk at the same time and one insert has to wait for another.
In these situations, it is better to use a UNION ALL component and put the data flows together before using a bulk insert operation. Bulk insert is really fast trying to turn it into some kind of parallel operation usually degrades performance.
September 16, 2008 at 4:18 am
My experience on this agrees with what Michael said. I had 16 feeds coming and and then tried to load these to the same table. At times it would run, but for the majority of the time, SQL Server was issuing locks on the table, which in turn led to the package failing.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply