Blog Post

New Microsoft SSIS transform

,

Microsoft has just released a new SSIS transform, called the SSIS Balanced Data Distributor (BDD).  You can download it here.

This transform takes a single input and distributes the incoming rows to one or more outputs uniformly via multithreading.  The BDD will split the data, but there is no way to know how it is deciding to split the data or to know which input records will go to which output.  So, the purpose of the BDD would be to split a bunch of input records into two outputs so you can, for example, have both outputs use a data conversion task that would run on separate threads, taking advantage of multi-processor and multi-core servers, thereby converting all the records up to twice as fast.  Then use a merge join task to combine all the records into one dataset again.  It is useful if you want to increase parallelism in the dataflow.

Boyan Penev has done an excellent comparison to show that the overall performance improvement is around 35-45% over a script component which splits the input stream in two.  And there is an excellent extended description about the BDD by Len Wyatt.

Another nice tool for your tool bag.

Rate

You rated this post out of 5. Change rating

Share

Share

Rate

You rated this post out of 5. Change rating