Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 

James Serra's Blog

James is currently a Senior Business Intelligence Architect/Developer and has over 20 years of IT experience. James started his career as a software developer, then became a DBA 12 years ago, and for the last five years he has been working extensively with Business Intelligence using the SQL Server BI stack (SSAS, SSRS, and SSIS). James has been at times a permanent employee, consultant, contractor, and owner of his own business. All these experiences along with continuous learning has helped James to develop many successful data warehouse and BI projects. James has earned the MCITP Business Developer 2008, MCITP Database Administrator 2008, and MCITP Database Developer 2008, and has a Bachelor of Science degree in Computer Engineering. His blog is at .

New Microsoft SSIS transform

Microsoft has just released a new SSIS transform, called the SSIS Balanced Data Distributor (BDD).  You can download it here.

This transform takes a single input and distributes the incoming rows to one or more outputs uniformly via multithreading.  The BDD will split the data, but there is no way to know how it is deciding to split the data or to know which input records will go to which output.  So, the purpose of the BDD would be to split a bunch of input records into two outputs so you can, for example, have both outputs use a data conversion task that would run on separate threads, taking advantage of multi-processor and multi-core servers, thereby converting all the records up to twice as fast.  Then use a merge join task to combine all the records into one dataset again.  It is useful if you want to increase parallelism in the dataflow.

Boyan Penev has done an excellent comparison to show that the overall performance improvement is around 35-45% over a script component which splits the input stream in two.  And there is an excellent extended description about the BDD by Len Wyatt.

Another nice tool for your tool bag.

Comments

Posted by Anonymous on 16 June 2011

Pingback from  Dew Drop – June 16, 2011 | Alvin Ashcraft's Morning Dew

Posted by Jason Brimhall on 16 June 2011

This will be very handy.  Thanks for pointing it out.

Leave a Comment

Please register or log in to leave a comment.