Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 

James Serra's Blog

James is a big data and data warehousing technology specialist at Microsoft. He is a thought leader in the use and application of Big Data technologies, including MPP solutions involving hybrid technologies of relational data, Hadoop, and private and public cloud. Previously he was an independent consultant working as a Data Warehouse/Business Intelligence architect and developer. He is a prior SQL Server MVP with over 30 years of IT experience. James is a popular blogger (JamesSerra.com) and speaker, having presented at dozens of PASS events including the PASS Business Analytics conference and the PASS Summit. He is the author of the book “Reporting with Microsoft SQL Server 2012”. He received a Bachelor of Science degree in Computer Engineering from the University of Nevada-Las Vegas.

New Microsoft SSIS transform

Microsoft has just released a new SSIS transform, called the SSIS Balanced Data Distributor (BDD).  You can download it here.

This transform takes a single input and distributes the incoming rows to one or more outputs uniformly via multithreading.  The BDD will split the data, but there is no way to know how it is deciding to split the data or to know which input records will go to which output.  So, the purpose of the BDD would be to split a bunch of input records into two outputs so you can, for example, have both outputs use a data conversion task that would run on separate threads, taking advantage of multi-processor and multi-core servers, thereby converting all the records up to twice as fast.  Then use a merge join task to combine all the records into one dataset again.  It is useful if you want to increase parallelism in the dataflow.

Boyan Penev has done an excellent comparison to show that the overall performance improvement is around 35-45% over a script component which splits the input stream in two.  And there is an excellent extended description about the BDD by Len Wyatt.

Another nice tool for your tool bag.

Comments

Posted by Anonymous on 16 June 2011

Pingback from  Dew Drop – June 16, 2011 | Alvin Ashcraft's Morning Dew

Posted by Jason Brimhall on 16 June 2011

This will be very handy.  Thanks for pointing it out.

Leave a Comment

Please register or log in to leave a comment.