Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Access tasks in Data Flow Task dynamically Expand / Collapse
Author
Message
Posted Thursday, November 15, 2012 10:10 PM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Tuesday, October 29, 2013 12:10 AM
Points: 3, Visits: 181
Hi friends,

I have a package where the number of columns in DB tables could be changed. I want to change the meta data of data flow task dynamically. Also the mapping details will be in a seperate table as the source and destination column names need not be same.

I am able to show the newly added columns in the package but I am not able to map the columns in Data Flow Task.
Could anyone please help me on the same.

Thanks and Regards,
Reva
Post #1385464
Posted Friday, November 16, 2012 12:22 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Sunday, July 20, 2014 11:20 PM
Points: 13,252, Visits: 10,133
Out of the box, SSIS doesn't support changing metadata.
There are some 3rd party products, such as Dataflow+ of CozyRoc that can do this.

If you need to stay with native SSIS, then you'll need to resort to scripting.




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1385500
Posted Friday, November 16, 2012 1:25 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Tuesday, October 29, 2013 12:10 AM
Points: 3, Visits: 181
Thanks for the reply.
But can't we do the column mapping in C# and use Pipeline dlls?
Post #1385528
Posted Friday, November 16, 2012 1:28 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Sunday, July 20, 2014 11:20 PM
Points: 13,252, Visits: 10,133
That's a bit out of my comfort zone.
I know that some people regenerate the package dynamically every time, so that when it runs the dataflow is mapped as it should be.

But modify an existing package on the fly? I don't know.




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1385530
Posted Friday, November 16, 2012 1:32 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Tuesday, October 29, 2013 12:10 AM
Points: 3, Visits: 181
I am able to add the newly added columns in my package via C# but am not able to map the columns using my reference table in DB. Also here my Source and Destination column names can be different.
Post #1385532
Posted Friday, November 16, 2012 1:34 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Sunday, July 20, 2014 11:20 PM
Points: 13,252, Visits: 10,133
I know you can change the source and destination tables dynamically, but I have no clue how you would handle the changing metadata.



How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1385533
Posted Monday, March 4, 2013 12:40 PM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Yesterday @ 11:39 PM
Points: 1,015, Visits: 1,289
Koen Verbeeck (11/16/2012)
I know you can change the source and destination tables dynamically, but I have no clue how you would handle the changing metadata.


I am able to modify table names dynamically in data flow tasks. But, I need to change destination column names dynamically. How can we do that?


Thanks
Post #1426416
Posted Monday, March 4, 2013 12:55 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 3:25 PM
Points: 6,133, Visits: 7,175
There are things you can change easily in an SSIS package dynamically. The easy way to tell if you can is if they're in the list of properties under expressions for the source/destination or the datasource itself.

There are things you CAN'T change easily in an SSIS package dynamically, one of which is the metadata of the dataflow. The dataflow uses internal IDs based on the object that 'sources' the column to determine its type, value, and other items. There are third party objects out there that can allow you to have some variable source constructions, but even they are limited to having the same output results into the metadata (at least for used columns).

SSIS isn't meant to be that dynamic. In the same way that you need to write dynamic T-SQL to allow for major changes to queries, you need to write dynamic packages to account for variable metadata. This is nowhere near as easy as dynamic SQL however, and will require a pretty solid background in coding.



- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions | Forum Netiquette
For index/tuning help, follow these directions. |Tally Tables

Twitter: @AnyWayDBA
Post #1426422
Posted Wednesday, March 6, 2013 11:52 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Yesterday @ 11:39 PM
Points: 1,015, Visits: 1,289
Evil Kraig F (3/4/2013)
There are things you can change easily in an SSIS package dynamically. The easy way to tell if you can is if they're in the list of properties under expressions for the source/destination or the datasource itself.

There are things you CAN'T change easily in an SSIS package dynamically, one of which is the metadata of the dataflow. The dataflow uses internal IDs based on the object that 'sources' the column to determine its type, value, and other items. There are third party objects out there that can allow you to have some variable source constructions, but even they are limited to having the same output results into the metadata (at least for used columns).

SSIS isn't meant to be that dynamic. In the same way that you need to write dynamic T-SQL to allow for major changes to queries, you need to write dynamic packages to account for variable metadata. This is nowhere near as easy as dynamic SQL however, and will require a pretty solid background in coding.

I am able to update column names.


Thanks
Post #1427566
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse