SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


ssis data flow transform question


ssis data flow transform question

Author
Message
erics44
erics44
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1180 Visits: 739
Hi
I have one large table in one database and i am am going to move the data and turn it into a normalised relational database with a number of tables.

As all the data is coming from a single table is there a way of just importing it once (say into a recordset or something) and then running other data flow tasks from that recordset?

and is there a pause transform that will wait till one item is finished before starting?

Thanks in advance
dave-dj
dave-dj
SSCrazy
SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)SSCrazy (2.5K reputation)

Group: General Forum Members
Points: 2508 Visits: 1149
your best solution would be to create multiple dataflows in the control flow area of your package.

create your first dataflow to move the data to the single table on your new server.

the create the remaining data flows within a sequence container and use the control flow expressions to completetion to ensure your data flows then execute in the correct order .....

alternatively, you may just want to have a second data flow reading of the single table and use conditional splits to split the data into logic paths........

(I don't know your data or it's complexity but is an alternative). Hope that helps

_____________________________________________________________________________
MCITP: Business Intelligence Developer (2005)
Evil Kraig F
Evil Kraig F
SSCoach
SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)SSCoach (19K reputation)

Group: General Forum Members
Points: 19609 Visits: 7660
You might want to look into the use of .RAW files in SSIS. They're basically SSIS's version of temp tables. What they do is pull off a single system into a file (local to the SSIS server is best) in a high-speed access version of the data (you won't be able to read it directly).

That raw can then be manipulated repeatedly. My personal use for it is usually when dealing with multiple warehouses, where I need to do heavy transforms and then port the results to multiple targets. First I'll do all the transforms into the raw, and then start pumping it out to the target systems. The reason for this is so I can checkpoint the transformation, keeping me from having to hit the source system more than once in case of a connection failure or somesuch.


- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions | Forum Netiquette
For index/tuning help, follow these directions. |Tally Tables

Twitter: @AnyWayDBA
erics44
erics44
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1180 Visits: 739
thanks very much for the replies guys

ill look into both methods
Thomas LeBlanc
Thomas LeBlanc
SSCarpal Tunnel
SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)SSCarpal Tunnel (4.6K reputation)

Group: General Forum Members
Points: 4561 Visits: 913
You can also use a multicast to direct the recordset to multiple Destination objects in one Data Flow.

Thomas LeBlanc, MCITP DBA 2005, 2008 & MCDBA 2000
http://thesmilingdba.blogspot.com/
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search