Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

SSIS 101: Isolating Data in a Data Flow Expand / Collapse
Author
Message
Posted Wednesday, February 11, 2009 12:16 AM


Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Wednesday, April 16, 2014 1:39 PM
Points: 388, Visits: 1,034
Comments posted to this topic are about the item SSIS 101: Isolating Data in a Data Flow

Andy Leonard
CSO, Linchpin People
Follow me on Twitter: @AndyLeonard
Post #654486
Posted Wednesday, February 11, 2009 6:05 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Friday, March 28, 2014 5:06 AM
Points: 1,204, Visits: 915
Hi Andy,

Nice article but afraid I lost you along the way. You know, where you started with the Union All's and the rest, I still have the red error mark on my ContactStage but don't see where you handled that part. Excuse me if I am silly or stupid but can you help me out here please. I am busy training myself in SSIS and your aticle came on the right time.


Manie Verster
Developer
Johannesburg
South Africa

I can do all things through Christ who strengthens me. - Holy Bible
I am a man of fixed and unbending principles, the first of which is to be flexible at all times. - Everett Mckinley Dirkson (Well, I am trying. - Manie Verster)
Post #654705
Posted Wednesday, February 11, 2009 6:32 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 7:08 AM
Points: 10,910, Visits: 12,549
Nice article Andy. I've never used the multicast task and didn't have an idea why I'd use it, but this is a good way use for it in my situation. I've always edited my source query with a where clause to check out problematic or specific rows.

This is another tool in the toolbox. Thanks.




Jack Corbett

Applications Developer

Don't let the good be the enemy of the best. -- Paul Fleming

Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
How to Post Performance Problems
Crosstabs and Pivots or How to turn rows into columns Part 1
Crosstabs and Pivots or How to turn rows into columns Part 2
Post #654731
Posted Wednesday, February 11, 2009 7:53 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Thursday, December 30, 2010 9:54 AM
Points: 4, Visits: 9
Here is where Andy tells how to correct the target information:


This scenario occurs with enough frequency to justify the Restore Invalid Column References Editor, shown above. The ContactID column with Lineage ID 32 is shown as invalid, but the References Editor located (and suggests) a column with the same name and data type: the ContactID column with Lineage ID 333. In fact, it’s located new columns for all the invalid references (although the error did not report these additional columns).

Click OK to replace the invalid references with the suggested columns

You have several options in the Restore Invalid Column References Editor. One of them is to let the editor try to restore the invalid links but I usually highlight all of the rows listed and click on the delete invalid rows option and hit "Apply" (there is no OK button). This will close the editor and remove the invalid references. When you reopen the target again everything should fix itself and you are on your way.
Post #654838
Posted Wednesday, February 11, 2009 8:52 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Thursday, March 13, 2014 10:40 AM
Points: 1,035, Visits: 408
Nice article Andy, I never really understood what the data viewers were. I sure wish I'd had this information a year ago...


/*****************

If most people are not willing to see the difficulty, this is mainly because, consciously or unconsciously, they assume that it will be they who will settle these questions for the others, and because they are convinced of their own capacity to do this. -Friedrich August von Hayek



*****************/
Post #654934
Posted Wednesday, February 11, 2009 9:02 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Tuesday, February 18, 2014 7:14 AM
Points: 1,344, Visits: 1,983
Great article. I too am having to do more and more work with SSIS and, naturally, don't have time/budget to go on training hence learning about the tools described here is very helpful.

Thanks.


Derek
Post #654949
Posted Wednesday, February 11, 2009 9:49 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Thursday, April 17, 2014 3:28 PM
Points: 1,343, Visits: 7,164
I like the Data Probe idea. That will come in handy for sure.

Thanks,
Burke


bc
Post #655023
Posted Wednesday, February 11, 2009 10:01 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Thursday, October 03, 2013 8:20 PM
Points: 223, Visits: 83
Hi Andy,

Great intro article on SSIS! I've been working with it for over two years, but had never tried anything similar to your "data probe" technique. It makes perfect sense.

One thing I saw in this article that's a little different is the Multicast as sort of a dummy endpoint. I've often seen a Rowcount as a dummy endpoint but never a multicast. Does the endpoint actually do anything in either case in this type of scenario?

Cheers,

-Matthew



Post #655036
Posted Wednesday, February 11, 2009 10:14 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 7:08 AM
Points: 10,910, Visits: 12,549
mcloney (2/11/2009)
Hi Andy,

Great intro article on SSIS! I've been working with it for over two years, but had never tried anything similar to your "data probe" technique. It makes perfect sense.

One thing I saw in this article that's a little different is the Multicast as sort of a dummy endpoint. I've often seen a Rowcount as a dummy endpoint but never a multicast. Does the endpoint actually do anything in either case in this type of scenario?

Cheers,

-Matthew


I actually use the konesans trash destination when I need a dummy end point.




Jack Corbett

Applications Developer

Don't let the good be the enemy of the best. -- Paul Fleming

Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
How to Post Performance Problems
Crosstabs and Pivots or How to turn rows into columns Part 1
Crosstabs and Pivots or How to turn rows into columns Part 2
Post #655052
Posted Wednesday, February 11, 2009 7:39 PM


Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Wednesday, April 16, 2014 1:39 PM
Points: 388, Visits: 1,034
Thanks for all the comments!

Matthew - Like Jack, I usually use the Konesans Trash Destination for an endpoint as well. I didn't mention it here. I also use the Script Component and the Derived Column transformations as dummy endpoints for data flow paths - basically anything that will succeed without configuration will do.

:{> Andy


Andy Leonard
CSO, Linchpin People
Follow me on Twitter: @AndyLeonard
Post #655354
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse