Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase ««12

Level 3 of the Stairway to Integration Services - Adding Rows in Incremental Loads Expand / Collapse
Author
Message
Posted Wednesday, September 21, 2011 10:41 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Thursday, December 18, 2014 12:52 PM
Points: 266, Visits: 2,613
You have great skill in explaining things. Thanks for taking the time to share your information in a way that is actually helpful!
Post #1178844
Posted Wednesday, September 21, 2011 11:18 AM


Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Monday, December 15, 2014 2:46 PM
Points: 388, Visits: 1,047
Hi JJ,

I have an excellent editor, Steve Jones.

:{>


Andy Leonard
CSO, Linchpin People
Follow me on Twitter: @AndyLeonard
Post #1178874
Posted Monday, October 1, 2012 8:05 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, November 2, 2012 10:30 AM
Points: 2, Visits: 33
Hi Andy,

Thanks for this great article!!! I have a question; what if the Person.Contacts table is located on an another server, what would you do? We definitely don't want to get all rows as it impacts performance and may have timeout issues for huge data. How can we join Person.Contact and dbo.Contact when they are located on different servers, then only select the rows we need to insert from Person.Contact?

Thanks again!
Post #1366584
Posted Friday, December 14, 2012 5:39 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Friday, September 19, 2014 7:12 AM
Points: 291, Visits: 182
Andy I'am enjoing the series but I have a small problem with the 'Update'.

I'ts working, but it only updates about 700 records each run, I can't find any option to change this behavior.
The only thing I can imagine is that I use a very limited virtual server and that this is caused by the Full Cache option of the Lookup.
But I see the correct number of records at each Data Flow.

I do see 2 warnings, but I don't think they have any thing to do with the update problem:
[SSIS.Pipeline] Warning: Warning: Could not open global shared memory to communicate with performance DLL; data flow performance counters are not available. To resolve, run this package as an administrator, or on the system's console.

and
[SSIS.Pipeline] Warning: The output column "LkUp_BusinessEntityID" (131) on output "Lookup Match Output" (103) and component "Lookup" (101) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance.

Rob

Post #1396598
Posted Friday, March 29, 2013 2:19 PM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Thursday, December 18, 2014 12:08 PM
Points: 69, Visits: 383
I agree that the second method is better than the first, but the duration of each method can only be measured correctly if you clear the procedure cache for the previous dataset.
Post #1437092
Posted Sunday, March 31, 2013 10:17 PM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Wednesday, June 11, 2014 11:59 PM
Points: 29, Visits: 138
This is a good article for the beginners. Great work Andy!

I would also stress on looking at Merge TSQL statement. Have used it quite a few times. It would be interesting to compare the performance and pros/cons with Lookup transformation.

Also, with Merge we can handle SCD1 and SCD2 quite easily. I reckon this is required in most scenarios for incremental dimension loading. This way we can do away with bad SCD transformation available with SSIS out of the box.

My 2 cents.
Post #1437354
Posted Wednesday, April 3, 2013 2:55 PM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Sunday, July 7, 2013 3:58 PM
Points: 6, Visits: 68
Write a book on SSIS.
I suspect it will be 200,000 pages .....
The blow-by-blow (with background explanations) are
immensely helpful.

I will still buy it .... even if it costs $100
Post #1438585
Posted Thursday, April 4, 2013 12:22 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Friday, September 19, 2014 7:12 AM
Points: 291, Visits: 182
in fact there is a book about SSIS although the title suggests something else..
the title is Implementing a Data Warehouse with Microsoft SQL server 2012 (exam 70-463)
It covers a lot of SSIS but also Master data services, Data Cleansing and Data Warehouse, most of the tasks are done using SSIS.

I found 2 problems,
1 the ODBC connections only work in 32-bit mode
2 the Master Data web interface is in Dutch I have configured the server to be in my home country, suddenly the exercises are a real puzzle
Post #1438654
« Prev Topic | Next Topic »

Add to briefcase ««12

Permissions Expand / Collapse