<a href="http://g.adspeed.net/ad.php?do=clk&zid=15220&wd=468&ht=60&pair=as" target="_blank"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15220&wd=468&ht=60&pair=as" alt="i" width="468" height="60"/></a>
Log in
::
Register
::
Not logged in
Home
Tags
Articles
Editorials
Stairways
Forums
Scripts
Videos
Blogs
QotD
Books
Ask SSC
SQL Jobs
Training
Authors
About us
Contact us
Newsletters
Write for us
<a href="http://g.adspeed.net/ad.php?do=clk&zid=15491&wd=120&ht=300&pair=as" target="_top"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15491&wd=120&ht=300&pair=as" alt="i" width="120" height="300"/></a>
Recent Posts
Recent Posts
Popular Topics
Popular Topics
Home
Search
Members
Calendar
Who's On
Home
»
Article Discussions
»
Article Discussions by Author
»
Discuss content posted by Adam Aspin
»
A faster way to prepare dimensional databases...
39 posts, Page 3 of 4
««
«
1
2
3
4
»»
A faster way to prepare dimensional databases
Rate Topic
Display Mode
Topic Options
Author
Message
magarity kerns
magarity kerns
Posted Thursday, February 25, 2010 12:16 PM
Old Hand
Group: General Forum Members
Last Login: Monday, May 13, 2013 10:22 PM
Points: 358,
Visits: 393
No, I've never had a problem with using md5. I'm inspired to write my own article on how to use md5 in an update/insert strategy for loading data warehouses if I can get Steve the admin to accept it.
Post #872939
tskelley
tskelley
Posted Thursday, February 25, 2010 12:48 PM
SSC Rookie
Group: General Forum Members
Last Login: 2 days ago @ 6:03 AM
Points: 33,
Visits: 759
magarity kerns (2/25/2010)
No, I've never had a problem with using md5. I'm inspired to write my own article on how to use md5 in an update/insert strategy for loading data warehouses if I can get Steve the admin to accept it.
By all means, please write away. We currently truncate and replace, but would be extremely interested in a more incremental strategy. For example, my understanding is BIDS Slowly Changing Dimension transformation is a performance hound for large tables.
Post #872954
magarity kerns
magarity kerns
Posted Thursday, February 25, 2010 11:02 PM
Old Hand
Group: General Forum Members
Last Login: Monday, May 13, 2013 10:22 PM
Points: 358,
Visits: 393
We don't need no stinkin' changing dimension widgets... Took me a while with the article-submitting editor, but I got my treatise on using MD5 submitted. We'll see how fast it gets rejected.
Post #873183
Adam Aspin
Adam Aspin
Posted Friday, February 26, 2010 1:51 AM
SSC Journeyman
Group: General Forum Members
Last Login: Yesterday @ 9:52 AM
Points: 80,
Visits: 847
Fabulous! I look forward to reading it!
Adam
Post #873250
SQLRNNR
SQLRNNR
Posted Friday, December 17, 2010 12:08 AM
SSCoach
Group: General Forum Members
Last Login: 2 days ago @ 1:07 PM
Points: 18,733,
Visits: 12,332
Thanks for the article.
Jason
AKA CirqueDeSQLeil
I have given a name to my pain...
MCM SQL Server 2008
SQL RNNR
Posting Performance Based Questions - Gail Shaw
Posting Data Etiquette - Jeff Moden
Hidden RBAR - Jeff Moden
VLFs and the Tran Log - Kimberly Tripp
Post #1036354
Jason-299789
Jason-299789
Posted Friday, December 17, 2010 1:46 AM
SSC Eights!
Group: General Forum Members
Last Login: 2 days ago @ 3:38 AM
Points: 803,
Visits: 2,123
Its a good article.
However, did you experience any problems with datatypes being incorrectly assigned using the SELECT..INTO.. method, I notice you mention that nulls are handled in the consolidation layer so it may mitigate the issue.
I'm also not sure how this can be adapted to an incremental DW unless you are not concerned about maintaining the history, as by dropping the fact table you loose the ability to maintain the historic Fact record, and thus you surely defeat the object one of the objectives of an incremental, which is to allow you to perform point in time analysis, which can be quite critical for DW's in the Finance industry.
_________________________________________________________________________
SSC Guide to Posting and Best Practices
Post #1036380
hallidayd
hallidayd
Posted Friday, December 17, 2010 2:33 AM
SSC-Enthusiastic
Group: General Forum Members
Last Login: Sunday, March 03, 2013 2:50 AM
Points: 112,
Visits: 294
INSERT INTO <table> WITH (TABLOCK)
will minimally log in the same way
SELECT .... INTO
will by default.
Post #1036397
dave-dj
dave-dj
Posted Friday, December 17, 2010 3:25 AM
Old Hand
Group: General Forum Members
Last Login: 2 days ago @ 11:47 AM
Points: 380,
Visits: 1,014
For example, my understanding is BIDS Slowly Changing Dimension transformation is a performance hound for large tables.
FAO tskelly:
If your finding the standard SCD component of SQL slow, have a look at Todd McDermitts Kimball SCD component.
I've switched to using his Kimball SCD for a number of reasons. First of all it's faster. As mentioned in the details, it doesn't destroy the dataflow when you need to make changes. Also, it provide a wealth of auditing information. Well worth a look!
You can find it here: http://kimballscd.codeplex.com/
_____________________________________________________________________________
MCITP: Business Intelligence Developer (2005)
Post #1036423
Gosta Munktell
Gosta Munktell
Posted Friday, December 17, 2010 3:28 AM
SSC-Enthusiastic
Group: General Forum Members
Last Login: Friday, April 26, 2013 4:05 AM
Points: 193,
Visits: 1,248
Thank you Adam.
At first it is almost necessary to have control of the the whole "ETL" process. Missing data or
bad quality is major problem in many cases. You can seldom avoid outer joins. I use ISNULL(expression,'MISSING') and have a default dimension member MISSING in most of the dimensions. Your approach is also usefull in "non OLAP" situations like with the new feature PowerPivot.
We do have dimension tables and one or more fact tables to import. PowerPivot then creates the "CUBE" by it self.
Gosta M
Post #1036424
Jason-299789
Jason-299789
Posted Friday, December 17, 2010 3:57 AM
SSC Eights!
Group: General Forum Members
Last Login: 2 days ago @ 3:38 AM
Points: 803,
Visits: 2,123
dave-dj (12/17/2010)
For example, my understanding is BIDS Slowly Changing Dimension transformation is a performance hound for large tables.
FAO tskelly:
If your finding the standard SCD component of SQL slow, have a look at Todd McDermitts Kimball SCD component.
I've switched to using his Kimball SCD for a number of reasons. First of all it's faster. As mentioned in the details, it doesn't destroy the dataflow when you need to make changes. Also, it provide a wealth of auditing information. Well worth a look!
You can find it here: http://kimballscd.codeplex.com/
We use Todds component as well, it has a few performance issues with large datasets, we did some alpha testing the v1.6 of the component and that seems to have fixed some of the perfomance issues, unfortuantely we're just waiting for the final release.
_________________________________________________________________________
SSC Guide to Posting and Best Practices
Post #1036439
« Prev Topic
|
Next Topic »
39 posts, Page 3 of 4
««
«
1
2
3
4
»»
Permissions
You
cannot
post new topics.
You
cannot
post topic replies.
You
cannot
post new polls.
You
cannot
post replies to polls.
You
cannot
edit your own topics.
You
cannot
delete your own topics.
You
cannot
edit other topics.
You
cannot
delete other topics.
You
cannot
edit your own posts.
You
cannot
edit other posts.
You
cannot
delete your own posts.
You
cannot
delete other posts.
You
cannot
post events.
You
cannot
edit your own events.
You
cannot
edit other events.
You
cannot
delete your own events.
You
cannot
delete other events.
You
cannot
send private messages.
You
cannot
send emails.
You
may
read topics.
You
cannot
rate topics.
You
cannot
vote within polls.
You
cannot
upload attachments.
You
may
download attachments.
You
cannot
post HTML code.
You
cannot
edit HTML code.
You
cannot
post IFCode.
You
cannot
post JavaScript.
You
cannot
post EmotIcons.
You
cannot
post or upload images.
Copyright © 2002-2013 Simple Talk Publishing. All Rights Reserved.
Privacy Policy.
Terms of Use.
Report Abuse.
