Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Auditing Changes in ETL with SSIS Expand / Collapse
Author
Message
Posted Tuesday, December 23, 2008 12:26 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Monday, April 07, 2014 10:43 AM
Points: 1,042, Visits: 2,708
Comments posted to this topic are about the item Auditing Changes in ETL with SSIS



Tim Mitchell, SQL Server MVP
Independent Business Intelligence Consultant
www.TimMitchell.net
@Tim_Mitchell

Post #624488
Posted Thursday, December 25, 2008 4:35 AM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Thursday, March 20, 2014 7:32 AM
Points: 5,193, Visits: 1,368
Nice one...:)


Post #625650
Posted Tuesday, December 30, 2008 5:40 PM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Friday, April 18, 2014 5:57 PM
Points: 535, Visits: 729
Very good example of auditing in SSIS. Something to keep in mind is if the percentage of "bad" records is significant, your Audit table will rival in size the actual destination table, so this should be considered at design time. Thank you!


Post #627743
Posted Saturday, September 05, 2009 9:34 PM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Monday, April 07, 2014 10:43 AM
Points: 1,042, Visits: 2,708
mishaluba, you're right - if you're doing a lot of transformation, storage will be a key consideration. I worked recently with a very large ETL process in which tens of millions of records were converted from a legacy system, and every single record had some sort of transformation on it. The auditing tables more than doubled the size of our staging database, but my staff has had to return to that auditing data several different times for forensic purposes.



Tim Mitchell, SQL Server MVP
Independent Business Intelligence Consultant
www.TimMitchell.net
@Tim_Mitchell

Post #783406
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse