using SSIS to perform data validation in ETL app

• Phil Parkin

  SSC Guru

  Points: 247159

  More actions

  June 22, 2009 at 6:27 pm

  #1013654

  jreece (6/22/2009)

  ---

  John,

  --

  I am now going to add a few derived columns - I thought "Create_Date" and "Change_Date" might be very useful for tracking updates.

  You might also consider doing these at the database level. Your CreateDate field just needs a DefaultValue of GetDate() and ChangeDate needs an UPDATE trigger to keep it maintained ...

  ---

• John Rowan

  SSC Guru

  Points: 56440

  More actions

  June 23, 2009 at 9:42 am

  #1014050

  Phil Parkin (6/22/2009)

  ---

  jreece (6/22/2009)

  ---

  John,

  --

  I am now going to add a few derived columns - I thought "Create_Date" and "Change_Date" might be very useful for tracking updates.

  You might also consider doing these at the database level. Your CreateDate field just needs a DefaultValue of GetDate() and ChangeDate needs an UPDATE trigger to keep it maintained ...

  I think that depends on what you are after. I routinely use column defaults on CreateDate columns in OLTP systems. For OLAP or data warehouse DBs, I typically use a LoadDate column when loading data and I assign the LoadDate value from a variable in the SSIS package that gets its value at SSIS startup time so that everything that was loaded as part of the SSIS package run has the same LoadDate.

  John Rowan

  ======================================================
  ======================================================
  Forum Etiquette: How to post data/code on a forum to get the best help[/url] - by Jeff Moden

• jreece

  Ten Centuries

  Points: 1138

  More actions

  June 23, 2009 at 10:29 am

  #1014090

  That's exactly what I want to do, tag all the records loaded with the same date stamp so I can identify them by batch. Thanks again - I can see it will take me a long time to get up to speed with this tool. I don't even know what I don't know. (sorry to get all Yogi Berra on you)

• Deepmala

  Newbie

  Points: 1

  More actions

  October 16, 2020 at 12:45 pm

  #3797953

  ?Well I have always used Daton's SQL connector  for my Warehousing. You can add any Data Source here in a single place

• Phil Parkin

  SSC Guru

  Points: 247159

  More actions

  October 16, 2020 at 2:22 pm

  #3798014

  Deepmala wrote:

  ?Well I have always used Daton's SQL connector  for my Warehousing. You can add any Data Source here in a single place

  Unless you are going to justify this comment by describing exactly how this product performs data validation, I am going to have to mark your comment as spam.

  ---