SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


ETL Testing Tools


ETL Testing Tools

Author
Message
vick12
vick12
SSC Rookie
SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)SSC Rookie (46 reputation)

Group: General Forum Members
Points: 46 Visits: 86
Hi All,

I need to know if there is any tool which can be used to for post migration ETL testing? The ETL has been written using sql scripts to move the data from Legacy database to a new database and both have different database structures We need to test the tranformed data for accuracy and check for data quality and quantity after the ETL process is executed.
I have questions about the following:
1.what should be the best approach/ best practices to test the data in the new database after transformation?
2. I would like to know if there is any tool which can be used to compare the data in old vs new database (both having different structures) ? Thanks.
FREDERICK
FREDERICK
Forum Newbie
Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)Forum Newbie (6 reputation)

Group: General Forum Members
Points: 6 Visits: 29
Hello

You may want to check out
tSQLt – The Database Unit Testing Framework for SQL Server
http://tsqlt.org/

Tests are automatically run within transactions – this keeps tests independent and reduces any cleanup work you need

* Tests can be grouped together within a schema – allowing you to organize your tests and use common setup methods

* Output can be generated in plain text or XML – making it easier to integrate with a continuous integration tool

* Provides the ability to fake tables and views, and to create stored procedure spies – allowing you to isolate the code which you are testing


FRED



Orlando Colamatteo
Orlando Colamatteo
SSChampion
SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)SSChampion (14K reputation)

Group: General Forum Members
Points: 14983 Visits: 14396
If you're moving to a different structure then I would not trust anything to validate that anyhow. Chances are you will need custom methods to validate your custom transformations. Common methods are reconciling row counts between systems and picking (or conjuring) a row on the source that should exercise all parts of a given transformation, e.g. 1 row on source becomes three rows on destination, and validating that you see the expected result on the destination.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)SSC Guru (86K reputation)

Group: General Forum Members
Points: 86422 Visits: 41098
I know this is an older post but I agree with OPC.Three. If there were a testing tool that could validate your ETL results against the source, why wouldn't you just use that same testing tool to do the actual import?

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search