Post reply

ETL Testing Tools

  • October 23, 2012 at 7:48 am

    #269955

    Hi All,

    I need to know if there is any tool which can be used to for post migration ETL testing? The ETL has been written using sql scripts to move the data from Legacy database to a new database and both have different database structures We need to test the tranformed data for accuracy and check for data quality and quantity after the ETL process is executed.

    I have questions about the following:

    1.what should be the best approach/ best practices to test the data in the new database after transformation?

    2. I would like to know if there is any tool which can be used to compare the data in old vs new database (both having different structures) ? Thanks.

  • February 9, 2013 at 12:20 pm

    #1586247

    Hello

    You may want to check out

    tSQLt – The Database Unit Testing Framework for SQL Server

    http://tsqlt.org/

    Tests are automatically run within transactions – this keeps tests independent and reduces any cleanup work you need

    * Tests can be grouped together within a schema – allowing you to organize your tests and use common setup methods

    * Output can be generated in plain text or XML – making it easier to integrate with a continuous integration tool

    * Provides the ability to fake tables and views, and to create stored procedure spies – allowing you to isolate the code which you are testing

    FRED

  • February 9, 2013 at 2:02 pm

    #1586251

    If you're moving to a different structure then I would not trust anything to validate that anyhow. Chances are you will need custom methods to validate your custom transformations. Common methods are reconciling row counts between systems and picking (or conjuring) a row on the source that should exercise all parts of a given transformation, e.g. 1 row on source becomes three rows on destination, and validating that you see the expected result on the destination.

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

  • September 4, 2014 at 7:44 pm

    #1743374

    I know this is an older post but I agree with OPC.Three. If there were a testing tool that could validate your ETL results against the source, why wouldn't you just use that same testing tool to do the actual import?

    --Jeff Moden

    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.

    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply