Post reply

separate bad data from valid data

  • December 8, 2015 at 10:14 am

    #303148

    Hi,

    Apart from the script component, is there another task is ssis which does the same job?

    thanks

  • December 8, 2015 at 10:25 am

    #1844727

    arkiboys (12/8/2015)

    Hi,

    Apart from the script component, is there another task is ssis which does the same job?

    thanks

    What determines that the data is bad?

    Michael L John
    If you assassinate a DBA, would you pull a trigger?
    To properly post on a forum:
    http://www.sqlservercentral.com/articles/61537/

  • December 8, 2015 at 10:30 am

    #1844729

    Depending on the type of bad data. A conditional split could work, or simply error handling to redirect bad rows.

    Luis C.
    General Disclaimer:
    Are you seriously taking the advice and code from someone from the internet without testing it? Do you at least understand it? Or can it easily kill your server?

    How to post data/code on a forum to get the best help: Option 1 / Option 2

  • December 9, 2015 at 3:30 am

    #1844840

    Luis Cazares (12/8/2015)

    Depending on the type of bad data. A conditional split could work, or simply error handling to redirect bad rows.

    For example if the feeds coming in (.txt files), some of the columns may have text instead of the expected int values.

    What is the best way to handle this so that the package does not fail?

    Thank you

  • December 9, 2015 at 7:26 am

    #1844903

    I would define the columns from the text file as strings, then use a data conversion or derived column transformation to change the columns' types. That transformation should have 2 outputs, one for valid rows that will go into the table, and one for invalid rows that might go into a new text file or error table so you can understand what's wrong with those rows.

    Remember to allow enough amount of errors to handle all the rows.

    Luis C.
    General Disclaimer:
    Are you seriously taking the advice and code from someone from the internet without testing it? Do you at least understand it? Or can it easily kill your server?

    How to post data/code on a forum to get the best help: Option 1 / Option 2

  • December 9, 2015 at 7:50 am

    #1844907

    Luis Cazares (12/9/2015)

    I would define the columns from the text file as strings, then use a data conversion or derived column transformation to change the columns' types. That transformation should have 2 outputs, one for valid rows that will go into the table, and one for invalid rows that might go into a new text file or error table so you can understand what's wrong with those rows.

    Remember to allow enough amount of errors to handle all the rows.

    How do you: change column types?

    thanks

  • December 9, 2015 at 7:54 am

    #1844908

    arkiboys (12/9/2015)

    Luis Cazares (12/9/2015)

    I would define the columns from the text file as strings, then use a data conversion or derived column transformation to change the columns' types. That transformation should have 2 outputs, one for valid rows that will go into the table, and one for invalid rows that might go into a new text file or error table so you can understand what's wrong with those rows.

    Remember to allow enough amount of errors to handle all the rows.

    How do you: change column types?

    thanks

    In the connection manager, in the advanced tab.

    Luis C.
    General Disclaimer:
    Are you seriously taking the advice and code from someone from the internet without testing it? Do you at least understand it? Or can it easily kill your server?

    How to post data/code on a forum to get the best help: Option 1 / Option 2

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply