conditional split

  • hi

    i am getting 2 files where i need to check if data has been chnaged from previous file,
    i am using merge join on full outer and then conditional split

    condition : s_column1==d_column2&&s_column2==d_column2&&s_column3==d_coulm3&&(s_column4!=s_column4||s_column5!=s_column5)

    like this first 3 are primary key and rest i need to check if any of the field r not matching ,its change of record.

    but i am not getting same record as i suppose to get.

    any idea whats wrong in my condition.

  • coool_sweet - Tuesday, June 6, 2017 2:06 PM

    condition : s_column1==d_column2&&s_column2==d_column2&&s_column3==d_coulm3&&(s_column4!=s_column4||s_column5!=s_column5)

    It appears that there may be some mismatches in your comparison statements.  First there's the beginning.  You have the following:
    s_column1==d_column2&&...

    But based on the pattern you showed in the rest of the code, shouldn't it be:
    s_column1==d_column1&&...

    Also, final column check towards the end appear to be all self-referencing the same source columns:
    ...(s_column4!=s_column4||s_column5!=s_column5)

    Shouldn't that be:
    ...(s_column4!=d_column4||s_column5!=d_column5)

    If those assumptions are correct, I suggest you change it to the following
    s_column1==d_column1&&s_column2==d_column2&&s_column3==d_coulm3&&(s_column4!=d_column4||s_column5!=d_column5)

    That should get you going.

Viewing 2 posts - 1 through 1 (of 1 total)

You must be logged in to reply to this topic. Login to reply