June 6, 2013 at 6:49 am
Correction to the title... Can't distinguish between "42" and "IM"
I have a For Loop that iterates through text files and brings them in. Recently, our vendor started getting tricky with the formats and sometimes they use an old style, sometimes a new style. The data is imported into an Evaluation table and we skim off the top row (Select top 1 State.....). That result is passed to an expression. The connectors use a Precedence Constraint (Expression and Constraint, Success, @[User::IndexFormatFlag] != "00") to determine which connection manager to use. If the result is "42" it uses the old format, if it is "00" it's the new format (since "42" != "00")
Recently we started receiving a 3rd format. The structure is completely different; however, I can successfully import it into the eval table and strip of the top. Since the new format has a header row, It gets picked up in the query (Select Top 1 State) has the value "Im". I created a new Precedence Constraint to evaluate it and it works. The constraints for the 3 versions are as follows:
@[User::IndexFormatFlag] == "42" (Old)
@[User::IndexFormatFlag] == "00" (New)
@[User::IndexFormatFlag] == "Im" (Newest)
The problem now is that when the folder with the text files has the newest ("Im") and the oldest ("42"), the precedent constraint seems to be getting confused with the 42 vs. the IM and it tries to load the Old format into the Newest format and then subsequently dies. The data is getting streamed to both at once in fact.
Any ideas as to why this might be happening? I'm thinking about creating a second constraint just for the newest format. That way it has to meet both conditions to continue.
June 7, 2013 at 9:10 am
Anyone? Bueller?
June 7, 2013 at 2:41 pm
CptCrusty1 (6/6/2013)
The problem now is that when the folder with the text files has the newest ("Im") and the oldest ("42")
How existence of different files in a folder can impact anything? Do you load all the files into a staging table and then decide what provider to use?
Any ideas as to why this might be happening?
You have a bug for sure. Best tool in this case is a debugger.
June 10, 2013 at 10:34 am
Alexander Suprun (6/7/2013)
How existence of different files in a folder can impact anything?[/Quote]
It doesn't. That isn't the problem.
[Quote]
Do you load all the files into a staging table and then decide what provider to use?
Yes, I stated earlier that it's loaded to an Evaluation table to determine which staging table to send it to.
You have a bug for sure. Best tool in this case is a debugger.
It's not a bug, its an missunderstanding of how the logic is being evaluated. I solved the issue a different way, but I haven't figured out why it's doing it in the first place....
Thanks
Crusty
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply