Dynamic File Column mapping in SSIS.

Question

Dynamic File Column mapping in SSIS.

November 19, 2018 at 12:23 am

This was removed by the editor as SPAM

Viewing 6 posts - 16 through 20 (of 20 total)

You must be logged in to reply to this topic. Login to reply

ghalib.mustafa Valued Member Points: 67 More actions · Answer 1

Looking at the progress I found this:

[SSIS.Pipeline] Warning: The output column "Column5" (50) on output "Output 0" (43) and component "Read File Contents" (32) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance.

Maybe this is the issue, but I don't understand it because to my understanding, Column5 has been used repeatedly as like other columns

Hiske Bekkering SSC Enthusiast Points: 141 More actions · Answer 2

ghalib.mustafa - Sunday, November 18, 2018 11:34 PM
Hiske Bekkering - Friday, November 16, 2018 9:03 AM
Hi Ghalib,
You need to add the new column to the outputs as well!
Open the editor of the "Read File Contents" script, go to the "Inputs and Outputs" page, expand "Output 0" and add the definition for the new column.
See the below image.
Success!
Regards,
Hiske
Thankyou sir,
This is exactly what I was missing. It did resolved the error but still, the package can't make the mappings.
I have debugged the code and looked in every for loops as if it is missing the "Column5" or "A2" anywhere but the code is not missing anything and everything is looking fine to me. Here is the screenshot of column mappings:
it must be really annoying to help a dumb developer, but I would really appreciate that and there may be thousands more like me, who could get help from this.

Hi Ghalib,

Of course, you must also set the mapping for Column5!
So in your screenshot of the mappings page, where it says "<Ignore>" you must select "Column5"

Regards,
Hiske

ghalib.mustafa Valued Member Points: 67 More actions · Answer 3

Hiske Bekkering - Monday, November 19, 2018 1:24 AM
Hi Ghalib,
Of course, you must also set the mapping for Column5!
So in your screenshot of the mappings page, where it says "<Ignore>" you must select "Column5"
Regards,
Hiske

Hi Sir,
Thank-you for your precious time.
Ok, I got it, it is working fine now. But, I was wondering if it is possible to make this more dynamic, as if we don't have to do the step mentioned above or adding output columns to script task. it should just read the mapping from mapping table and map the columns as is dynamically.

Actually, I am looking for the way by which we can transfer our data from source to destination without having the knowledge of data(types/metadata/no of columns)of source and destination.
I am new to this BI field and it's been three weeks since I joined and I see others making new package every-time for different kinds of sources and destinations. I understand that I am not at the level to question that but it is kind of intriguing to me that it is very basic SSIS operation, and it is not dynamic.

Regards,
Ghalib

Hiske Bekkering SSC Enthusiast Points: 141 More actions · Answer 4

ghalib.mustafa - Monday, November 19, 2018 3:49 AM
Hiske Bekkering - Monday, November 19, 2018 1:24 AM
Hi Ghalib,
Of course, you must also set the mapping for Column5!
So in your screenshot of the mappings page, where it says "<Ignore>" you must select "Column5"
Regards,
Hiske
Hi Sir,
Thank-you for your precious time.
Ok, I got it, it is working fine now. But, I was wondering if it is possible to make this more dynamic, as if we don't have to do the step mentioned above or adding output columns to script task. it should just read the mapping from mapping table and map the columns as is dynamically.
Actually, I am looking for the way by which we can transfer our data from source to destination without having the knowledge of data(types/metadata/no of columns)of source and destination.
I am new to this BI field and it's been three weeks since I joined and I see others making new package every-time for different kinds of sources and destinations. I understand that I am not at the level to question that but it is kind of intriguing to me that it is very basic SSIS operation, and it is not dynamic.
Regards,
Ghalib

Hi Ghalib,

Well, it might be possible, although I do not know how to do it.
As my solution fixed the problem I was facing, I did not investigate further.

But perhaps a commercial product such as Cozyroc (as recommended by Mark Tassin) can provide what you want!

Regards,

Hiske

ghalib.mustafa Valued Member Points: 67 More actions · Answer 5

Hiske Bekkering - Monday, November 19, 2018 4:56 AM
Hi Ghalib,
Well, it might be possible, although I do not know how to do it.
As my solution fixed the problem I was facing, I did not investigate further.
But perhaps a commercial product such as Cozyroc (as recommended by Mark Tassin) can provide what you want!
Regards,
Hiske

Hello Sir,

Yes Sir, I am aware that, with the help of Cozyroc we can achieve that as I have seen few tutorials about it, but the issue with Cozyroc is same with me as with you. I am not even Senior SE'r. However I believe, that if Cozyroc can achieve that, perhaps individual can achieve it too, doesn't matter if it is not that fast.
Thank-you very much sir for your time and effort, as you have shown us a way and helped us to attain dynamic mapping and made us believe that it is possible any how. Before this, I wasn't even able to visualize the behavior of package but now I could.
May God bless you for helping us.

Regards,
Ghalib