April 1, 2015 at 2:07 pm
Greetings,
I am hoping to get some community feedback on this subject, as I am not sure of where I stand on this proposed change of direction.
The project on which I am working uses an SSIS based process that accepts flat file type documents from various client folder locations, performs a varying amount of transformation (formatting) actions, and loads the data into our transaction system. These files are not all necessarily in the same common format, they sometimes vary by delimiter, qualifier, and column header name, but all contain a similar set of data elements. This process was a bit of a rush job during our initial development and currently, I have client specific packages built to handle the nuances of each individual client's specifications. Unfortunately though as our customer base has grown, so has the amount of maintenance to keep the packages compliant and working.
Overall, the process works well in it's current form and scale, but overtime I can see this design causing major headaches down the road, especially when an overarching change is necessary.
As it were, my team and I are currently re-engineering of the core database structure of the project in order to move forward in the products development and meet upcoming business needs. As a part of this, the need to re-engineer the SSIS Client File Load process has been brought back to my queue.
My plan was to rebuild the process as dynamically as possible in the next iteration, and removing the need for duplicate code for each client. Getting every client to use a standard format is unfortunately a pipe-dream in this case, so there will be no getting around some custom coding, but I feel at least half of the packages in the current project could be consolidated into one of a handful of common package load formats as a start.
During a design discussion with the management, enterprise architects, and development team members, the question was raised by management as to why this process could not be handled via BizTalk, as we currently use it for several other types of data feeds? The discussion then turned to dumping the entire SSIS project and rebuilding it in BizTalk.
Personally, I can't see how this could be "better" for the task at hand. Especially, as management is looking to move the management of this process to the operations side of our business.
Does anyone out there have any thoughts on the business uses of BizTalk and SSIS, that could help me drive this discussion further?
My fears are that going the BizTalk route will over complicate the process as a whole.
Thanks,
April 1, 2015 at 11:05 pm
I think you answered this question already
😎
My fears are that going the BizTalk route will over complicate the process as a whole.
April 2, 2015 at 5:52 am
Have a look at the links I've provided. They both come to roughly the same conclusion which is that BizTalk is useful for complex workflows involving small amounts of data in near-realtime whereas SSIS is better at handling large amounts of data requiring less complex workflows. In my limited experience of BizTalk, I've found it to be very 'chatty' so finding and fixing problems can be very difficult.
Regards
Lempster
April 2, 2015 at 6:11 am
Eirikur Eiriksson (4/1/2015)
I think you answered this question already😎
My fears are that going the BizTalk route will over complicate the process as a whole.
I had a feeling I was going to get this response.
April 2, 2015 at 6:11 am
Lempster (4/2/2015)
Have a look at the links I've provided. They both come to roughly the same conclusion which is that BizTalk is useful for complex workflows involving small amounts of data in near-realtime whereas SSIS is better at handling large amounts of data requiring less complex workflows. In my limited experience of BizTalk, I've found it to be very 'chatty' so finding and fixing problems can be very difficult.Regards
Lempster
Thank you, I'll take a look.
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply