Hi i was surprised to see the approach my coworkers used to sunset talend , a combo etl and job scheduling tool. They were in a hurry.
Basically, the talend transforms /jobs they needed to do something with ran a proc on the "ERP Servers" and loaded the extracted data into various databases on our warehouse server. Mostly two step (E and L) straightforward mapping right now because they are in their infant stages.
The group decided to set up sql server agent jobs (unfortunately 1 per erp per transform) in their place. Which isnt much different than what we had in talend (but now its one step per as inserts will take place right in the sprocs). I'm sure the multiple jobs can be consolidated later into multi step jobs (one per domain, eg puchasing, labor ets etc) for smoother review.
Its not really clear why they didnt also use ssis but i believe the reasons are 1) they dont know ssis and they were in a hurry, 2) these are simple mappings, at least right now.
so the ability of sql to generate ssis for you not necessarily being of interest to them , they found themselves needing to change( i believe) and move those procs to our warehouse server where they would do their thing thru linked servers of which we have plenty.
The main question i have is does this then prevent use of what i called "where used" analysis built into sql where if im on the erp server i can see easily what this "object" depends on and vice versa? With the missing piece of the puzzle being the sproc itself?
As simple as these mappings are, I've seen what they morph into here in the form of unmaintainable t-sql. Whereas etl tools tend to mitigate that issue by dividing and conquering tasks. I wish they'd taken a few extra minutes to make these ssis so at a glance we could also see the health of etl thru the green and red flags in the ssis dashboard as well as maintaining consistency with the way we do everything else. Talend had such a dashboard , and now we've lost that. To me this feels like classic tech debt .
July 15, 2025 at 12:10 pm
Thanks for posting your issue and hopefully someone will answer soon.
This is an automated bump to increase visibility of your question.
as soon as you put anything outside the Database itself, you loose all referencing capabilities.
SQL Server Agent jobs in my opinion the worst option, specially if they go with 1 job per transform per ERP. Talend for that would be better, so would many other tools, including, I hate to say it, SSIS.
Viewing 3 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply