I hope I am describing the scenario correctly. We have an ETL process that is composed of several stored procs. The ETL process is initiated using a web service (I think that's what it is called). The stored procs and the web service are on separate servers. What we have found is that at times there are network glitches which causes the service to lose contact with the SQL Server and the ETL process. Not knowing the status of the ETL process on the SQL Server, it restarts the process from the beginning. This almost always results in conflicts and the process fails, mostly due to one of the processes deleting tmp tables that the other process needs.
We are considering packaging the ETL process as a SSIS package to employ constraints, checkpoints, etc. The architeture would be the same. The SSIS package would be on one server and the web service on another. Would using SSIS to manage the process resolve the issue of the web service restarting the ETL process before the previous process completed?
Any suggestions would be greatly appreciated.
Michael A. Hardy