SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Execution of SQL Job, the moment a file exist


Execution of SQL Job, the moment a file exist

Author
Message
dakshinamurthy-655138
dakshinamurthy-655138
Ten Centuries
Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)

Group: General Forum Members
Points: 1266 Visits: 1056
Hi,

I have a scenario..

I have a SQL Job which needs to be executed based on demand, means the moment it finds a file in a path it should execute. if there is no file in a path it should not execute. Execution or start of the SQL Job should be based on the existence of a file.

Assume i have folder D:\jobfiles\ , in this folder the moment a file is dumped, it should start the job else invoke of SQL job should not happen.

With Regards
Dakshina Murthy
Orlando Colamatteo
Orlando Colamatteo
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39210 Visits: 14411
A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

You could also do this a few different ways using an SSIS package that runs continuously watching for files:

Using the WMI Event Watcher Task in SSIS to Process Data Files
Using the Konesans File Watcher Task in SSIS to Process Data Files
Using the Script Task in SSIS to Process Data Files When They Arrive

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
MyDoggieJessie
MyDoggieJessie
SSChampion
SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)

Group: General Forum Members
Points: 12162 Visits: 7444
opc.three (2/11/2013)
A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

You could also do this a few different ways using an SSIS package that runs continuously watching for files:

Using the WMI Event Watcher Task in SSIS to Process Data Files
Using the Konesans File Watcher Task in SSIS to Process Data Files
Using the Script Task in SSIS to Process Data Files When They Arrive
+1 to the SSIS option. Use a For...Loop container to search through the folder you want to monitor. It's really simple

______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience" ;-)
Orlando Colamatteo
Orlando Colamatteo
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39210 Visits: 14411
How would a For Loop allow you to watch for a file and process it the moment it arrives?

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
MyDoggieJessie
MyDoggieJessie
SSChampion
SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)

Group: General Forum Members
Points: 12162 Visits: 7444
At the package level create a package-level variable, set it to data type = String

Create a connection manager for the file (only really needed for the initial set up, as the variable will handle the name of the file when the package is executed)

In General Properties of the control:
> Add a ForEach Loop Control
> Specify the Foreach File Enumerator
Under the Enumerator Configuration properties:
> Specify the folder to monitor
> Specify a file, file type, extensions, etc (wild cards work - i.e. MyFile*.txt)
> Set the file name to "Fully qualified"
Under Variable Mappings:
> Assign the package-level variable you created, setting the index to 0

Add an Execute Script task, add:
EXEC msdb.dbo.sp_start_job N'MySQLAgentJobToExecuteWhenFilesPresent'



Add this package to a SQL Agent job that executes the package.

______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience" ;-)
Orlando Colamatteo
Orlando Colamatteo
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39210 Visits: 14411
Ermm That doesn't really get you there. The key operative word in the ask is "watch."

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
MyDoggieJessie
MyDoggieJessie
SSChampion
SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)

Group: General Forum Members
Points: 12162 Visits: 7444
Sorry I have to disagree...I believe it would work perfectly.

If the OP created a job that ran the package I'm describing say every 5 seconds, the package does a check for a file(s) present in the folder. If no files exist when the job runs the loop container does nothing and the job completes successfully. In the event there is a file present, the execute SQL task inside the container fires off the code to execute the SQL Agent job they want to fire based upon the existence of a file.

I have an identical process in place (but checking only every 5 mins). Obviously there's more elegant solutions available but this will work for the issue they're describing.

______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience" ;-)
Orlando Colamatteo
Orlando Colamatteo
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39210 Visits: 14411
What is there to disagree about? That's not the same as watching a directory and processing a file as soon as it exists.
Depending on the size of the package there is a good chance the package could not even be validated and complete in under 5 seconds when run via Agent.

This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
MyDoggieJessie
MyDoggieJessie
SSChampion
SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)SSChampion (12K reputation)

Group: General Forum Members
Points: 12162 Visits: 7444
This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds
This is a good point, I didn't give any thought to it and that it would become an issue - thanks much for pointing it out :-)

______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience" ;-)
Orlando Colamatteo
Orlando Colamatteo
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39210 Visits: 14411
If you have a look at any of the three articles I linked to above you'll see where I talk about this issue and the pros of "watching" for a file versus starting a package every n-seconds or minutes that exits of the file is not there.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search