Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Execution of SQL Job, the moment a file exist Expand / Collapse
Author
Message
Posted Monday, February 11, 2013 4:37 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Monday, July 21, 2014 3:13 AM
Points: 330, Visits: 960
Hi,

I have a scenario..

I have a SQL Job which needs to be executed based on demand, means the moment it finds a file in a path it should execute. if there is no file in a path it should not execute. Execution or start of the SQL Job should be based on the existence of a file.

Assume i have folder D:\jobfiles\ , in this folder the moment a file is dumped, it should start the job else invoke of SQL job should not happen.

With Regards
Dakshina Murthy
Post #1418315
Posted Monday, February 11, 2013 7:29 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Monday, July 21, 2014 4:52 PM
Points: 7,079, Visits: 12,569
A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

You could also do this a few different ways using an SSIS package that runs continuously watching for files:

Using the WMI Event Watcher Task in SSIS to Process Data Files
Using the Konesans File Watcher Task in SSIS to Process Data Files
Using the Script Task in SSIS to Process Data Files When They Arrive


__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1418418
Posted Monday, February 11, 2013 8:11 PM


Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Monday, July 14, 2014 2:06 PM
Points: 3,860, Visits: 7,130
opc.three (2/11/2013)
A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

You could also do this a few different ways using an SSIS package that runs continuously watching for files:

Using the WMI Event Watcher Task in SSIS to Process Data Files
Using the Konesans File Watcher Task in SSIS to Process Data Files
Using the Script Task in SSIS to Process Data Files When They Arrive
+1 to the SSIS option. Use a For...Loop container to search through the folder you want to monitor. It's really simple


______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience"
Post #1418718
Posted Monday, February 11, 2013 8:21 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Monday, July 21, 2014 4:52 PM
Points: 7,079, Visits: 12,569
How would a For Loop allow you to watch for a file and process it the moment it arrives?

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1418723
Posted Monday, February 11, 2013 8:59 PM


Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Monday, July 14, 2014 2:06 PM
Points: 3,860, Visits: 7,130
At the package level create a package-level variable, set it to data type = String

Create a connection manager for the file (only really needed for the initial set up, as the variable will handle the name of the file when the package is executed)

In General Properties of the control:
> Add a ForEach Loop Control
> Specify the Foreach File Enumerator
Under the Enumerator Configuration properties:
> Specify the folder to monitor
> Specify a file, file type, extensions, etc (wild cards work - i.e. MyFile*.txt)
> Set the file name to "Fully qualified"
Under Variable Mappings:
> Assign the package-level variable you created, setting the index to 0

Add an Execute Script task, add:
EXEC msdb.dbo.sp_start_job N'MySQLAgentJobToExecuteWhenFilesPresent'

Add this package to a SQL Agent job that executes the package.


______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience"
Post #1418730
Posted Monday, February 11, 2013 9:19 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Monday, July 21, 2014 4:52 PM
Points: 7,079, Visits: 12,569
That doesn't really get you there. The key operative word in the ask is "watch."

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1418735
Posted Monday, February 11, 2013 9:28 PM


Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Monday, July 14, 2014 2:06 PM
Points: 3,860, Visits: 7,130
Sorry I have to disagree...I believe it would work perfectly.

If the OP created a job that ran the package I'm describing say every 5 seconds, the package does a check for a file(s) present in the folder. If no files exist when the job runs the loop container does nothing and the job completes successfully. In the event there is a file present, the execute SQL task inside the container fires off the code to execute the SQL Agent job they want to fire based upon the existence of a file.

I have an identical process in place (but checking only every 5 mins). Obviously there's more elegant solutions available but this will work for the issue they're describing.


______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience"
Post #1418739
Posted Monday, February 11, 2013 10:14 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Monday, July 21, 2014 4:52 PM
Points: 7,079, Visits: 12,569
What is there to disagree about? That's not the same as watching a directory and processing a file as soon as it exists.
Depending on the size of the package there is a good chance the package could not even be validated and complete in under 5 seconds when run via Agent.

This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds


__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1418747
Posted Tuesday, February 12, 2013 6:15 AM


Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Monday, July 14, 2014 2:06 PM
Points: 3,860, Visits: 7,130
This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds
This is a good point, I didn't give any thought to it and that it would become an issue - thanks much for pointing it out


______________________________________________________________________________
"Never argue with an idiot; They'll drag you down to their level and beat you with experience"
Post #1418922
Posted Tuesday, February 12, 2013 7:39 AM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Monday, July 21, 2014 4:52 PM
Points: 7,079, Visits: 12,569
If you have a look at any of the three articles I linked to above you'll see where I talk about this issue and the pros of "watching" for a file versus starting a package every n-seconds or minutes that exits of the file is not there.

__________________________________________________________________________________________________
There are no special teachers of virtue, because virtue is taught by the whole community. --Plato
Post #1418967
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse