Execution of SQL Job, the moment a file exist

  • Hi,

    I have a scenario..

    I have a SQL Job which needs to be executed based on demand, means the moment it finds a file in a path it should execute. if there is no file in a path it should not execute. Execution or start of the SQL Job should be based on the existence of a file.

    Assume i have folder D:\jobfiles\ , in this folder the moment a file is dumped, it should start the job else invoke of SQL job should not happen.

    With Regards

    Dakshina Murthy

  • A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

    You could also do this a few different ways using an SSIS package that runs continuously watching for files:

    Using the WMI Event Watcher Task in SSIS to Process Data Files[/url]

    Using the Konesans File Watcher Task in SSIS to Process Data Files[/url]

    Using the Script Task in SSIS to Process Data Files When They Arrive[/url]

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

  • opc.three (2/11/2013)


    A SQL Agent Alert that leverages WMI could run a Job for you. Lookup SQL Agent Alerts and the WMI Event Class __InstanceCreationEvent.

    You could also do this a few different ways using an SSIS package that runs continuously watching for files:

    Using the WMI Event Watcher Task in SSIS to Process Data Files[/url]

    Using the Konesans File Watcher Task in SSIS to Process Data Files[/url]

    Using the Script Task in SSIS to Process Data Files When They Arrive[/url]

    +1 to the SSIS option. Use a For...Loop container to search through the folder you want to monitor. It's really simple

    ______________________________________________________________________________Never argue with an idiot; Theyll drag you down to their level and beat you with experience

  • How would a For Loop allow you to watch for a file and process it the moment it arrives?

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

  • At the package level create a package-level variable, set it to data type = String

    Create a connection manager for the file (only really needed for the initial set up, as the variable will handle the name of the file when the package is executed)

    In General Properties of the control:

    > Add a ForEach Loop Control

    > Specify the Foreach File Enumerator

    Under the Enumerator Configuration properties:

    > Specify the folder to monitor

    > Specify a file, file type, extensions, etc (wild cards work - i.e. MyFile*.txt)

    > Set the file name to "Fully qualified"

    Under Variable Mappings:

    > Assign the package-level variable you created, setting the index to 0

    Add an Execute Script task, add:

    EXEC msdb.dbo.sp_start_job N'MySQLAgentJobToExecuteWhenFilesPresent'

    Add this package to a SQL Agent job that executes the package.

    ______________________________________________________________________________Never argue with an idiot; Theyll drag you down to their level and beat you with experience

  • :ermm: That doesn't really get you there. The key operative word in the ask is "watch."

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

  • Sorry I have to disagree...I believe it would work perfectly.

    If the OP created a job that ran the package I'm describing say every 5 seconds, the package does a check for a file(s) present in the folder. If no files exist when the job runs the loop container does nothing and the job completes successfully. In the event there is a file present, the execute SQL task inside the container fires off the code to execute the SQL Agent job they want to fire based upon the existence of a file.

    I have an identical process in place (but checking only every 5 mins). Obviously there's more elegant solutions available but this will work for the issue they're describing.

    ______________________________________________________________________________Never argue with an idiot; Theyll drag you down to their level and beat you with experience

  • What is there to disagree about? That's not the same as watching a directory and processing a file as soon as it exists.

    Depending on the size of the package there is a good chance the package could not even be validated and complete in under 5 seconds when run via Agent.

    This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

  • This is not to mention that having an Agent job run a job every 5 seconds would be a foolish thing to do when considering other aspects of the system. The calls Agent makes to msdb would bloat your plan cache to no end with only running a job every 15 seconds

    This is a good point, I didn't give any thought to it and that it would become an issue - thanks much for pointing it out 🙂

    ______________________________________________________________________________Never argue with an idiot; Theyll drag you down to their level and beat you with experience

  • If you have a look at any of the three articles I linked to above you'll see where I talk about this issue and the pros of "watching" for a file versus starting a package every n-seconds or minutes that exits of the file is not there.

    There are no special teachers of virtue, because virtue is taught by the whole community.
    --Plato

Viewing 10 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic. Login to reply