Append output from SQLCMD in batch file

Question

Append output from SQLCMD in batch file

carissa.chris

Old Hand

Points: 348
More actions
June 12, 2009 at 4:06 pm

#131118

I have a batch file that runs all the sql scripts in a specified folder. The output is written to output.txt. However, each time a sql script is executed, the output.txt file is overwritten. Is there a way to append to the output.txt file?
Here is my batch file:
for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"
for %%a in ("E:\Source\APP_30\SQL\SP\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"
for %%a in ("E:\Source\APP_30\SQL\Triggers\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

Viewing 15 posts - 1 through 15 (of 19 total)

You must be logged in to reply to this topic. Login to reply

WayneS SSC Guru Points: 95409 More actions · Answer 1

carissa.chris (6/12/2009)
I have a batch file that runs all the sql scripts in a specified folder. The output is written to output.txt. However, each time a sql script is executed, the output.txt file is overwritten. Is there a way to append to the output.txt file?
Here is my batch file:
for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"
for %%a in ("E:\Source\APP_30\SQL\SP\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"
for %%a in ("E:\Source\APP_30\SQL\Triggers\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

Well, BOL clearly states that the output file will be overwritten.

I suggest you make the output filename dynamic:

for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\%%aoutput.txt"

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 2

Leave the first "-o" alone. Change all of the other "-o" to ">>".

Old DOS warrior trick. 😛

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

WayneS SSC Guru Points: 95409 More actions · Answer 3

Jeff Moden (6/12/2009)
Leave the first "-o" alone. Change all of the other "-o" to ">>".
Old DOS warrior trick. 😛

Ahem. It's not very often that someone gets to tell Jeff that he's wrong, AND can prove it...

for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\SP\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\Triggers\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

Jeff's suggestion is to change it to:

for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\SP\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a >> "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\Triggers\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a >> "E:\output.txt"

The first line is processing each .sql file in the "Structure" directory. For each file, it runs the SQLCMD, which will have the -o parameter, overwriting the output file if it already exists. So if there are 3 .sql files in this directory, the output will only have the output from the third file.

It's not until you get to the second "for" line that Jeff's suggestion will start appending the results to the file.

Edit: added this suggestion:

if you were to start off with:

echo %date% %time% > E:\output.txt

you could then replace ALL of the -o with >>

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 4

WayneS (6/12/2009)
Ahem. It's not very often that someone gets to tell Jeff that he's wrong, AND can prove it...

Heh... wrong for what? If you want to keep the log forever, then yes, you need to replace the first "-o" with the ">>". But if you want the log to be only for the batch, the leave the first "-o" as it is. 😛 Didn't know which way the OP wanted to go and I probably should have said so. :hehe:

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

WayneS SSC Guru Points: 95409 More actions · Answer 5

Okay, here's some code to demonstrate what I'm talking about.

Save this code as a batch file, and run it:

for %%a in ("%windir%\*.exe") do SQLCMD -S ServerName -d master -E -Q"set nocount on;select '%%a'" -o "%temp%\output.txt"

type %temp%\output.txt

pause

cls

echo %date% %time% > "%temp%\output.txt

for %%a in ("%windir%\*.exe") do SQLCMD -S ServerName -d master -E -Q"set nocount on;select '%%a'" >> "%temp%\output.txt"

type %temp%\output.txt

pause

del %temp%\output.txt

At the first pause, the contents of the output file are displayed. All that is in it is the last *.exe.

At the second pause, the contents of the output file are again displayed. The prior contents are erased, and all exe files are listed.

This demonstrates the point the OP was making.

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 6

WayneS (6/13/2009)
Okay, here's some code to demonstrate what I'm talking about.
Save this code as a batch file, and run it:
for %%a in ("%windir%\*.exe") do SQLCMD -S ServerName -d master -E -Q"set nocount on;select '%%a'" -o "%temp%\output.txt"
type %temp%\output.txt
pause
cls
echo %date% %time% > "%temp%\output.txt
for %%a in ("%windir%\*.exe") do SQLCMD -S ServerName -d master -E -Q"set nocount on;select '%%a'" >> "%temp%\output.txt"
type %temp%\output.txt
pause
del %temp%\output.txt
At the first pause, the contents of the output file are displayed. All that is in it is the last *.exe.
At the second pause, the contents of the output file are again displayed. The prior contents are erased, and all exe files are listed.
This demonstrates the point the OP was making.

You've added a line of code the OP didn't have in the original problem and that new line is the one responsible for the incorrect operation.

echo %date% %time% > "%temp%\output.txt

You've neglected to use ">>" in that line of code and that's what resets file. 😉

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

WayneS SSC Guru Points: 95409 More actions · Answer 7

Jeff Moden (6/14/2009)
You've added a line of code the OP didn't have in the original problem and that new line is the one responsible for the incorrect operation.
echo %date% %time% > "%temp%\output.txt
You've neglected to use ">>" in that line of code and that's what resets file. 😉

Correct, I reset the file between the runs, to demonstrate the effects of the two different methods. If the OP wants the file reset between runs, then use this line. If not, then get rid of it.

But the point was that with using the "-o" in the first part, the file is reset for every file being processed. Which is the behavior that the OP was trying to get rid of.

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 8

WayneS (6/14/2009)
Jeff Moden (6/14/2009)
You've added a line of code the OP didn't have in the original problem and that new line is the one responsible for the incorrect operation.
echo %date% %time% > "%temp%\output.txt
You've neglected to use ">>" in that line of code and that's what resets file. 😉
Correct, I reset the file between the runs, to demonstrate the effects of the two different methods. If the OP wants the file reset between runs, then use this line. If not, then get rid of it.
But the point was that with using the "-o" in the first part, the file is reset for every file being processed. Which is the behavior that the OP was trying to get rid of.

Understood. My point is that I wasn't sure if the OP was going for a continuous "forever" log or just wanted a single log with the results of all the individual steps for a single run of the batch. I posted the solution for the latter. It wasn't wrong, it just wasn't what you expected. 🙂

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

matt stockham SSCrazy Eights Points: 9942 More actions · Answer 9

Jeff Moden (6/14/2009)
Understood. My point is that I wasn't sure if the OP was going for a continuous "forever" log or just wanted a single log with the results of all the individual steps for a single run of the batch. I posted the solution for the latter. It wasn't wrong, it just wasn't what you expected. 🙂

I agree with Wayne. Your method wouldn't give a "forever" log or the results of all steps ... if there were 2 files for example, the results of the first iteration of the first step would be lost as the second iteration would overwrite the log.

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 10

matt stockham (6/14/2009)
Jeff Moden (6/14/2009)
Understood. My point is that I wasn't sure if the OP was going for a continuous "forever" log or just wanted a single log with the results of all the individual steps for a single run of the batch. I posted the solution for the latter. It wasn't wrong, it just wasn't what you expected. 🙂
I agree with Wayne. Your method wouldn't give a "forever" log or the results of all steps ... if there were 2 files for example, the results of the first iteration of the first step would be lost as the second iteration would overwrite the log.

Heh... another one. I didn't say it would give a "forever" log and didn't write the code that way. I wrote the code to give a "complete" log of all steps for the given batch. If you want a "forever" log, you have to replace ALL the "-o" with ">>".

I've really got to take the target of my back. 😛

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

WayneS SSC Guru Points: 95409 More actions · Answer 11

Jeff Moden (6/14/2009)
Heh... another one. I didn't say it would give a "forever" log and didn't write the code that way. I wrote the code to give a "complete" log of all steps for the given batch. If you want a "forever" log, you have to replace ALL the "-o" with ">>".

But Jeff, that is where you made your error.

The SQLCMD is not running once, and processing all files in the directory, It is being run repeatedly for EACH file in the specified directory. With using the -o parameter, the output file is overwritten each time it is executed. Resulting in only the LAST file (of that first directory) being in the log... this is NOT a complete log, which is what the OP wants.

I've really got to take the target of my back. 😛

I'm not trying to shoot at you. I just see that you're missing a minor point here, which makes a big difference in what the OP wants. Heck, without your initial post, I wouldn't have been able to figure out how to solve the OP's issue. You gave me what was needed to complete it.

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Florian Reischl SSC-Dedicated Points: 37301 More actions · Answer 12

Jeff,

sorry, but seems that I'm a second guy who agrees with Wayne.

Your suggestion:

for %%a in ("E:\Source\APP_30\SQL\Structure\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a -o "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\SP\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a >> "E:\output.txt"

for %%a in ("E:\Source\APP_30\SQL\Triggers\*.sql") do SQLCMD -S ServerName -d DatabaseName -E -i %%a >> "E:\output.txt"

If there are more than one SQL file in "E:\Source\APP_30\SQL\Structure" you overwrite the previous outputs 😉

Flo

carissa.chris Old Hand Points: 348 More actions · Answer 13

I wanted a "forever" log and the ">> E:\output.txt" accomplishes exactly what I needed. Thanks for all the input.

🙂

Jeff Moden SSC Guru Points: 1003863 More actions · Answer 14

Thanks for the feedback Carissa.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)