February 13, 2018 at 10:14 am
Hi first post I think this is the right forum. A bit of a novice but working on it.
I am working with an Arcgis Enterprise SDE Database in SQL Server 2008.
I am attempting to use a SQL update query to update a new field for many of the records in the database. This field is populated from a table that is created in other processing steps. In the query I update Table Views, which can then be QA/QC'd before being integrating into the database.
(see below for code)
When I test the query in SSMS, for all the datasets I want to update, it works with no errors. The issue is that when I attempt to run the same query via a batch file in command line it hangs indefinitely (16+ hours with no response). It does not even output messages to suggest it has started processing. I have no idea if it is a specific table that is causing the problem, or if it is throwing some kind of error.
My major confusion is that it only hangs when I run the test for all of the datasets. If I run the query for a small subset of the tables (1 or 2) it works in both SSMS and in Command Line and prints outputs to command line as intended.
My DBA insists that my query will not work, even though it works without error in SSMS. She says I must use a cursor to do this. However she has yet to give any kind of explanation as to why she believes this. Thus I am asking the community.
My main question is, what is happening here? Or, if this is not enough information, how can I potentially debug this to identify and fix the issue?
Thanks!!
SQL QUERY:SET NOCOUNT ON
EXEC dbase.sde.set_current_version "VERSION.TEST";
EXEC dbase.sde.edit_version "VERSION.TEST", 1;
BEGIN TRY
BEGIN TRAN;
RAISERROR ('B processing begun', 10,1) WITH NOWAIT
UPDATE BV
SET BV.MA =
(
SELECT BJ.MC FROM BJ
WHERE BJ.Key = BV.Key
AND BJ.MA <> BJ.MC
)
WHERE EXISTS
(
SELECT 1 FROM BJ
WHERE BJ.Key = BV.Key
AND BJ.MA <> BJ.MC
);
-- Repeat above for each desired dataset starting with RAISEERROR
RAISERROR ('All Processing Complete, Commiting Transaction.', 10,1) WITH NOWAIT
COMMIT;
EXEC devsde.sde.edit_version "SYOKLIC1.TEST", 2;
END TRY
BEGIN CATCH
COMMIT;
EXEC dbase.sde.edit_version "VERSION.TEST", 2;
PRINT '*************Error Detail****************';
PRINT 'Error Number :' + CAST(ERROR_NUMBER() AS VARCHAR);
PRINT 'Error Severity:' + CAST(ERROR_SEVERITY() AS VARCHAR);
PRINT 'Error State :' + CAST(ERROR_STATE() AS VARCHAR);
PRINT 'Error Line :' + CAST(ERROR_LINE() AS VARCHAR);
PRINT 'Error Message :' + ERROR_MESSAGE();
END CATCH
BATCH FILE:@echo on
python preprocessing.py
sqlcmd -U username -P password -S server\dbase -i myquery.sql
python postprocessing.py
pause
February 13, 2018 at 11:36 am
Well first, what happens when you try to run just the sqlcmd part of your batch file?
February 13, 2018 at 2:43 pm
It is very possible the problem is the connection settings, most likely ARITHABORT, which SSMS sets to ON, but SQLCMD sets to OFF.
Microsoft says this should always be set to ON, as it is in SSMS, even though for .Net and other applications the default is OFF:
https://docs.microsoft.com/en-us/sql/t-sql/statements/set-arithabort-transact-sql
You can configure an instance to default all connections to ARITHABORT ON using the server properties, connections page, arithmetic abort setting:
https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/server-properties-connections-page
February 13, 2018 at 3:52 pm
Why are you doing a COMMIT in your CATCH block instead of a ROLLBACK which would be more logical.
February 13, 2018 at 3:54 pm
I don't use a lot of batch scripts so just curious what the pause at the end of batch statements is doing.
February 14, 2018 at 6:44 am
Lynn Pettis - Tuesday, February 13, 2018 3:54 PMI don't use a lot of batch scripts so just curious what the pause at the end of batch statements is doing.
That's probably there so any output from the commands in the batch can be seen on the screen, otherwise the command window would just close when the batch script was complete. The other option that people use to get around that side effect is to redirect standard output to a text file to essentially log any results the commands return using the > operator to create a new text file and >> operator to append to an existing file.
February 14, 2018 at 8:06 am
Chris Harshman - Wednesday, February 14, 2018 6:44 AMLynn Pettis - Tuesday, February 13, 2018 3:54 PMI don't use a lot of batch scripts so just curious what the pause at the end of batch statements is doing.That's probably there so any output from the commands in the batch can be seen on the screen, otherwise the command window would just close when the batch script was complete. The other option that people use to get around that side effect is to redirect standard output to a text file to essentially log any results the commands return using the > operator to create a new text file and >> operator to append to an existing file.
The easiest way is to just run your batch script in a cmd /c "<batch file>" wrapper from the command prompt.
February 14, 2018 at 7:16 pm
Hi all, thank you for your interest in my question.
To answer sequentially - I hope this makes sense -
1. When I run the batch file for only the sqlcmd piece it behaves the same, essentially, nothing happens. It opens command prompt and then just sits there.
2. Connection Setting makes a lot of sense to me, or something similar. It did not make sense to me why it would work in one place but not the other. That being said does that account for why the small query would work but the full query would not? I have not been able to mess with this yet but intend to at my soonest opportunity.
3. The COMMIT, CATCH, ROLLBACK issue has been brought to my attention and I intend to change it.
4. In response to the conversation about the 'pause'. I do intended to eventually have the outputs write to a log file but have found it easier to just use the 'pause' to leave the command window open for testing, which is what it does.
Sorry for taking so long to respond I appreciate the input!! 😀
February 15, 2018 at 7:16 am
What happens if you execute the following SQLCMD command from a command prompt?sqlcmd -U username -P password -S server\dbase -Q"SELECT * FROM sys.databases"
This command will just execute a simple and fast query and should return a list of all databases on the instance.
If this command will also hang with no output, open SSMS and connect to the instance. Query the sys.dm_exec_sessions DMV (see below) to check if the SQLCMD is actually connected to the instance. If not there is a problem (typo?) with the specified server/instance name or perhaps there is a firewall or alike blocking the connection.
DMV query:select * from sys.dm_exec_sessions where session_id > 50
February 21, 2018 at 9:22 am
HanShi - Thursday, February 15, 2018 7:16 AMWhat happens if you execute the following SQLCMD command from a command prompt?sqlcmd -U username -P password -S server\dbase -Q"SELECT * FROM sys.databases"This command will just execute a simple and fast query and should return a list of all databases on the instance.
If this command will also hang with no output, open SSMS and connect to the instance. Query the sys.dm_exec_sessions DMV (see below) to check if the SQLCMD is actually connected to the instance. If not there is a problem (typo?) with the specified server/instance name or perhaps there is a firewall or alike blocking the connection.DMV query:
select * from sys.dm_exec_sessions where session_id > 50
Hi HanShi,
The outputs of the first query are below. It does not hang it executes very quickly, but it outputs the below errors. I'm not sure what that implies, it looks like a connectivity/access issue. I have relatively limited permissions to this database as I am not a DBA, if that is a consideration.
Error messages:
"
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : SQL Server Network Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF]. .
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : Login timeout expired.
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online..
"
I ran the other query in SSMS just to see what would happen, it returned a single record referencing my user name, some security information, and I believe a date from the last time I tested my query.
Thanks for your help!
February 21, 2018 at 11:54 am
sjyoklic - Wednesday, February 21, 2018 9:22 AMHi HanShi,
The outputs of the first query are below. It does not hang it executes very quickly, but it outputs the below errors. I'm not sure what that implies, it looks like a connectivity/access issue. I have relatively limited permissions to this database as I am not a DBA, if that is a consideration.
Error messages:
"
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : SQL Server Network Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF]. .
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : Login timeout expired.
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online..
"I ran the other query in SSMS just to see what would happen, it returned a single record referencing my user name, some security information, and I believe a date from the last time I tested my query.
Thanks for your help!
The error message from the SQLCMD indicates an error during the connection to the SQL server. That could have different rootcauses:
* typo in the servername/instance on the SQLCMD commandline
* firewall somewhere between your host and the the SQL server
* the SQL server is setup to use a different listening port (which in that case you need to specify on the commandline)
One more question:
Are you running the SQLCMD on the same host as you use for SSMS? If so, most likely only the first rootcause stated above is the one to double check.
February 22, 2018 at 2:09 pm
HanShi - Wednesday, February 21, 2018 11:54 AMsjyoklic - Wednesday, February 21, 2018 9:22 AMHi HanShi,
The outputs of the first query are below. It does not hang it executes very quickly, but it outputs the below errors. I'm not sure what that implies, it looks like a connectivity/access issue. I have relatively limited permissions to this database as I am not a DBA, if that is a consideration.
Error messages:
"
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : SQL Server Network Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF]. .
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : Login timeout expired.
Sqlcmd: Error: Microsoft ODBC Driver 13 for SQL Server : A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online..
"I ran the other query in SSMS just to see what would happen, it returned a single record referencing my user name, some security information, and I believe a date from the last time I tested my query.
Thanks for your help!
The error message from the SQLCMD indicates an error during the connection to the SQL server. That could have different rootcauses:
* typo in the servername/instance on the SQLCMD commandline
* firewall somewhere between your host and the the SQL server
* the SQL server is setup to use a different listening port (which in that case you need to specify on the commandline)One more question:
Are you running the SQLCMD on the same host as you use for SSMS? If so, most likely only the first rootcause stated above is the one to double check.
It turns out there was a typo in the test query (the one you suggested I run). That is what caused the error. However, there was no typo in batch file. When I fixed the test query it returned the list of databases, as it should have.
February 22, 2018 at 2:23 pm
ZZartin - Tuesday, February 13, 2018 11:36 AMWell first, what happens when you try to run just the sqlcmd part of your batch file?
My above response where I say that running the SQL only as part of the batch file did not do anything different than running them all together is still the case. However, something potentially interesting happened.
I was doing a test run and I made a mistake. I forgot to create the "version" as designated by the EXEC in the beginning of the query and I forgot to register the tables as versioned. I have a script that does these steps but I mistakenly thought I ran it when I did not. This mistake means that the table views that are being edited do not exist, and the version where the changes are being registered to also does not exist.
What happened was that the query ran, output an error that said "the version cannot be found" and then committed the changes to the default tables instead.
What that leads me to believe is that the problem has to do with accessing the "Version" I designate or the Table Views themselves. But, I am still not sure why this would only be a problem that shows up with the batch file and not in SSMS.
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply