October 11, 2009 at 10:08 pm
Comments posted to this topic are about the item Parallel Processing
October 12, 2009 at 2:30 am
This is an excellent article the gave an idea have to solve some problems that arise from the need of paraller processing. Thank you very very much.
As all we want is right resource planning I show in the following article that its better to create temporary tables and use INSERT INTO instead of SELECT INTO. As the article says its a resources matter.
The article is here:http://www.sqlservercentral.com/articles/Basic+Querying/temptablesinsqlserver/1279/
Thank you.
October 12, 2009 at 2:36 am
As I am new to SQL i cannot understand why you are using the statement
SELECT * INTO #LockHelp FROM Rainfall_Events WITH (TABLOCKX) WHERE 1=2. This statement does not fetch any rows?
I't must be somenthing about the lock time, bit can you explain me why you are using "WHERE 1=2"?
Thank you
October 12, 2009 at 3:12 am
iermis (10/12/2009)
As I am new to SQL i cannot understand why you are using the statementSELECT * INTO #LockHelp FROM Rainfall_Events WITH (TABLOCKX) WHERE 1=2. This statement does not fetch any rows?
It must be something about the lock time, bit can you explain me why you are using "WHERE 1=2"?
Thank you
I think Sunny is using this "WHERE 1 = 2" for creating the #LockHelp table only, having no data.
But not sure why, because he has not refer #LockHelp in the SP.
October 12, 2009 at 3:14 am
Nice article. Keep it up.
October 12, 2009 at 3:16 am
Hi iermis
Thanks for the comments.
The query you mentioned is only for getting a Table Lock on the table. We dont actually need any records from the table but at the same time we want to have the table locked from the Rest_OrphanLocks SP until the records are marked as processed. The condition "1=2" does exactly that. It does not spend time retrieving records but locks the table. #LockHelp Temporary table is optional. We could as well have a SELECT query instead of SELECT INTO
Thanks
Satish More
October 12, 2009 at 3:52 am
mohd.nizamuddin (10/12/2009)
I think Sunny is using this "WHERE 1 = 2" for creating the #LockHelp table only, having no data.
But not sure why, because he has not refer #LockHelp in the SP.
Thanks Nizamuddin. #LockHelp is optional. You can as well have only a SELECT query instead of SELECT INTO. The only difference is that it returns an empty result set back to the SP caller
Thanks
Satish More
October 12, 2009 at 4:02 am
Thank you all for your fast replies.
🙂
October 12, 2009 at 4:33 am
I recently (within the last 2 years) created a system to monitor our SQL instances from a central instance. Without going into details about the whole process, I had a requirement to run multiple tasks to retrieve data from the monitored instances in parallel (one or more instances unavailable would cause delays in retrieving data from all the other instances).
There's an easier way to do this than messing with lock hints, use the UPDATE ... OUTPUT format of the UPDATE query. Here's a snippet of the code that I use:
-- ******************************************************************************************
-- Create the local temp tables used in this proc
CREATE TABLE #DRT_Update (
DataRetrievalTaskIDINT)
-- ******************************************************************************************
-- ******************************************************************************************
-- Clear the temp table ...
ProcessNextServer:
DELETE #DRT_Update
-- ******************************************************************************************
-- ******************************************************************************************
-- Get the next Data Retrieval Task to be processed - set the status to 2 for the record selected.
UPDATE DataRetrievalTask
SET [Status] = 2
OUTPUT inserted.DataRetrievalTaskID
INTO #DRT_Update
WHERE DataRetrievalTaskID = (SELECT MIN(DataRetrievalTaskID)
FROM DataRetrievalTask
WHERE [Status] = 1
AND StartTime <= GETDATE() )
-- ******************************************************************************************
This process is easily scalable - I just have to create another job to execute the Stored Proc to add another process. This way, you dont have to worry about Locks etc - the SQL Engine does it for you.
October 12, 2009 at 4:39 am
Can you explain the code?
October 12, 2009 at 5:08 am
Simon's post about using the OUTPUT clause on the update becomes available for us to use in SQL 2005 databases. It is an awesome feature (Oracle has had it for a while).
October 12, 2009 at 5:22 am
Really interesting article.
Could the clean up process use something like sp_who to find out if a process was currently running or orphaned. I'm always a bit dubious about things that depend on timings which would need to be tuned for each installation depending upon resources, data sizes etc.
October 12, 2009 at 6:41 am
[p]Scott was exactly right. The OUTPUT clause of the UPDATE allows us to get data back from the inserted and deleted tables (the same as we reference in DML triggers). In the case of my code snippet, I always want to process the first record in the table with Status = 1 (Ready to Process) and a StartTime < Now. I contemplated using Transactions and Setting Lock hints, but it was a lot of work compared to using the UPDATE ... OUTPUT statement. This way the record is updated to Status = 2 (In Process) and the row ID value returned to the code in a single statement - no extended or manually coded locking required. [/p]
[p]BOL has a good article on the OUTPUT clause; but a couple of points - (1) you can get back any data from the inserted or deleted tables, not just a single column and (2) the INTO table has to pre-exist, hence the CREATE TABLE statement at the top of the code.[/p]
[p]The Code snippet is written as part of a loop (using the forbidden GOTO statement - lets see who jumps on that, but that's a different discussion), hence the DELETE statement.[/p]
Hope that helps.
October 12, 2009 at 7:06 am
Kevin.McEvoy (10/12/2009)
Really interesting article.Could the clean up process use something like sp_who to find out if a process was currently running or orphaned. I'm always a bit dubious about things that depend on timings which would need to be tuned for each installation depending upon resources, data sizes etc.
Thanks Kevin. Yes I too tried digging into any such readily available command which could tell whether the process is currently active. Hope to get more leads in this forum. The UPDATE..OUTPUT statement suggested by Simon appears to be very helpful. Thanks Simon 🙂
Thanks
Satish
October 12, 2009 at 8:22 am
Interesting article....I'm surprised at the hoops you have to jump through.
On DB2 for i, all I need to do is set QQRYDEGREE for the system or for individual jobs to something besides *NONE (or *NBRTASKS 1).
http://www-03.ibm.com/servers/enable/site/education/abstracts/4aea_abs.html
From the above course:
How SMP works
The DB2 SMP feature provides the optimizer and database engine with additional methods and strategies for retrieving data and processing data in a parallel manner. SMP enables database parallelism on a single system or LPAR - where multiple (CPU and I/O) processors that share memory and disk resources - to work simultaneously toward achieving a single end result. This parallel processing means that the database engine can have more than one (or all) the processors working on a single query at the same time. You can significantly improve the performance of a processor-bound query with this feature on multiple-processor systems by distributing the processor load across more than one processor.
Although using SMP does not require the presence of more than one processor, database parallelism is most effective when more than one physical processor is available to run the tasks or threads.
Given that SMP is achieved through the use of the System i server and its i5/OS advanced architecture, table partitioning is not required for database parallelism.
Charles Wilt
Viewing 15 posts - 1 through 15 (of 31 total)
You must be logged in to reply to this topic. Login to reply