SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Best code for generating sequence numbers


Best code for generating sequence numbers

Author
Message
jaffar.yelavalli
jaffar.yelavalli
SSC Rookie
SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)

Group: General Forum Members
Points: 27 Visits: 18
Hi All,
We have a requirement where we need to generate IDs from a table.
For each kind of ID we have a row defined with a name.
ex Table:
IDName next_value
ShipID 1
PackingID 1

What would be best code sequence where we would end up with no deadlocks, blockings or duplicates IDs.

Tweaking of code I am looking for is:

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN

select @nextvalue = next_value
from table_num_scheme where name = 'ShipID'

update table_num_scheme set next_value = (next_value + 1) where name = 'ShipID'

COMMIT TRAN GRAB_SHP_NBR

SET TRANSACTION ISOLATION LEVEL READ COMMITTED

Any other suggestions to generate the sequence numbers from the same row.
Lynn Pettis
Lynn Pettis
SSC-Dedicated
SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)SSC-Dedicated (39K reputation)

Group: General Forum Members
Points: 39201 Visits: 38529
How about something like this?

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN
update table_num_schema set
@nextvalue = next_value = next_value + 1;
where
name = 'ShipID'

COMMIT TRAN GRAB_SHP_NBR

SET TRANSACTION ISOLATION LEVEL READ COMMITTED

set @nextvalue = @nextvalue - 1;



Cool
Lynn Pettis

For better assistance in answering your questions, click here
For tips to get better help with Performance Problems, click here
For Running Totals and its variations, click here or when working with partitioned tables
For more about Tally Tables, click here
For more about Cross Tabs and Pivots, click here and here
Managing Transaction Logs

SQL Musings from the Desert Fountain Valley SQL (My Mirror Blog)
jaffar.yelavalli
jaffar.yelavalli
SSC Rookie
SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)SSC Rookie (27 reputation)

Group: General Forum Members
Points: 27 Visits: 18
That seems great.
I tried the same but with default read committed isolation and ended up with duplicates.
I am quite unable to understand why I am running into duplicates even though I am doing everything in an atomic transaction:

Code I already tried is:
Begin Tran
UPDATE table_num_scheme SET @nextvalue = next_value = (next_value + 1) WHERE name = 'Ship ID'
Commit tran

Begin Tran
UPDATE table_num_scheme with(rowlock, updlock) SET @nextvalue = next_value = (next_value + 1) WHERE name = 'Ship ID'
Commit tran

I haven't tried with serialization though.
Tergum Rufus
Tergum Rufus
SSC Journeyman
SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)

Group: General Forum Members
Points: 75 Visits: 108
5th Feb 2010
Lynn,
We have a mission critical app which has this problem in the very guts of the business logic and is causing deadlocks & major disruption on a 50 user installation. I am not a DBA but it appears that your tuning suggestion removes a read and thus halves the disk activity. I would also like to understand better the implications and effect of the two statements regarding transaction isolation levels and how the changes proposed below might affect our application.

I believe we could make an immediate improvement by just modifying our stored procedure as indicated by the lines below marked **:

ALTER PROCEDURE [dbo].[sp_nxnumber]
@Result Int output,
@NumberType Varchar(20)
AS
**SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRANSACTION
UPDATE nxno WITH (tablockx) SET NxNumber = NxNumber + 1 WHERE numberType = @NumberType
IF @@error <> 0
BEGIN
ROLLBACK TRANSACTION
RAISERROR ('NxNumber Update Error', 16, 2) WITH NOWAIT, SETERROR
RETURN 0
END
** REMOVE NEXT LINE
-- SET @Result = (Select NxNumber FROM NxNo WITH (tablockx) WHERE NumberType = @NumberType)

COMMIT TRANSACTION
**SET TRANSACTION ISOLATION LEVEL READ COMMITTED
**SET @Result = NxNumber

ANY OTHER COMMENTS WELCOME

Peter HORSLEY
Melbourne, Australia
Lamprey13
Lamprey13
Mr or Mrs. 500
Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)Mr or Mrs. 500 (562 reputation)

Group: General Forum Members
Points: 562 Visits: 1657
Admittedly, I’m not sure I fully understand the problem. But, the simple solution seems to be to use a IDENTITY column. Perhaps you need a different table for you “Numbers.” So if you Insert into that table you capture the newly created IDENTITY number and use that. It’s quick and sql handles the locking for you.
Michael Valentine Jones
Michael Valentine Jones
SSCertifiable
SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)SSCertifiable (5.7K reputation)

Group: General Forum Members
Points: 5720 Visits: 11771
If you really need to do something like this, you don't need to worry about transactions if you use the OUTPUT clause of the UPDATE statement to return the new value.

Also, you will reduce the posibility of deadlocking if you have one table for each table you are keeping a sequence number for, instead of having one table for all of them.

As mentioned before, IDENTITY is a much better way to handle this.


declare @MyOut table ( next_value int )

update table_num_scheme
set
next_value = next_value + 1
output
inserted.next_value
into
@MyOut
where
name = 'ShipID'

select next_value from @MyOut




Tergum Rufus
Tergum Rufus
SSC Journeyman
SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)

Group: General Forum Members
Points: 75 Visits: 108
Thank you for the feedback
Agreed, the IDENTITY column works fine for creating a unique RECNUM ID, but this type of process is required when creating a 'unique' sequential number that will link multiple records, possible in multiple tables, such as an INVOICE, EVENT or DOCUMENT number.

Really my question is about when one is obliged to use this technique, what is the optimum way to do it to maximise concurrency performance and avoid deadlocks. ie minimise disk activity and lock time. Using a separate table for each entity is a bit clunky because it would create a large record set that would need regular truncation.

In summary:
What are the relative merits of handling the transaction ISOLATION LEVEL and/or the TABLOCKX, HOLDLOCK query Optimiser Hint at the SP level? Are they the same thing?
Best wishes,
Peter
Jeff Moden
Jeff Moden
SSC Guru
SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)SSC Guru (85K reputation)

Group: General Forum Members
Points: 85641 Visits: 41082
peter-970097 (2/5/2010)
Thank you for the feedback
Agreed, the IDENTITY column works fine for creating a unique RECNUM ID, but this type of process is required when creating a 'unique' sequential number that will link multiple records, possible in multiple tables, such as an INVOICE, EVENT or DOCUMENT number.

Really my question is about when one is obliged to use this technique, what is the optimum way to do it to maximise concurrency performance and avoid deadlocks. ie minimise disk activity and lock time. Using a separate table for each entity is a bit clunky because it would create a large record set that would need regular truncation.

In summary:
What are the relative merits of handling the transaction ISOLATION LEVEL and/or the TABLOCKX, HOLDLOCK query Optimiser Hint at the SP level? Are they the same thing?
Best wishes,
Peter


If you remove all ISOLATION LEVEL commands and remove the explicit BEGIN TRAN/COMMIT from Lynn's code, 90% of your deadlocks will simply vanish. You'll need to move usage of the proc outside of any external transactions to get rid of the 10%.

So far as when to use something like this "sequence table" method in SQL Server goes, my answer would be "almost never". Improper design and use of a similar function caused an average of 640 deadlocks per day with spikes to 4,000 per day at a previous company I worked for. Yes, the new correctly written function solved most of that but even the new function which would allow you to "reserve" a given "increment" of numbers was a pain to use and, done incorrectly, would result in duplication of some IDs.

Here's the code that my old DBA and I came up with. The day we replaced the original code with the following code, the deadlocks dropped for about 640 to 12. Most of the deadlocks that remained were from other sources. I also have to tell you that all of the error checking we built into the code is ridiculous because the core code has never failed since we installed it. Since it takes virtually no time to do those error checks, we left them in.

If you can't figure out how to use the code to do setbased updates using the increment value instead of using a loop to do just one row at a time, please post back because it's an important concept to using such (ugh!) sequence tables in a setbased manner.

CREATE PROCEDURE dbo.GetNextID
/****************************************************************************************
Purpose:
This stored procedure is used to get a NextID for the table identified by the @KeyID
parameter. It will "reserve" a block of ID's according to the @IncrementValue parameter.
The @NextID returned is always the first ID of a reserved block of numbers. The reserved
block size defaults to 1.

Usage:
EXEC @return = dbo.GetNextID @KeyID,@IncrementValue,@NextID=@NextID OUTPUT

Outputs:
1. Returns a -1 if error and 0 if success.
2. @NextID will be a -1 if an error occured. Otherwise, it will contain the first
NextID of the requested block of NextID's.

Notes:
1. This procedure has been enhanced compared to the original...
a. The UPDATE statement sets both the @NextID variable and the NextID column in the
NextID table eliminating the need for a separate SELECT from NextID after the
UPDATE.
b. Because of (1.a) above, there is no longer a need for a transaction. If the
UPDATE didn't work, there is no need for a ROLLBACK because nothing was updated.
c. Previous error handling did not correctly return the invalid KeyID if present.
d. A test has been added to ensure a negative value for @IncrementValue was not
passed in.
e. A test to ensure that @NextID was correctly updated has been added.
f. Repairs to the previous error routines have been made so that the values returned
to @@ERROR and @@ROWCOUNT are correctly used by more than one statement.

Revisions:
REV 01 - 01 Mar 2005 - Kalpa Shah, Jeff Moden --Rewrite original
REV 02 - 06 Feb 2010 - Jeff Moden -- Removed all company references
****************************************************************************************/
--=======================================================================================
-- Define the I/O parameters used by this procedure
--=======================================================================================

--===== Declare the passed parameters
@KeyID INTEGER, --Identifies table to get the NextID for
@IncrementValue INTEGER = 1, --Number of NextIDs to "reserve"
@NextID INTEGER OUTPUT --Returns start # of block of IDs
AS

--=======================================================================================
-- Main body of procedure
--=======================================================================================

--===== Suppress auto-display of row counts for appearance and speed
SET NOCOUNT ON

--===== Declare variables local to the loop
DECLARE @MyError INTEGER --Holds @@ERROR for additional processing
DECLARE @ErrMessage VARCHAR(100) --Holds calculated error messages because RaisError
--cannot calulate messages on the fly.
DECLARE @MyRowCount INTEGER --Hold @@ROWCOUNT for additional processing

--===== Preset @NextID to an error condition
SET @NextID = -1 --Defaults don't work consistently on OUTPUT parameters

--===== If the increment is not greater than zero, raise and error and exit immediately
IF @IncrementValue <= 0
BEGIN --Start of error processing
--===== Process errors (RaisError cannot do calcs for error message)
SET @ErrMessage = 'The NextID row could not be updated. '
+ 'Increment was set to '
+ CONVERT(VARCHAR(11),@IncrementValue) + '.'
RAISERROR (@ErrMessage,1,1)
RETURN -1 --Returns an error indication to calling procedure
END --End of error processing

--===== Update the correct NextID row according to the KeyID passed in.
-- Sets @NextID and the column to the previous value + the increment
-- simultaneously so we don't need to read from the NextID table to
-- get the value of @NextID in the following steps.
UPDATE dbo.NextID WITH (UPDLOCK)
SET @NextID = NextID = NextID + @IncrementValue
WHERE KeyID = @KeyID

-- Get the error value and rowcount
SELECT @MyError = @@ERROR, @MyRowCount = @@ROWCOUNT

--===== Check for errors, a rowcount of 1, and a non-default value for @NextID
IF @MyError <> 0 --An error did occur
OR @MyRowCount <> 1 --The row was not updated
OR @NextID = -1 --A new value for @NextID was not returned
BEGIN --Start of error processing
--===== Process errors (RaisError cannot do calcs for error message)
IF @MyError <> 0 --Error occured
SET @ErrMessage = 'The NextID row could not be updated.'
ELSE --1 row or @NextID was not not updated
SET @ErrMessage = 'The NextID row could not be updated. KeyID '
+ CONVERT(VARCHAR(11),@KeyID)
+ ' may not exist.'
RAISERROR (@ErrMessage,1,1)
RETURN -1 --Returns an error indication to calling procedure
END --End of error processing

--===== Calculate and return the first number in the block of reserved NextID's
-- to the @NextID output parameter
SELECT @NextID = @NextID - @IncrementValue

--===== Return a "success" indication to the calling procedure
RETURN 0
GO




To reiterate, using sequence tables in SQL Server just isn't the right thing to do and it took me a lot to say it that nicely ;-).

--Jeff Moden

RBAR is pronounced ree-bar and is a Modenism for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column.
If you think its expensive to hire a professional to do the job, wait until you hire an amateur. -- Red Adair

Helpful Links:
How to post code problems
How to post performance problems
Forum FAQs
SQLRNNR
SQLRNNR
SSC-Dedicated
SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)SSC-Dedicated (32K reputation)

Group: General Forum Members
Points: 32204 Visits: 18552
Would a sequential guid work in place of an int?

It should be less of problematic and it doesn't necessarily need to be the primary key (it could be a unique key and used as a foreign key as well).


http://sqlblogcasts.com/blogs/martinbell/archive/2009/05/25/GUID-Ordering-in-SQL-Server.aspx


http://developmenttips.blogspot.com/2008/03/generate-sequential-guids-for-sql.html




Jason AKA CirqueDeSQLeil
I have given a name to my pain...
MCM SQL Server, MVP


SQL RNNR

Posting Performance Based Questions - Gail Shaw

Tergum Rufus
Tergum Rufus
SSC Journeyman
SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)SSC Journeyman (75 reputation)

Group: General Forum Members
Points: 75 Visits: 108
Thanks Jeff for the informative reply.
Independently of this blog, we very quickly canned any further thought on ISOLATION LEVEL et aliter
All the tests we did with this were catastrophic!

We will work on your suggestion and blog the outcome.
Rgds
Peter
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search