HexToSmallInt

,

Hans asked if it could be faster. This is about 10% faster; not much. His is admittedly more readable, and mine will act very strangely with invalid hex digits.

How does it work? I'm converting the string '1234' to the value 0x31323334 (for example), then subtracting '0000' so that it is 0-based in each byte (CONVERT(INT,0x30303030) = 808464432), then masking out each byte, subtracting 7 if the value is between 16 and 31 instead of 0-9 (since the hexadecimal digits will always have that bit set), then shifting to the appropriate place with division, and adding the nybbles together.

Confused? Me too. But it works. (So use Hans's; it's easier to understand. Not sure a 10% to 20% speed boost is worth this.)

-- By Aaron West, 5/4/2005
-- This version allows negative numbers
/*
	Input has to be a Valid Hexadecimal number. Example 4F or 04F or 004F. 
	7FFF is the maximum value and it gives 32767 (Max SMALLINT)
	By Aaron West. (Challenged by Hans Lindgren.)
*/
ALTER FUNCTION dbo.HexToSMALLINT
(
	@Value VARCHAR(4)
)
RETURNS SMALLINT
AS
BEGIN
DECLARE @I INT
SET @I = CAST(CAST(RIGHT( UPPER( '0000' + @Value ) , 4 )
	           AS BINARY(4)) AS INT) - 808464432 
SET @I = @I-(@I&269488144)*7/16
RETURN CAST(CAST(
  (@I&15)
+((@I/16)&240)
+((@I/256)&3840)
+((@I/4096)&61440)
AS BINARY(2))AS SMALLINT)
END
GO

Rate

Share

Share

Rate