How to match a UNICODE NULL character within an nVarchar(128) string?

Click here to monitor SSC

<a href="http://g.adspeed.net/ad.php?do=clk&zid=15220&wd=468&ht=60&pair=as" target="_blank"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15220&wd=468&ht=60&pair=as" alt="i" width="468" height="60"/></a>

SQLServerCentral is supported by Red Gate Software Ltd.

Log in :: Register :: Not logged in

<a href="http://g.adspeed.net/ad.php?do=clk&zid=15491&wd=120&ht=300&pair=as" target="_top"><img style="border:0px;" src="http://g.adspeed.net/ad.php?do=img&zid=15491&wd=120&ht=300&pair=as" alt="i" width="120" height="300"/></a>

Recent Posts

Popular Topics

Home

Home » SQL Server 2008 » SQL Server 2008 - General » How to match a UNICODE NULL character within...

Add to briefcase

17 posts, Page 2 of 2

««1 2

How to match a UNICODE NULL character within an nVarchar(128) string?

Expand / Collapse

Author

Message

Posted Friday, January 18, 2013 3:11 PM

SSC Rookie

Group: General Forum Members
Last Login: Monday, February 18, 2013 2:25 PM
Points: 32, Visits: 87

Magoo,

I think you're right about how there are embedded NULLS between each character. I ran the series of convert functions you provided and it showed AHSGuest††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††.

I really appreciate you providing this. I might use a modified version of this in a stored procedure as I create a programmatic method of cleaning all the columns of all our tables.

I'll update this thread with that stored procedure.

mtf

Post #1409121

Posted Friday, January 18, 2013 3:18 PM

SSC Rookie

Group: General Forum Members
Last Login: Monday, February 18, 2013 2:25 PM
Points: 32, Visits: 87

Lowell,

That was just the ticket!

The select StripNonAlphaNumeric(name) produces exactly the ASCII without the UNICODE NULLs embedded. The update also seems to work in the testing I've done so far!!!

For applying it to the entire table, I wanted to limit it to just the affected rows, so I use the earlier nested selects to narrow it down:

update my_user_table
set name=dbo.StripNonAlphaNumeric(name)
WHERE ID in
		(select DISTINCT B.id FROM 
			(select id, name, N as Position,
  				SUBSTRING(name,N,1) As TheChar,
  				ASCII(SUBSTRING(name,N,1)) TheAsciiCode
				from my_user_table
				CROSS APPLY (SELECT TOP 255 ROW_NUMBER() OVER (ORDER BY (SELECT NULL) -1) AS n
             	FROM   sys.columns) MiniTally
				WHERE  MiniTally.n BETWEEN 0 AND 255 
				AND MiniTally.n < LEN(name)+1) B
		WHERE B.TheAsciiCode=0)

We have these UNICODE NULLS infecting many of the columns in many of our tables. I'm going to write a stored procedure that utilizes this function to programmatically clean every column in every table. I'll update this thread with my solution soon.

For now, though, I wanted to let you know how much I appreciate the people who have helped me in this thread. I am HUGELY appreciative. Thank you so much.

mtf

Post #1409122

Posted Friday, January 18, 2013 3:22 PM

SSChampion

Group: General Forum Members
Last Login: Today @ 3:33 PM
Points: 11,648, Visits: 27,768

mrTexasFreedom I'm glad my code helped a bit, but mister magoo really identified the culprit, i think;

some process imported(and maybe still imports) data that should be nvarchar instead of varchar;

you should try to track down whatever that process is and fix it at the source; otherwise this cleanup-after-the-mess thing is going to be needed every time that other process runs.

Lowell

--There is no spoon, and there's no default ORDER BY in sql server either.
Actually, Common Sense is so rare, it should be considered a Superpower. --my son

Post #1409123

Posted Friday, January 18, 2013 3:44 PM

SSC Rookie

Group: General Forum Members
Last Login: Monday, February 18, 2013 2:25 PM
Points: 32, Visits: 87

Thank you for the endorsement on Magoo's convert trick. I am going to adjust it so the final character isn't repeated and if I can get that to work, I'll use it in my stored proc.

The source of those documents has been identified and we're putting processes in place to prevent more corrupt data from being imported. Our developers are also working on a front-end filter to dump anything that isn't standard UTF8. I'm the person tasked with cleaning up the mess that's already been created.

Have a great weekend!

mtf

Post #1409125

Posted Friday, January 18, 2013 4:34 PM

Ten Centuries

Group: General Forum Members
Last Login: Today @ 5:07 PM
Points: 1,308, Visits: 3,900

You should be able to just wrap the existing method in LEFT(....,LEN(name)) to get the correct answer...

but only if the original data was no more than half as long as your column can hold, otherwise you have lost some data...

MM

Post #1409133

prathibha_aviator

Posted Monday, February 18, 2013 1:01 PM

SSC Journeyman

Group: General Forum Members
Last Login: Tuesday, May 21, 2013 9:12 AM
Points: 95, Visits: 223

mrTexasFreedom (1/17/2013)

Lowell,

Sadly, this isn't matching any rows:

select id, name 
FROM my_user_table
WHERE CHARINDEX(name,CHAR(0),1) > 0

No rows returned. Likewise, the replace operation doesn't touch any of the rows. Ideas?

Thank you very much for the help you've provided thus far. I feel like I'm on the verge of clobbering this beast thanks to your assistance!

mtf

Its not returning any rows because U cannot use a search expression first followed by a find expression in the syntax of CHARINDEX

if you use CHARINDEX (CHAR(0), name, 1 ), I surely believe you will catch the bad hidden characters.. I know this thread is pretty old. But I just loved the way you all made it so quick learning session.. Thanks to Lowell for sharing this thread to me..

--Pra Smile

Smile

Smile

--------------------------------------------------------------------------------

Post #1421335

Posted Monday, February 18, 2013 1:54 PM

SSC Rookie

Group: General Forum Members
Last Login: Monday, February 18, 2013 2:25 PM
Points: 32, Visits: 87

Thanks for spotting that. I'll try that out in my test DB and see if it is a more efficient method.

Appreciatively,

mtf

Post #1421345

« Prev Topic | Next Topic »

Add to briefcase

17 posts, Page 2 of 2

««1 2

Permissions

Expand / Collapse