SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


udf very slow?


udf very slow?

Author
Message
Eric Mamet
Eric  Mamet
SSCommitted
SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)

Group: General Forum Members
Points: 1587 Visits: 893
I created a User Defined Function that performs some arithmetic calculations on a few columns and returns an amount.
The UDF does not do any lookup on the database, just uses the parameters passed (and calls another UDF with similar characteristics).

I organised my underlying table so that all required columns are available in that table.

I expected the UDF performance to be extremely quick but it happens to be VERY slow.
Retrieving ~300,000 rows with the UDF takes about 10mns.
Just removing the function makes it about 15 seconds (scanning the table and displaying in SSMS).

There is no index at all on my table and I don't specify any criteria so both Tests involve just a table scan.

I was under the impression that I would have no performance problem as long as there was no database access within the UDF but this seems wrong.
I suppose I could get the required performance using a CLR function but I would rather avoid that because it is way beyond the technical skills of my customer (I am a consultant).

My bottleneck is entirely CPU

Any idea how to improve this?
Koen Verbeeck
Koen Verbeeck
One Orange Chip
One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)

Group: General Forum Members
Points: 27185 Visits: 13268
If you have 300,000 rows, the UDF will be called 300,000 times if you defined a scalar UDF.

More info:
User Defined Functions and Performance


Using a table valued function could improve performance.


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Eric Mamet
Eric  Mamet
SSCommitted
SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)

Group: General Forum Members
Points: 1587 Visits: 893
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!! :-D
Koen Verbeeck
Koen Verbeeck
One Orange Chip
One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)One Orange Chip (27K reputation)

Group: General Forum Members
Points: 27185 Visits: 13268
Eric Mamet (10/31/2013)
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!! :-D


Never seen someone so excited about CLR :-)


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Sean Pearce
Sean Pearce
SSCommitted
SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)SSCommitted (1.8K reputation)

Group: General Forum Members
Points: 1764 Visits: 3432
It is much faster to apply a table function.

CREATE TABLE Test1 (ID INT);
GO

INSERT INTO Test1
SELECT TOP 1000000 ROW_NUMBER() OVER (ORDER BY a.object_id)
FROM sys.all_columns a
CROSS JOIN sys.all_columns b
GO

CREATE FUNCTION UDF_INLINE (@Input INT)
RETURNS INT
AS
BEGIN
DECLARE @I INT;
SET @I = @Input * 0.14;
RETURN @I;
END;
GO

CREATE FUNCTION UDF_APPLY (@Input INT)
RETURNS TABLE
AS
RETURN (SELECT @Input * 0.14 AS Result);
GO

SET STATISTICS IO ON;
SET STATISTICS TIME ON;

-- Return the column with no function
SELECT
ID
INTO
#test1
FROM
Test1;

/*
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 437 ms, elapsed time = 533 ms.
*/

-- Return the inline function
SELECT
dbo.UDF_INLINE(ID) AS RowName
INTO
#test2
FROM
Test1;

/*
Table '#test2'. Scan count 0, logical reads 1001607
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 10389 ms, elapsed time = 13965 ms.
*/

-- Apply the function
SELECT
b.Result
INTO
#test3
FROM
Test1 t
CROSS APPLY
dbo.UDF_APPLY(t.ID) AS b;

/*
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 577 ms, elapsed time = 578 ms.
*/





The SQL Guy @ blogspot

@SeanPearceSQL

About Me
Alan.B
Alan.B
SSCertifiable
SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)SSCertifiable (5.1K reputation)

Group: General Forum Members
Points: 5107 Visits: 7702
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!!


I don't know if a CLR is going to be the way to go. I have been a big proponent of CLRs for some things because they do have their place but, in this case, there is no reason to believe that a CLR is going to perform aggregations faster than a T-SQL function. Remember, creating and Implementing a CLR is not a trivial task and introduces new risks and overhead for your SQL environment. You may just be adding more work without any added benefits.

Take a look at this article How to Make Scalar UDFs Run Faster (SQL Spackle). As Sean was saying and demonstrated: you should get much better results turning your function into an inline table valued function. You will have to change your query logic to include a cross apply but that is something numerous people on this site can help you with if you get stuck.

P.S. I believe there is a new SQL Server Central Stairway on CLRs coming soon. I expect that it will be a good read. ;-)

-- Alan Burstein



Best practices for getting help on SQLServerCentral
Need to split a string? Try DelimitedSplit8K or DelimitedSplit8K_LEAD (SQL 2012+)
Need a pattern-based splitter? Try PatternSplitCM
Need to remove or replace those unwanted characters? Try PatExclude8K and PatReplace8K.

"I can't stress enough the importance of switching from a 'sequential files' mindset to 'set-based' thinking. After you make the switch, you can spend your time tuning and optimizing your queries instead of maintaining lengthy, poor-performing code. " -- Itzek Ben-Gan 2001
Eric Mamet
Eric  Mamet
SSCommitted
SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)

Group: General Forum Members
Points: 1587 Visits: 893
Actually, I am not too sure to understand what the table function is doing in all this...

As for the speed, it's shockingly different!

To retrieve about 1.5 million rows using a simple TSQL UDF took about 4 minutes.

Replacing the TSQL UDF by a CLR function shrinks the 4 minutes to 18"

Removing any function still gives me about the same (17").

In other words, the CLR function is practically invisible in terms of performance! w00t

In fact, I should have remembered because this is an experiment that Itzik Ben Gan had already demonstrated in his wonderful book "Inside SQL Server 2005 TSQL Querying"
Silly me...
Luis Cazares
Luis Cazares
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16349 Visits: 19076
CLR might perform very well, but it seems to me that you're cracking nuts with a sledgehammer (or as said in spanish, killing flies with cannonballs).
Your bottleneck was CPU using UDFs because it will limit your CPU use to one (in other words, you're not using parallelism).
Good T-SQL should be enough for your problem, but it's all up to you.


Luis C.
General Disclaimer:
Are you seriously taking the advice and code from someone from the internet without testing it? Do you at least understand it? Or can it easily kill your server?


How to post data/code on a forum to get the best help: Option 1 / Option 2
Xedni
Xedni
Say Hey Kid
Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)Say Hey Kid (667 reputation)

Group: General Forum Members
Points: 667 Visits: 716
What's the nature of the function you're trying to apply to the data? As Koen says, using functions against large sets of rows can have serious performance implications. If you can do it all inline, you may consider a computed column on the table, which SQL might be able to optimize better than calling a function for each row.

Executive Junior Cowboy Developer, Esq.
Eric Mamet
Eric  Mamet
SSCommitted
SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)SSCommitted (1.6K reputation)

Group: General Forum Members
Points: 1587 Visits: 893
The machine I used was a dual processor and yes CPU was limited to 1 processor.

So I had one processor flat out for 4 mns.

I doubt that using 2 processors instead of one would improve the performance to a couple of seconds...

I don't have access to the actual procedure now but I'll try to post it tomorrow.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search