Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 123»»»

udf very slow? Expand / Collapse
Author
Message
Posted Thursday, October 31, 2013 3:53 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Friday, August 8, 2014 5:42 AM
Points: 1,191, Visits: 790
I created a User Defined Function that performs some arithmetic calculations on a few columns and returns an amount.
The UDF does not do any lookup on the database, just uses the parameters passed (and calls another UDF with similar characteristics).

I organised my underlying table so that all required columns are available in that table.

I expected the UDF performance to be extremely quick but it happens to be VERY slow.
Retrieving ~300,000 rows with the UDF takes about 10mns.
Just removing the function makes it about 15 seconds (scanning the table and displaying in SSMS).

There is no index at all on my table and I don't specify any criteria so both Tests involve just a table scan.

I was under the impression that I would have no performance problem as long as there was no database access within the UDF but this seems wrong.
I suppose I could get the required performance using a CLR function but I would rather avoid that because it is way beyond the technical skills of my customer (I am a consultant).

My bottleneck is entirely CPU

Any idea how to improve this?
Post #1510082
Posted Thursday, October 31, 2013 4:34 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 2:51 PM
Points: 13,639, Visits: 10,534
If you have 300,000 rows, the UDF will be called 300,000 times if you defined a scalar UDF.

More info:
User Defined Functions and Performance


Using a table valued function could improve performance.




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1510096
Posted Thursday, October 31, 2013 4:37 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Friday, August 8, 2014 5:42 AM
Points: 1,191, Visits: 790
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!!
Post #1510098
Posted Thursday, October 31, 2013 4:39 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 2:51 PM
Points: 13,639, Visits: 10,534
Eric Mamet (10/31/2013)
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!!


Never seen someone so excited about CLR




How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1510099
Posted Thursday, October 31, 2013 8:31 AM


SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Today @ 5:09 AM
Points: 883, Visits: 2,807
It is much faster to apply a table function.

CREATE TABLE Test1 (ID INT);
GO

INSERT INTO Test1
SELECT TOP 1000000 ROW_NUMBER() OVER (ORDER BY a.object_id)
FROM sys.all_columns a
CROSS JOIN sys.all_columns b
GO

CREATE FUNCTION UDF_INLINE (@Input INT)
RETURNS INT
AS
BEGIN
DECLARE @I INT;
SET @I = @Input * 0.14;
RETURN @I;
END;
GO

CREATE FUNCTION UDF_APPLY (@Input INT)
RETURNS TABLE
AS
RETURN (SELECT @Input * 0.14 AS Result);
GO

SET STATISTICS IO ON;
SET STATISTICS TIME ON;

-- Return the column with no function
SELECT
ID
INTO
#test1
FROM
Test1;

/*
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 437 ms, elapsed time = 533 ms.
*/

-- Return the inline function
SELECT
dbo.UDF_INLINE(ID) AS RowName
INTO
#test2
FROM
Test1;

/*
Table '#test2'. Scan count 0, logical reads 1001607
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 10389 ms, elapsed time = 13965 ms.
*/

-- Apply the function
SELECT
b.Result
INTO
#test3
FROM
Test1 t
CROSS APPLY
dbo.UDF_APPLY(t.ID) AS b;

/*
Table 'Test1'. Scan count 1, logical reads 3345
CPU time = 577 ms, elapsed time = 578 ms.
*/





The SQL Guy @ blogspot

@SeanPearceSQL

About Me
Post #1510211
Posted Thursday, October 31, 2013 11:49 AM


Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Today @ 1:31 PM
Points: 564, Visits: 2,626
I know but I was hoping it would be fast...

Just talked to my customer and I'll try using CLR!
Yipee!!!


I don't know if a CLR is going to be the way to go. I have been a big proponent of CLRs for some things because they do have their place but, in this case, there is no reason to believe that a CLR is going to perform aggregations faster than a T-SQL function. Remember, creating and Implementing a CLR is not a trivial task and introduces new risks and overhead for your SQL environment. You may just be adding more work without any added benefits.

Take a look at this article How to Make Scalar UDFs Run Faster (SQL Spackle). As Sean was saying and demonstrated: you should get much better results turning your function into an inline table valued function. You will have to change your query logic to include a cross apply but that is something numerous people on this site can help you with if you get stuck.

P.S. I believe there is a new SQL Server Central Stairway on CLRs coming soon. I expect that it will be a good read.


-- Alan Burstein



Read this article for best practices on asking questions.
Need to split a string? Try this (Jeff Moden)
Need a pattern-based string spitter? Try this (Dwain Camps)
My blog
Post #1510363
Posted Thursday, October 31, 2013 11:58 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Friday, August 8, 2014 5:42 AM
Points: 1,191, Visits: 790
Actually, I am not too sure to understand what the table function is doing in all this...

As for the speed, it's shockingly different!

To retrieve about 1.5 million rows using a simple TSQL UDF took about 4 minutes.

Replacing the TSQL UDF by a CLR function shrinks the 4 minutes to 18"

Removing any function still gives me about the same (17").

In other words, the CLR function is practically invisible in terms of performance!

In fact, I should have remembered because this is an experiment that Itzik Ben Gan had already demonstrated in his wonderful book "Inside SQL Server 2005 TSQL Querying"
Silly me...
Post #1510368
Posted Thursday, October 31, 2013 12:16 PM


Hall of Fame

Hall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of FameHall of Fame

Group: General Forum Members
Last Login: Today @ 4:13 PM
Points: 3,563, Visits: 7,697
CLR might perform very well, but it seems to me that you're cracking nuts with a sledgehammer (or as said in spanish, killing flies with cannonballs).
Your bottleneck was CPU using UDFs because it will limit your CPU use to one (in other words, you're not using parallelism).
Good T-SQL should be enough for your problem, but it's all up to you.



Luis C.
Are you seriously taking the advice and code from someone from the internet without testing it? Do you at least understand it? Or can it easily kill your server?

Forum Etiquette: How to post data/code on a forum to get the best help
Post #1510373
Posted Thursday, October 31, 2013 1:42 PM
SSC Journeyman

SSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC Journeyman

Group: General Forum Members
Last Login: Tuesday, August 26, 2014 3:31 PM
Points: 90, Visits: 297
What's the nature of the function you're trying to apply to the data? As Koen says, using functions against large sets of rows can have serious performance implications. If you can do it all inline, you may consider a computed column on the table, which SQL might be able to optimize better than calling a function for each row.


Executive Junior Cowboy Developer, Esq.
Post #1510405
Posted Thursday, October 31, 2013 3:27 PM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Friday, August 8, 2014 5:42 AM
Points: 1,191, Visits: 790
The machine I used was a dual processor and yes CPU was limited to 1 processor.

So I had one processor flat out for 4 mns.

I doubt that using 2 processors instead of one would improve the performance to a couple of seconds...

I don't have access to the actual procedure now but I'll try to post it tomorrow.


Post #1510439
« Prev Topic | Next Topic »

Add to briefcase 123»»»

Permissions Expand / Collapse