Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Function in where clause Expand / Collapse
Author
Message
Posted Thursday, March 21, 2013 4:47 AM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Today @ 3:04 AM
Points: 591, Visits: 1,585
Hi all

Generally speaking I know that using functions in a where clause is bad practice as it can confuse the query optimiser however I have a situation here which I am wondering about.

I have a stored procedure used in a SSRS report that uses multi-valued parameters, as such the function I am talking about splits the string and returns a table dataset, and is then queried using the IN clause.

My question is, in this case, would that be the best way to code this or would splitting the string first and placing it in a temp table before being queried be better coding practice?

Thanks for any tips in advance
Post #1433709
Posted Thursday, March 21, 2013 4:57 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Today @ 3:47 AM
Points: 2,837, Visits: 3,953
Kwisatz78 (3/21/2013)
My question is, in this case, would that be the best way to code this or would splitting the string first and placing it in a temp table before being queried be better coding practice?

See this is the general/frequently used approach while dealing with this kind of split string query.

You query performace will be dependent on how much string(data) will get splitted and join back to another table.
and above of this , this function will get called for every record.


-------Bhuvnesh----------
I work only to learn Sql Server...though my company pays me for getting their stuff done
Post #1433712
Posted Thursday, March 21, 2013 5:11 AM
SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Friday, July 4, 2014 3:55 AM
Points: 2,836, Visits: 5,062
Bhuvnesh (3/21/2013)
Kwisatz78 (3/21/2013)
My question is, in this case, would that be the best way to code this or would splitting the string first and placing it in a temp table before being queried be better coding practice?

See this is the general/frequently used approach while dealing with this kind of split string query.

You query performace will be dependent on how much string(data) will get splitted and join back to another table.
and above of this , this function will get called for every record.



It will be per row if it's scalar UDF and called like that:

SELECT dbo.YourFunction(Col)
FROM...

But it will be called once if your UDF returns a table and used in WHERE clause like that:

SELECT ...
FROM ...
WHERE Col IN (SELECT Item FROM dbo.YourSplitFunction(@InputParam))

Don't forget to use J.Moden splitter (http://www.sqlservercentral.com/articles/Tally+Table/72993/)


_____________________________________________
"The only true wisdom is in knowing you know nothing"
"O skol'ko nam otkrytiy chudnyh prevnosit microsofta duh!"
(So many miracle inventions provided by MS to us...)

How to post your question to get the best and quick help
Post #1433721
Posted Thursday, March 21, 2013 5:15 AM
SSChasing Mays

SSChasing MaysSSChasing MaysSSChasing MaysSSChasing MaysSSChasing MaysSSChasing MaysSSChasing MaysSSChasing Mays

Group: General Forum Members
Last Login: Today @ 5:19 AM
Points: 646, Visits: 1,854
I've no supporting evidence to hand...but I'd be inclined to insert into a temp table (with appropriate KEY) and then join on the temp table (rather than use IN). Less opportunities for the optimiser to do something unfortunate.
Post #1433722
Posted Thursday, March 21, 2013 5:30 AM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Today @ 3:04 AM
Points: 591, Visits: 1,585


But it will be called once if your UDF returns a table and used in WHERE clause like that:

SELECT ...
FROM ...
WHERE Col IN (SELECT Item FROM dbo.YourSplitFunction(@InputParam))

Don't forget to use J.Moden splitter (http://www.sqlservercentral.com/articles/Tally+Table/72993/)


Hi - yes the call is as per this example so should only be called once, however I can see in the proc the same table is called again later on, so putting in a temp table may be the way to go.
Post #1433733
Posted Thursday, March 21, 2013 6:56 AM
SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Friday, July 4, 2014 3:55 AM
Points: 2,836, Visits: 5,062
... however I can see in the proc the same table is called again later on, so putting in a temp table may be the way to go.


Then it's logical to put the result into temp table.


_____________________________________________
"The only true wisdom is in knowing you know nothing"
"O skol'ko nam otkrytiy chudnyh prevnosit microsofta duh!"
(So many miracle inventions provided by MS to us...)

How to post your question to get the best and quick help
Post #1433762
Posted Thursday, March 21, 2013 8:09 AM


SSC-Insane

SSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-Insane

Group: General Forum Members
Last Login: Yesterday @ 9:50 PM
Points: 23,045, Visits: 31,570
I think this is a good example of why you (the OP) should provide full disclosure when asking a question. Eugene's first answer is a good one based limited information. Had we been able to see the actual stored procedure a better answer could have been given up front.

Who knows, we may have even seen a way to improve it overall.




Lynn Pettis

For better assistance in answering your questions, click here
For tips to get better help with Performance Problems, click here
For Running Totals and its variations, click here or when working with partitioned tables
For more about Tally Tables, click here
For more about Cross Tabs and Pivots, click here and here
Managing Transaction Logs

SQL Musings from the Desert Fountain Valley SQL (My Mirror Blog)
Post #1433804
Posted Thursday, March 21, 2013 9:38 AM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Today @ 3:04 AM
Points: 591, Visits: 1,585
Yes you're right Lynn so I have posted it below, thanks - I have already stripped out the functions for temp tables. I would appreciate you taking a peek.

SET STATISTICS IO ON
SET STATISTICS TIME ON

SET DATEFORMAT dmy;
GO

DECLARE
@paramCompany varchar(3),
@paramSalesCategory varchar(max),
@paramSalesStatus varchar(100),
@paramSite varchar(max),
@paramVehicleType varchar(max),
@paramActShipDateFrom datetime,
@paramActShipDateTo datetime

SET @paramCompany = 'abb'
SET @paramSalesCategory = 'All' --,001,002,003,004,005,006,100,200,250,300,350,Classical,Ebay,Ebay SM,Landscape,OrdrSelect,Paver,Stonemarkt,StoneShip'
SET @paramSalesStatus = '99' --,Open order,Delivered'
SET @paramSite = 'All' --,ABR,ALC,BBD,BFT,BLD,CAN,CAR,CHE,EAG,FAL,FEB,FLX,GEE,GGR,GMS,HAM,HDV,HEA,HIP,HLI,HLL,HPO,LLY,LSH,MAN,MLT,MRL,MSQ,MSS,MSU,NPT,NSH,NWL,PEN,PON,PTR,RBT,REG,RYT,SAN,SAW,SHN,SIT,SND,STN,STV,THL,WAF,WLA,ZZZ'
SET @paramVehicleType = 'All' --@VehicleType
SET @paramActShipDateFrom = '01/01/2013' --@ActShipDateFrom
SET @paramActShipDateTo = '21/03/2013' --@ActShipDateTo

--Create SalesCategory temp table variable
DECLARE @SalesCategoryTable TABLE
(
SalesCategory VARCHAR(100)
)

INSERT INTO @SalesCategoryTable (SalesCategory)
SELECT item FROM dbo.udf_MultiValueParameterHandlingString(@paramSalesCategory)

--Create SalesStatus temp table variable
DECLARE @SalesStatusTable TABLE
(
SalesStatus VARCHAR(100)
)

INSERT INTO @SalesStatusTable (SalesStatus)
SELECT item FROM dbo.udf_MultiValueParameterHandlingString(@paramSalesStatus)

--Create Site temp table variable
DECLARE @SiteTable TABLE
(
[SITE] VARCHAR(100)
)

INSERT INTO @SiteTable ([SITE])
SELECT item FROM dbo.udf_MultiValueParameterHandlingString(@paramSite)

--Create VehicleType temp table variable
DECLARE @VehicleTypeTable TABLE
(
VehicleType VARCHAR(100)
)

INSERT INTO @VehicleTypeTable (VehicleType)
SELECT item FROM dbo.udf_MultiValueParameterHandlingString(@paramVehicleType)


SELECT
st.SALESID as 'SalesOrder',
st.MARSALESCATEGORY as 'SalesCategory',
st.MARVEHICLETYPE as 'VehicleType',
st.MARSPECINST1 as 'SpecialInstructions1',
st.MARSPECINST2 as 'SpecialInstructions2',
st.SHIPPINGDATEREQUESTED as 'ExpShipDate',
sohead.ShippedDate as 'ActShipDate',
st.RECEIPTDATEREQUESTED as 'DelDate',
st.CUSTACCOUNT as 'CustomerId',
st.SALESNAME as 'CustomerName',
st.DELIVERYNAME as 'DeliveryName',
st.DELIVERYADDRESS as 'DeliveryAddress',
st.DELIVERYZIPCODE as 'PostCode',
ctt.Telephone,
CASE WHEN jz.ItemId is null THEN 'No Delivery Item' ELSE jz.ItemId END as 'ItemId',
CASE WHEN jz.ItemDescription is null THEN 'No Delivery Item' ELSE jz.ItemDescription END as 'ItemDescription',
jz.[Site],
jz.Warehouse,
ABS(isnull(jz.NoOfPallets,0)) as 'NoOfPallets',
ABS(isnull(jz.NoOfQtrPallets,0)) as 'NoOfQtrPallets',
'' as 'PalletsPicked',
'' as 'QtrPalletsPicked',
ABS(isnull(jz.Cost,0)) as 'Cost',
'' as 'CostPayable',
'' as 'Carrier',
'' as 'Notes',
CASE WHEN st.SALESTYPE = 4 THEN 'Return Order'
WHEN (isnull(sline.MinSalesQty,0) < 0 or isnull(sline.MinSalesQty,0) < 0) THEN 'Return Order'
ELSE 'SalesOrder' END as 'SalesType',
CASE WHEN st.SALESTYPE = 4 THEN 'COLLECT'
WHEN (isnull(sline.MinSalesQty,0) < 0 or isnull(sline.MinSalesQty,0) < 0) THEN 'COLLECT'
ELSE 'DELIVER' END as 'Del/Col'

FROM
DynamicsV5Realtime.dbo.SALESTABLE st
INNER JOIN
(SELECT distinct
tr.TRANSREFID as 'SalesId',
tr.DATAAREAID,
tr.DATEPHYSICAL as 'ShippedDate'
FROM
DynamicsV5Realtime.dbo.INVENTTRANS tr
WHERE
tr.DATEPHYSICAL between @paramActShipDateFrom and @paramActShipDateTo
and tr.DATAAREAID = @paramCompany) sohead
ON
sohead.SalesId = st.SALESID
and sohead.DATAAREAID = st.DATAAREAID
INNER JOIN
(SELECT
sl.SALESID as 'SalesId',
sl.DATAAREAID,
MIN(sl.SALESQTY) as 'MinSalesQty'
FROM
DynamicsV5Realtime.dbo.SALESLINE sl
WHERE
sl.DATAAREAID = @paramCompany
and (sl.SALESSTATUS IN(select SalesStatus from @SalesStatusTable) or @paramSalesStatus = '99')
and sl.SALESSTATUS IN(2,3)
GROUP BY
sl.SALESID,
sl.DATAAREAID ) sline
ON
sline.SalesId = st.SALESID
and sline.DATAAREAID = st.DATAAREAID

LEFT OUTER JOIN
(SELECT
sl.DATAAREAID,
sl.SALESID as 'SalesId',
sl.ITEMID as 'ItemId',
sl.NAME as 'ItemDescription',
id.INVENTSITEID as 'Site',
id.INVENTLOCATIONID as 'Warehouse',
CASE WHEN it.MARITEMFIXINGCODE = '001' THEN ABS(sum(tr.QTY)) ELSE 0 END as 'NoOfPallets',
CASE WHEN it.MARITEMFIXINGCODE = '002' THEN ABS(sum(tr.QTY)) ELSE 0 END as 'NoOfQtrPallets',
ABS(Sum(tr.QTY * itm.PRICE / CASE WHEN itm.PRICEUNIT = 0 THEN 1 ELSE itm.PRICEUNIT END)) as 'Cost',
tr.DATEPHYSICAL as 'ShippedDate'
FROM
DynamicsV5Realtime.dbo.SALESLINE sl
INNER JOIN
DynamicsV5Realtime.dbo.SALESTABLE st
ON
st.SALESID = sl.SALESID
and st.DATAAREAID = sl.DATAAREAID
INNER JOIN
DynamicsV5Realtime.dbo.INVENTTABLE it
ON
it.ITEMID = sl.ITEMID
and it.DATAAREAID = sl.DATAAREAID
INNER JOIN
DynamicsV5Realtime.dbo.INVENTTABLEMODULE itm
ON
itm.ITEMID = sl.ITEMID
and itm.DATAAREAID = sl.DATAAREAID
and itm.MODULETYPE = 1 -- purchase
LEFT OUTER JOIN
DynamicsV5Realtime.dbo.INVENTDIM id
ON
id.INVENTDIMID = sl.INVENTDIMID
and id.DATAAREAID = sl.DATAAREAID
LEFT OUTER JOIN
DynamicsV5Realtime.dbo.INVENTTRANS tr
ON
tr.INVENTTRANSID = sl.INVENTTRANSID
and tr.DATAAREAID = sl.DATAAREAID
WHERE
sl.DATAAREAID = @paramCompany
and sl.ITEMID like 'JZ%'
and st.SALESTYPE in(3,4)
and (st.SALESSTATUS IN(select SalesStatus from @SalesStatusTable) or @paramSalesStatus = '99')
and st.SALESSTATUS IN(2,3)
and (sl.SALESSTATUS IN(select SalesStatus from @SalesStatusTable) or @paramSalesStatus = '99')
and sl.SALESSTATUS IN(2,3)
and (st.MARSALESCATEGORY IN(select SalesCategory from @SalesCategoryTable) or @paramSalesCategory = 'All')
and (id.INVENTSITEID IN(select [SITE] from @SiteTable) or @paramSite = 'All')
and (tr.DATEPHYSICAL between @paramActShipDateFrom and @paramActShipDateTo)
GROUP BY
sl.DATAAREAID,
sl.SALESID,
sl.ITEMID,
sl.NAME,
id.INVENTSITEID,
id.INVENTLOCATIONID,
it.MARITEMFIXINGCODE,
tr.QTY,
tr.DATEPHYSICAL ) jz

ON
jz.DATAAREAID = st.DATAAREAID
and jz.SalesId = st.SALESID
and jz.ShippedDate = sohead.ShippedDate
INNER JOIN
(SELECT
ct.ACCOUNTNUM,
ct.DATAAREAID,
CASE WHEN ct.PHONE = '' THEN '' ELSE 'Tel: ' + ct.PHONE + ' ' END
+
CASE WHEN ct.CELLULARPHONE = '' THEN '' ELSE 'Mob: ' + ct.CELLULARPHONE END as 'Telephone'
FROM
DynamicsV5Realtime.dbo.CUSTTABLE ct
WHERE
ct.DATAAREAID = @paramCompany) ctt
ON
ctt.DATAAREAID = st.DATAAREAID
and ctt.ACCOUNTNUM = st.CUSTACCOUNT
WHERE
st.DATAAREAID = @paramCompany
and st.SALESTYPE IN(3,4)
and (st.SALESSTATUS IN(select SalesStatus from @SalesStatusTable) or @paramSalesStatus = '99')
and st.SALESSTATUS IN(2,3)
and (st.MARSALESCATEGORY IN(select SalesCategory from @SalesCategoryTable) or @paramSalesCategory = 'All')
and (st.INVENTSITEID IN(select [SITE] from @SiteTable) or @paramSite = 'All')
and (st.MARVEHICLETYPE IN(select VehicleType from @VehicleTypeTable) or @paramVehicleType = 'All')
ORDER BY
3,6,1,14

SET STATISTICS IO OFF
SET STATISTICS TIME OFF

Post #1433871
Posted Friday, March 22, 2013 3:23 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Today @ 3:47 AM
Points: 2,837, Visits: 3,953
Now people would be looking for exec plan here like me .

-------Bhuvnesh----------
I work only to learn Sql Server...though my company pays me for getting their stuff done
Post #1434162
Posted Friday, March 22, 2013 3:51 AM
Mr or Mrs. 500

Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500Mr or Mrs. 500

Group: General Forum Members
Last Login: Today @ 3:04 AM
Points: 591, Visits: 1,585
Hi there - I have attached it, I have already got rid of most of the RID lookups however it still takes 10mins to run I think mainly because of the amount of data its returning being 40, 000 rows, with hundreds of thousands being joined across.

I guess I need to figure out of there is a way to return less rows in the first place.

Thanks


  Post Attachments 
Queryplan.sqlplan (0 views, 905.29 KB)
Post #1434177
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse