SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
Search:  
 
 
        
Home       Members    Calendar    Who's On


12»»

-dynamic- function to convert several rows of a table to a csv list Expand / Collapse
Author
Message
Posted Tuesday, July 08, 2008 2:52 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, June 19, 2009 10:45 AM
Points: 8, Visits: 118
Hi all,

I'd like to convert several rows of my table to csv list

create table my_contracts(
customer_id int,
contract_id int
)


insert into my_contracts values (1,100)
insert into my_contracts values (1,101)
insert into my_contracts values (2,102)

select * from my_contracts


---------------------------
customer_id contract_id
---------------------------
1 100
1 101
2 102



I found some useful postings about this (recursive or via XML). Here astatic solution:

create FUNCTION my_list_static
(
-- Add the parameters for the function here
)
RETURNS varchar(1000)
AS
BEGIN

declare @list varchar(1000)
select @list= isnull(@list + ', ','') + convert(varchar(1000),contract_id) from contracts
where customer_id= 1 order by contract_id

RETURN @list

/*
select dbo.my_list_static() as list_of_contracts
*/

END
GO


my_list_static() works as it should, I find

list_of_contracts
-----------------
100, 101


Now I want to create a dynamic function, which can be used in general


alter FUNCTION my_list_dynamic
(
@table varchar(100),
@field_to_list varchar(100),
@criteria varchar(100)
)
RETURNS varchar(1000)
AS
BEGIN

declare @list varchar(1000)
declare @sql varchar(1000)

set @sql='select @list= isnull(@list + '', '','''') + convert(varchar(1000),' + @field_to_list + ')'
set @sql= @sql + ' from ' + @table
set @sql= @sql + ' where ' + @criteria + ' order by ' + @field_to_list


-- this is what I find in @sql:
--select @list= isnull(@list + ', ','') + convert(varchar(1000),contract_id) from my_contracts where customer_id= 1 order by contract_id

-- here I try to fetch the value into the variable by exec but it fails!
exec @sql

RETURN @list

END
GO

/*
select dbo.my_list_dynamic('my_contracts', 'contract_id','customer_id= 1') as list_of_contracts
*/

Any idea how I can resolve this problem?
Thanks you a lot for each answer...


Best Regards
Martin Berkl
Post #529828
Posted Wednesday, July 09, 2008 7:46 AM
SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Today @ 6:53 AM
Points: 866, Visits: 1,208
Take note of the added parentheses around @sql in your EXEC statement...

Steve
(aka smunson)
:):):)

smerg (7/8/2008)
Hi all,

I'd like to convert several rows of my table to csv list

create table my_contracts(
customer_id int,
contract_id int
)


insert into my_contracts values (1,100)
insert into my_contracts values (1,101)
insert into my_contracts values (2,102)

select * from my_contracts


---------------------------
customer_id contract_id
---------------------------
1 100
1 101
2 102



I found some useful postings about this (recursive or via XML). Here astatic solution:

create FUNCTION my_list_static
(
-- Add the parameters for the function here
)
RETURNS varchar(1000)
AS
BEGIN

declare @list varchar(1000)
select @list= isnull(@list + ', ','') + convert(varchar(1000),contract_id) from contracts
where customer_id= 1 order by contract_id

RETURN @list

/*
select dbo.my_list_static() as list_of_contracts
*/

END
GO


my_list_static() works as it should, I find

list_of_contracts
-----------------
100, 101


Now I want to create a dynamic function, which can be used in general


alter FUNCTION my_list_dynamic
(
@table varchar(100),
@field_to_list varchar(100),
@criteria varchar(100)
)
RETURNS varchar(1000)
AS
BEGIN

declare @list varchar(1000)
declare @sql varchar(1000)

set @sql='select @list= isnull(@list + '', '','''') + convert(varchar(1000),' + @field_to_list + ')'
set @sql= @sql + ' from ' + @table
set @sql= @sql + ' where ' + @criteria + ' order by ' + @field_to_list


-- this is what I find in @sql:
--select @list= isnull(@list + ', ','') + convert(varchar(1000),contract_id) from my_contracts where customer_id= 1 order by contract_id

-- here I try to fetch the value into the variable by exec but it fails!
exec (@sql)

RETURN @list

END
GO

/*
select dbo.my_list_dynamic('my_contracts', 'contract_id','customer_id= 1') as list_of_contracts
*/

Any idea how I can resolve this problem?
Thanks you a lot for each answer...


Best Regards
Martin Berkl
Post #530809
Posted Wednesday, July 09, 2008 7:47 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Tuesday, June 30, 2009 4:37 AM
Points: 319, Visits: 652
Hi,
As you've no doubt guessed - You can't execute a stored procedure from a function:

Msg 557, Level 16, State 2, Line 1
Only functions and extended stored procedures can be executed from within a function.

If I had to fix your problem, I'd probably consider going down the CLR route - Use managed code data-access to build up your CSV, then return that from the assembly? Any CLR experts want to offer help?

Good luck - Leo.
Post #530813
Posted Wednesday, July 09, 2008 8:28 PM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 12:34 PM
Points: 16,194, Visits: 8,835
Leo Mepham (7/9/2008)
Any CLR experts want to offer help?


No... but you don't need any CLR experts to do this either... Leo... I'm not picking on you but
I have a bone to pick with anyone that will listen...

SEE????!!!?!?! Everybody keeps saying "Oh, cursors have their place" or "Oh,
CLR's have their place"
... "You should only use them when no other way is possible..."

The problem is, PEOPLE WHO DON'T REALLY KNOW SQL ARE GOING TO USE THEM ALL THE
BLOODY TIME!!!! And they never learn T-SQL in the process...

If you use a cursor or a While loop or a CLR to get something done, there's a 99.9% chance
that it's because you don't know what you're doing in T-SQL (yet). Sure, there are some
routines that traverse databases or tables that might need a cursor or while loop (actually,
thanks to VARCHAR(MAX), NOT ANY MORE!)... you should probably use a CLR for things like
"RegEx replace" and maybe some very high level math... but the other 99.9% of the time,
you're just wrong to use them.

Take the simple problem posed on this thread...

Here's some test data...

--===== Create and populate the test table
CREATE TABLE MyContracts
(
CustomerID INT,
ContractID INT
)

INSERT INTO MyContracts
(CustomerID,ContractID)
SELECT 1,100 UNION ALL
SELECT 1,101 UNION ALL
SELECT 1,102 UNION ALL
SELECT 1,103 UNION ALL
SELECT 1,104 UNION ALL
SELECT 1,105 UNION ALL
SELECT 1,106 UNION ALL
SELECT 1,107 UNION ALL
SELECT 1,108 UNION ALL
SELECT 2,202 UNION ALL
SELECT 3,301 UNION ALL
SELECT 3,302

--===== Verify the contents of the test table
SELECT * FROM MyContracts


... and here's the solution...

--===== Do the concatenation without a CLR!
SELECT t1.CustomerID ,
STUFF((SELECT ',' + CAST(t2.ContractID AS VARCHAR(10))
FROM MyContracts t2
WHERE t1.CustomerID = t2.CustomerID FOR XML PATH(''))
,1,1,'')
FROM MyContracts t1
GROUP BY t1.CustomerID


Now, somebody tell me why we need a CLR for that? That answer is, because the original
folks doing the programming didn't have a clue how to do it in T-SQL and instead of trying
to find out how, the went to the non-database programmers Nirvana known as a CLR.
And OH so many think a CLR will run quicker... consider this... they don't know how to
write some T-SQL to begin with so they write some crap T-SQL and then compare that
against a CLR which may run good even if it too is crap... of course the CLR is going to win!
You fixed the bloody race! ;)

Heh... can you tell someone is standing on my last nerve about this very subject today? :D
Don't even get me started on people who write code so wide that you always have to
^(%*$&$! scroll to the right to read it


--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/
Post #531329
Posted Wednesday, July 09, 2008 8:51 PM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 12:34 PM
Points: 16,194, Visits: 8,835
Martin,

I also have to say that this is usually a very bad thing to do in SQL Server... most
will tell you that if you need to do it for the GUI, then do it in the GUI... that is, of
course, if you have a GUI. :P

Other than that, I hope my solution helps you... and don't let people talk you into
CLR's just because they can't figure it out in SQL without some good amount of
performance. ;) And, NO, putting something in a CLR does NOT automatically mean
that it's going to be faster. In fact, Matt Miller and I did some pretty severe testing
(I just gotta find that URL again) and we beat the pants off of every CLR problem
thrown at us with the exception of some very complicated RegEx Replace
functions... even then, we still came close!


--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/
Post #531332
Posted Thursday, July 10, 2008 1:47 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Tuesday, June 30, 2009 4:37 AM
Points: 319, Visits: 652
Jeff, I work with TSQL all day long, and I do read & enjoy your pieces on here - But I don't see that you've solved his problem? How would you create the dynamic function in your example? You've done a good example for static CSV creation, but I don't see how that could be converted into a dynamic-column-selecting CSV generator.

I went down the CLR route because the original poster has painted himself into a corner, and now needs something that should be impossible in standard SQL - He needs to execute dynamic SQL for each row of a resultset. We don't know WHY he needs to do this, so perhaps I shoul have established that in order to find a more compliant solution.

At my workplace, I don't use the CLR, I tried it for string splitting - We have ended up with lots of sprocs taking CSV lists of params, so I needed a fast splitter. Turned out that although the CLR was slightly faster, because it was opaque the query optimiser ended up mis-guessing the 'estimated rows' so badly that it was faster to go back to the tally table! Sommarskog's pages were invaluable there :D

If you have something that fulfills the OP's requirements, some TSQL that fits into a function that can be called with params, that will output CSVs based on column names passed in - I'd be interested in seeing that.

If the CLR is not acceptable, is the possible sample set small enough that you could make every combination function? So, you'd have fnCSVfromContractID, fnCSVfromCustomerID, fnCSVfromContractIDCustomerID. These could then be selected from a master function fnCSV based on which column names you pass in? Perhaps a little excessive, but it could solve the problem without recourse to the CLR?

Best wishes - Leo.
Post #531418
Posted Thursday, July 10, 2008 6:50 PM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 12:34 PM
Points: 16,194, Visits: 8,835
Leo Mepham (7/10/2008)
Jeff, I work with TSQL all day long, and I do read & enjoy your pieces on here - But I don't see that you've solved his problem? How would you create the dynamic function in your example? You've done a good example for static CSV creation, but I don't see how that could be converted into a dynamic-column-selecting CSV generator.


OP didn't ask for a dynamic-column-selecting CSV generator... OP asked for something that would create CSV's grouped by EmployeeID... Run my code example and read the OP... it does exactly what was asked.


--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/
Post #532071
Posted Friday, July 11, 2008 1:38 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Tuesday, June 30, 2009 4:37 AM
Points: 319, Visits: 652
Hi all, I'd like to convert several rows of my table to csv list


my_list_static() works as it should


Now I want to create a dynamic function, which can be used in general


alter FUNCTION my_list_dynamic
(
@table varchar(100),
@field_to_list varchar(100),
@criteria varchar(100)
)


Note also that he wants to pass in as a parameter, the column to create the CSV from. That's what was asked. OP - Are you still out there and could you clarify? Do you need to pass the column in as a parameter or was that just an example? In which case has Jeff solved this for you or do need further help?
Post #532193
Posted Friday, July 11, 2008 2:34 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, June 19, 2009 10:45 AM
Points: 8, Visits: 118
Hi,

I'm still here (very busy at the moment, so I didn't reply so quickly...). I need to pass column and tablename as parameters, so my problem is not yet solved.
Of course I could use a static solution, but this was not the first time I had similar problems. So I would really appreciate a solution.

Thanks
smerg

Post #532216
Posted Friday, July 11, 2008 3:12 AM
Old Hand

Old HandOld HandOld HandOld HandOld HandOld HandOld HandOld Hand

Group: General Forum Members
Last Login: Tuesday, June 30, 2009 4:37 AM
Points: 319, Visits: 652
Hi Smerg,
If that is the requirement then I really can't see any other way from the CLR. You just can't execute dynamic SQL in a function, so using the CLR as a 'cheat' is all I can think of.

If you haven't used the CLR its not so bad, hopefully you have some .Net experience?

Here is a good place to start:

http://msdn.microsoft.com/en-us/library/ms345135.aspx

You create a wrapper function in TSQL that accepts your table/column name plus other params as required, then passes these on to the method/class in the assembly you've created. It isn't as hard as it sounds, MSDN simple example hacked up a little to match more what you need:

using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;

public class CLRFunctions
{
[SqlFunction]
public static SqlString fnCSV(SqlString columnName, SqlString tableName)
{
// as usual, connection strings shouldn't be hardcoded for production code
using(SqlConnection conn = new SqlConnection(
"server=MyServer; database=AdventureWorks; " +
"user id=MyUser; password=MyPassword"))
{
conn.Open();

//Build up your CSV-making query here
SqlCommand cmd = new SqlCommand(
"SELECT " + columnName + " FROM " + tableName, conn);

SqlDataReader r = cmd.ExecuteReader();

string CSVResults = "";

while(r.Read())
{
CSVResults += r["" + columnName + ""].ToString() + ",";
}
}
return CSVResults;
}
}

So, in the above, you'd pass in the parameters, construct the query as if you were using dynamic SQL, then use the r.Read() loop to concatenate your CSV, which the assembly then returns to the SQL function, which returns it to the row in question.

By the way - The above is not tested or compiled, it's just to give you an example of the level of difficulty in dealing with the CLR - How is your .Net coding? I'm sure you'll be fine constructing the dynamic SQL though inside the function though, just add extra parameters to the above class and build the query as required.

To get the CLR up & running, get your class written then compile into a DLL using either Visual Studio, or CSC.exe (http://msdn.microsoft.com/en-us/library/78f4aasd(VS.80).aspx). Load the DLL into SQL Server and create the TSQL 'wrapper' for the assembly method (http://www.sqldbatips.com/showarticle.asp?ID=22).

Undoubtedly, this is not an ideal solution - Is there any other way you could approach this? Consider the usual methods in SQL Server - Constructing a function that can do anything means there is no way for SQL Server to optimise it - So performance will always be terrible.

Do take a look at http://www.sommarskog.se/dynamic_sql.html if you can. His site is full of good TSQL theory & tests, perhaps there is a way to solve your problem without recourse to dynamic SQL or the CLR, but without knowing your situation/requirements/tables etc there's not much else I can suggest.

Let me know if any of this helps, or if there's anything else you'd like to add?

Best of luck, now I'd better get out of here before Jeff comes to shout at me again
Post #532231
« Prev Topic | Next Topic »

12»»

Permissions Expand / Collapse