Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Put data directly into DataTable or dot net object ?


Put data directly into DataTable or dot net object ?

Author
Message
blasto_max
blasto_max
SSC-Enthusiastic
SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)

Group: General Forum Members
Points: 193 Visits: 480
Currently, I just create an OLEDB connection, run a query on it and load the result set into a SSIS object. I don't know what this Object really is. After that, I load this Object into a DataTable in my C# code. Instead of doing all this, is there a way to directly load this result set into a data table ? Also, is there a way to get the number of rows in the result set from the object itself ? Right now, I load it into a data table and then get the number of rows as myDataTable.Rows.Count;

Thanks.
LutzM
LutzM
SSCertifiable
SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)

Group: General Forum Members
Points: 7011 Visits: 13559
What you've described so far is HOW you do your corrent task. But not, WHAT this task really is.
If you'd describe the task you're faced with, we might be able to help you finding an alternative.



Lutz
A pessimist is an optimist with experience.

How to get fast answers to your question
How to post performance related questions
Links for Tally Table , Cross Tabs and Dynamic Cross Tabs , Delimited Split Function
CozyRoc
CozyRoc
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1152 Visits: 2235
If you use OLE DB connection, the result object is ADO Recordset.

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/


blasto_max
blasto_max
SSC-Enthusiastic
SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)

Group: General Forum Members
Points: 193 Visits: 480
LutzM (12/15/2013)
What you've described so far is HOW you do your corrent task. But not, WHAT this task really is.
If you'd describe the task you're faced with, we might be able to help you finding an alternative.


Thanks. I need to iterate a result set, do some simple processing on it and then write that processed data to a file or even a database. So, I find it inefficient to first load a result set into an SSIS object and then load it into a DataTable for processing. why not load it into a DataTable in the first place ?
CozyRoc
CozyRoc
Ten Centuries
Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)Ten Centuries (1.2K reputation)

Group: General Forum Members
Points: 1152 Visits: 2235
blasto_max (12/16/2013)
LutzM (12/15/2013)
What you've described so far is HOW you do your corrent task. But not, WHAT this task really is.
If you'd describe the task you're faced with, we might be able to help you finding an alternative.


Thanks. I need to iterate a result set, do some simple processing on it and then write that processed data to a file or even a database. So, I find it inefficient to first load a result set into an SSIS object and then load it into a DataTable for processing. why not load it into a DataTable in the first place ?



Loading into another in-memory object is not going to help too much. You are still going to have enormous memory consumption if you load millions of records .

---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/


LutzM
LutzM
SSCertifiable
SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)

Group: General Forum Members
Points: 7011 Visits: 13559
Why not doing all the data manipulation inside the database?



Lutz
A pessimist is an optimist with experience.

How to get fast answers to your question
How to post performance related questions
Links for Tally Table , Cross Tabs and Dynamic Cross Tabs , Delimited Split Function
blasto_max
blasto_max
SSC-Enthusiastic
SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)

Group: General Forum Members
Points: 193 Visits: 480
LutzM (12/16/2013)
Why not doing all the data manipulation inside the database?


What do you mean by that ?
LutzM
LutzM
SSCertifiable
SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)SSCertifiable (7K reputation)

Group: General Forum Members
Points: 7011 Visits: 13559
Based on your description
I need to iterate a result set, do some simple processing on it and then write that processed data to a file or even a database.

I'm wondering if it wouldn't be easier to load the data directly into the database and call a stored procedure to do "some simple processing".
It depends on what that "simple processing" actually is...



Lutz
A pessimist is an optimist with experience.

How to get fast answers to your question
How to post performance related questions
Links for Tally Table , Cross Tabs and Dynamic Cross Tabs , Delimited Split Function
blasto_max
blasto_max
SSC-Enthusiastic
SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)SSC-Enthusiastic (193 reputation)

Group: General Forum Members
Points: 193 Visits: 480
LutzM (12/16/2013)
Based on your description
I need to iterate a result set, do some simple processing on it and then write that processed data to a file or even a database.

I'm wondering if it wouldn't be easier to load the data directly into the database and call a stored procedure to do "some simple processing".
It depends on what that "simple processing" actually is...


Actually, I have to extract data, change it in C# script and then prepare batch INSERT statements in the C# code. After that, I have to execute these insert statements on a target database.

For now, I am assuming that I have to use only SSIS and not SQL server side code to do the job.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search