SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Best approach for Data transfer of large number of rows


Best approach for Data transfer of large number of rows

Author
Message
Megha P
Megha P
Right there with Babe
Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)

Group: General Forum Members
Points: 744 Visits: 2501
Hello,
we have our database in sql 2005 version Enterprise Edition .
Because of performance issue we have normalized database and now required data transfer from denormalized database to normalized database.all tables has been created.

Denormalized database have 429 tables, some of them have greater than 5 millions rows.
In this condition one table data may needs to be transferred in 3 different tables of normalized database.

which would be the best approach to do data transfer from one database to another
i.e script or ssis package..?

Thanks..
Koen Verbeeck
Koen Verbeeck
SSC Guru
SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)

Group: General Forum Members
Points: 76387 Visits: 13300
If the databases are on the same server, you can use SQL scripts.
Make sure logging is minimized to reduce overhead.

What kind of performance issues were you having with the denormalized system?


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Megha P
Megha P
Right there with Babe
Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)

Group: General Forum Members
Points: 744 Visits: 2501
Actually, we have tables with more than 150 columns and it's harder to manage it (insert) .
so tables are denormalized now.

source and destination databases are on different server.

Thanks,
megha
Koen Verbeeck
Koen Verbeeck
SSC Guru
SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)

Group: General Forum Members
Points: 76387 Visits: 13300
Megha P (4/3/2014)
Actually, we have tables with more than 150 columns and it's harder to manage it (insert) .
so tables are denormalized now.

source and destination databases are on different server.

Thanks,
megha


If they databases are on different servers, I would go for SSIS.


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Megha P
Megha P
Right there with Babe
Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)

Group: General Forum Members
Points: 744 Visits: 2501
ok...thanks..SSIS will be fast over script in case of different server data transfer?
free_mascot
free_mascot
SSCrazy Eights
SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)

Group: General Forum Members
Points: 8789 Visits: 2250
In addition to Koen; you can also drop all constrains on the destination table if no. of rows are more which will expedite the process. Once SSIS is done you can creat them again.

---------------------------------------------------
"Thare are only 10 types of people in the world:
Those who understand binary, and those who don't."
Koen Verbeeck
Koen Verbeeck
SSC Guru
SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)

Group: General Forum Members
Points: 76387 Visits: 13300
Megha P (4/3/2014)
ok...thanks..SSIS will be fast over script in case of different server data transfer?


Yes, because with scripts you would have to use linked servers. SSIS will be faster.


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Megha P
Megha P
Right there with Babe
Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)Right there with Babe (744 reputation)

Group: General Forum Members
Points: 744 Visits: 2501
Hi,
Is there any other advantages of using SSIS over using script , except faster loading for data transfer?

Thanks,
Megha
Koen Verbeeck
Koen Verbeeck
SSC Guru
SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)SSC Guru (76K reputation)

Group: General Forum Members
Points: 76387 Visits: 13300
Megha P (4/4/2014)
Hi,
Is there any other advantages of using SSIS over using script , except faster loading for data transfer?

Thanks,
Megha


You don't have to use linked servers.


How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?
My blog at SQLKover.

MCSE Business Intelligence - Microsoft Data Platform MVP
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search