Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Datawarehouse Flat files Bulk data load Slow Loading Expand / Collapse
Author
Message
Posted Thursday, November 28, 2013 11:21 PM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, July 25, 2014 6:38 AM
Points: 5, Visits: 51
Hi Friends,

We have to load 15-20 GB of file in Data warehouse daily. The current configuration of servers are:
We were able to load 15 GB of file in 20 minutes in INT server. While we started loading data in Production Environment, it taking 1 hour to load whereas the production environment has better RAM and CPUs.

INT.
Datawarehouse Server:

16 CPU, 64 GB RAM, SQL Server 2012 Enterprise Edition.

ETL Server
4 CPU, 32 GB RAM,, SQL Server 2012 Enterprise Edition.

Production:
Datawarehouse Server:

16 CPU, 128 GB RAM, SQL Server 2012 Enterprise Edition.

ETL Server
16 CPU, 64 GB RAM,, SQL Server 2012 Enterprise Edition.

Do anyone can help me find the problem ?

Thanks in advance
Vishal;
Post #1518444
Posted Friday, November 29, 2013 2:35 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: 2 days ago @ 12:52 PM
Points: 13,636, Visits: 11,509
Maybe the production server has more contention for resources?



How to post forum questions.
Need an answer? No, you need a question.
What’s the deal with Excel & SSIS?

Member of LinkedIn. My blog at LessThanDot.

MCSA SQL Server 2012 - MCSE Business Intelligence
Post #1518479
Posted Friday, November 29, 2013 2:54 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, July 25, 2014 6:38 AM
Points: 5, Visits: 51
Hi Koen,

Thanks for the reply.

I am not a DBA. Can you help us to troubleshoot the issue ?

Thanks
Vishal;
Post #1518481
Posted Friday, November 29, 2013 3:40 AM
SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Yesterday @ 3:33 AM
Points: 856, Visits: 1,506
when you say "Load data in production server"
its means you process the file or you bring the data from the Source database.

If its source database check if the what kind of response you are getting. because there are cases in which you get slow response from the source server due to network traffic, if its a remote location then might be your broadband connection speed.

there are many cases you need to check which side is causing the bottleneck.

hope it helps.
Post #1518495
Posted Friday, November 29, 2013 4:04 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, July 25, 2014 6:38 AM
Points: 5, Visits: 51
Hi SSC,

This is the flat file which is getting loaded. This is the cluster environment. ETL and DW are in separate nodes.

Let me know if you need more information.

Vishal;
Post #1518498
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse