Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Data Extraction Methods Part 1 Expand / Collapse
Author
Message
Posted Wednesday, April 5, 2006 12:40 PM


SSC Journeyman

SSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC Journeyman

Group: General Forum Members
Last Login: Friday, September 26, 2014 12:56 AM
Points: 81, Visits: 190
Comments posted to this topic are about the content posted at http://www.sqlservercentral.com/columnists/vRainardi/dataextractionmethodspart1.asp
Post #271368
Posted Tuesday, April 25, 2006 4:51 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Thursday, May 28, 2009 2:47 PM
Points: 30, Visits: 29

I appreciate the content of this well written article. I have been doing some of this myself, but have never had it described and documented. I look forward to Part 2.

Greg Bogard

Omaha NE




Regards,

Greg Bogard
Post #275088
Posted Tuesday, April 25, 2006 6:36 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: 2 days ago @ 5:58 AM
Points: 48, Visits: 532

It is very useful article to everyone, I got lot of answers from this article, thanks in advance for next article.

 

thanking you

sreeni

Post #275121
Posted Tuesday, April 25, 2006 9:25 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Thursday, April 11, 2013 6:44 AM
Points: 1, Visits: 13

Excellent article.  I appreciate the testing you have done to verify your theories and also the explanation behind each approcach.

Keep up the good work.

Post #275229
Posted Tuesday, April 25, 2006 9:44 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, January 24, 2014 6:46 AM
Points: 153, Visits: 47
Could somebody explain the math for 58 records should take less than a second at 100 Mbps - Thanks already


Post #275246
Posted Tuesday, May 2, 2006 8:56 AM


SSC Journeyman

SSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC Journeyman

Group: General Forum Members
Last Login: Friday, September 26, 2014 12:56 AM
Points: 81, Visits: 190

Hi Skarai, it's just based on my experience, not based on math. A table which contains 1000 to 5000 records (say 10 to 100 columns) on 100M LAN normally takes 1 to 5 seconds to download from DB2 on AS/400 using iSeries OLE DB driver or from Informix on Unix using Informix native OLE DB driver into a staging table with no index in SQL Server on Windows 2000 or 2003 server, configured with RAID 5 for MDF and RAID 1 for LDF using 15k RPM disks, running on DELL 2850 with 8 MB memory, performed using DTS or SSIS with no transformation or scripting (just plain direct column to column mapping).

Kind regards,
Vincent Rainardi

Post #276829
Posted Tuesday, May 2, 2006 10:40 AM
SSC-Enthusiastic

SSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-EnthusiasticSSC-Enthusiastic

Group: General Forum Members
Last Login: Friday, January 24, 2014 6:46 AM
Points: 153, Visits: 47

Thanks for the follow up Vincent. I understand your earlier statemet now given the added explanation/observation.  - Sassan




Post #276879
Posted Thursday, May 4, 2006 3:31 AM
SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Today @ 7:02 AM
Points: 817, Visits: 2,048

I don't recommend that you get in the habit of downloading entire tables.  It may only have 10,000 records today, but a year from now, how many will it have.  We had a consultant use that method on our database, and at first the download took hours (lots of tables).  After a couple of months, it was taking longer.  I then had to rework it to only take the records modified or added that day.  The download not only took less time, but the incremental increase was no longer there.

My recommendation:  avoid the need to rewrite in the future and go with an incremental strategy.




Post #277494
Posted Wednesday, April 25, 2007 10:14 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Wednesday, January 9, 2008 12:41 PM
Points: 1, Visits: 3
Very good explanation. Waiting for the part2
Post #360905
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse