Source is a custom SELECT (5 tidy columns) from SQL Server. Destination is a DBF file living on the same server's hard drive.
When I first launch the flow, it appears to be flowing 10K rows every 1 second which *could* suffice for production, but immediately starts to falter. By the time it gets to 100K rows, it's taking 5-8 seconds every 10K rows, and by the time we get to 300K records, it's taking 20 seconds for 10K rows.
There are no other significant processes happening on the box. I've also included the resource monitor's view of the disk load which seems light to me. CPU and memory are both un-taxed.
I've already tried using "Fastload" on the destination objects, but I don't think that setting is defined for the VFP OLEDB driver.
Screenshot shows 3 parallel flows, but I've also tried with just 1, no WHERE, no ORDER BY, various drives... all have the same issue.
I ultimately need to get 260MM rows into umpteen DBF files (Foxpro 2G/file limitation), so this is a major road block right now.
Does anyone have experience with exporting a massive set of data to DBF's with reasonable performance? How'd you do it?!?! Any insight as to what's happening here will help me, don't hesitate to respond.
PS --> just added the perfmon screeny... looks like there are quite a few page faults happening. Does this help? Other counters you'd like to see?