just an update

after some testing we have adopted a different approach - it wasn't service broker that was the problem (as mentioned in the previous 2 posts)

the 64 bit servers we have have to use the ACCESS drivers for reading CSV files through openquery - (MS in their infinite wisdom do not ship the standard text drivers with windows 64 bit) - we found that these drivers simply do not scale for large files. (we have explored bulk insert, but found it did not meet our requirements)

the time to import a file seems to rise exponentially based soley on the number of records in the file

for example a 50Mb file with 10k rows reads far faster(4 seconds) than 10mb file with 50k rows (69 seconds)

as such we are delegating that work to application servers and they are passing us the data as Table value parameters (TVP) via direct connection. all that remains for our procs is to insert the data. - actually this works better for us as we can then filter the incoming data to just the info required. - just the locking to deal with now!!!!!

service broker will be involved at a later stage, but not in such a main focus.

thanks guys for contributing, although the initial solution we proposed was not scalable at least we ruled it out and are trying something else