?How do you physically spit up the large files and track them in each thread?

?How is the master process receiving it's messages from the children processes?

FOR ABOVE TWO QUESTIONS, IN ARTICLE I HAVE GIVEN DETAILED EXPLANATION.

?In a SQL SERVER implementation would you most likely use CRL code to break up the files and Bulk Load to import it?

NOT SURE ABOUT THIS. I WOULD REQUEST EXPERTS IN THIS PARTICULAR AREA TO ADDRESS THIS.

?Any suggestions on foreign keys and indexes?

NOTHING MUCH THAT I CAN THINK OF NOW, FEW THINGS COULD BE, KEEP MAIN TABLE AND RELATED TABLE INSERTS/UPDATES IN ONE COMMIT/ROLLBACK BLOCK. TAKE CARE OF IDENTITY COLUMNS IN CASE OF ROLLBACK/FAILURE, WE'LL LOSE THE IDENTITY NUMBER IF ROLLBACK/FAILURE HAPPENS. INSERT/UPDATE THE RECORDS IN BULK LIKE THOUSANDS IN CHUNKS IN PREFERABLY TARGET TABLE PRIMARY INDEX ORDER. IN TARGET TABLES, BE STRICT IN INCLUDING INDEXES, TRY COMPOSITE INDEXES INSTEAD OF MULTIPLE INDIVIDUAL INDEXES.